deploy of manila with external ceph fails to update ceph.conf

Maybe we can add some validations but Ceph won't get configured unless the user gives instructions to do so.

This is done by deploying using -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-ceph-external.yaml

In addition to that, there are at least three additional parameters the user needs to provide (from a custom environment file) with the external Ceph cluster details:

parameter_defaults:
  # NOTE: These example parameters are required when using CephExternal
  #CephClusterFSID: '4b5c8c0a-ff60-454b-a1b4-9747aa737d19'
  #CephClientKey: 'AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ=='
  #CephExternalMonHost: '172.16.1.7, 172.16.1.8'

The deployer is expected to create on the external Ceph cluster a client.openstack (by default, but name can be customized) user, with appropriate privileges on the glance/cinder/nova pools, unless rbd is disabled for all these services (see puppet-ceph-external.yaml)

Finally, the deployer is expected to create on the external Ceph cluster a client.manila (again this is the default, can be customized in manila-cephfsnative-config.yaml) with appropriate privileges for Manila.

There are some instructions on how to configure the OpenStack deployment with an external Ceph cluster at http://tripleo.org/advanced_deployment/ceph_external.html

That needs to be done in addition to the inclusion of manila-cephfsnative-config.yaml and for Manila to work, the client.manila user needs to be created on the external Ceph cluster, or Manila needs to be configured to use client.openstack if the Ceph admin gave to that user the needed privileges.

Hi Guilio,
supposing that client.manila user is created on the external ceph server, the openstack node where manila-share is running should have following configuration snippet in /etc/ceph/ceph.conf:

[client.manila]
keyring = /etc/ceph/ceph.client.manila.keyring

So that manila ceph driver knows what keyring to use when connecting to the ceph server. This is different in compare to e.g. cinder/gnocchi where ceph keyring path is set directly in
the /etc/cinder/cinder.conf.

I don't have OC deployed with ceph-external templates so it might be that I'm wrong and the required config snippet is generated by ceph-external configuration (though "ManilaCephFSNativeCephFSAuthId" which sets manila ceph user is passed to manila puppet module only which does not update ceph.conf).

I'm also afraid that the default client.openstack is not sufficient ( maybe because it conflicts with cinder/gnocchi clients), if I use "openstack" user for manila driver, it fails with:

2016-12-01 09:22:02.788 232775 INFO manila.share.drivers.cephfs.cephfs_native [req-de3e597d-25c3-4be1-9394-868bdeedda7f - - - - -] [cephfs}] Ceph client found, connecting...
2016-12-01 09:22:02.817 232775 INFO ceph_volume_client [req-de3e597d-25c3-4be1-9394-868bdeedda7f - - - - -] evict clients with auth_name=openstack
2016-12-01 09:22:02.825 232775 INFO ceph_volume_client [req-de3e597d-25c3-4be1-9394-868bdeedda7f - - - - -] evict: joined all
2016-12-01 09:22:02.825 232775 ERROR ceph_volume_client [req-de3e597d-25c3-4be1-9394-868bdeedda7f - - - - -] Failed to evict client with auth_name=openstack from mds 0/14104: Error -1 ("") w
hile Sending evict to mds.14104
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager [req-de3e597d-25c3-4be1-9394-868bdeedda7f - - - - -] Error encountered during initialization of driver 'CephFSNativeDriver' on 'over
cloud-controller-0.localdomain@cephfsnative' host. Failed to evict client with auth_name=openstack from mds 0/14104: Error -1 ("") while Sending evict to mds.14104
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager Traceback (most recent call last):
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/manila/share/manager.py", line 269, in init_host
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager self.driver.check_for_setup_error()
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 88, in check_for_setup_error
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager self.volume_client
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/manila/share/drivers/cephfs/cephfs_native.py", line 156, in volume_client
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager self._volume_client.connect(premount_evict=premount_evict)
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager File "/usr/lib/python2.7/site-packages/ceph_volume_client.py", line 467, in connect
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager self.evict(premount_evict)
2016-12-01 09:22:02.826 232775 ERROR manila.share.manager File "/usr/lib/p...

Thanks Jan, I understand better the issue now. Would be nice if we could set a path for the keyring file though for Manila as well, so we won't need to change ceph.conf from the Manila profile.

Fix proposed to branch: master
Review: https://review.openstack.org/405999

Reviewed: https://review.openstack.org/405999
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=2dcc387fa67c6ee705c1bf5a9f79afe5cd818d4a
Submitter: Jenkins
Branch: master

commit 2dcc387fa67c6ee705c1bf5a9f79afe5cd818d4a
Author: Jan Provaznik <email address hidden>
Date: Wed Dec 14 12:04:17 2016 +0000

    Set ceph key when using manila ceph backend

    Manila ceph driver reads ceph's client configuration
    (keyring is the most important) from ceph.conf file
    (or any other file set by cephfs_conf_path). ceph.conf
    should be updated with keyring location.

    If ceph is deployed by tripleo then also manila ceph key
    is added into ceph and ceph filesystem is created.

    Depends-On: I18436a64fc991b9e697a1d79e369ac110cf8fe20
    Change-Id: Iac4a260af6738ed6afd4bcb107221a736d07c1b5
    Partial-Bug: #1644784
    Closes-Bug: #1646147

This issue was fixed in the openstack/puppet-tripleo 6.2.0 release.