OpenStack Shared File Systems Service (Manila)

EMC VNX manila driver: certificate verify failed

Bug #1669202 reported by Sam Wan on 2017-03-02

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Shared File Systems Service (Manila)	Fix Released	Undecided	Peter Wang

Bug Description

EMC VNX manila driver failed to initialize because 'certificate verify failed'

=====
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager [req-70ead0eb-1907-471f-b435-98b905e353bc - - - - -] Error encountered during initialization of driver 'EMCShareDriver' on 'vnxmanila2@london' host.
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager Traceback (most recent call last):
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/manager.py", line 276, in init_host
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager self.driver.do_setup(ctxt)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/dell_emc/driver.py", line 121, in do_setup
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager self.plugin.connect(self, context)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/dell_emc/plugins/vnx/utils.py", line 50, in inner
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager ret = func(self, *args, **kwargs)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/dell_emc/plugins/vnx/connection.py", line 567, in connect
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager self.manager = manager.StorageObjectManager(config)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/dell_emc/plugins/vnx/object_manager.py", line 44, in __init__
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager self.connectors['XML'] = connector.XMLAPIConnector(configuration)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/dell_emc/plugins/vnx/connector.py", line 50, in __init__
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager self._do_setup()
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/dell_emc/plugins/vnx/connector.py", line 58, in _do_setup
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager resp = self.url_opener.open(req)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/usr/lib/python2.7/urllib2.py", line 429, in open
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager response = self._open(req, data)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/usr/lib/python2.7/urllib2.py", line 447, in _open
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager '_open', req)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager result = func(*args)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager context=self._context)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager raise URLError(err)
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
2017-03-01 07:03:09.219 28938 ERROR manila.share.manager
2017-03-01 07:04:36.231 28938 DEBUG oslo_service.periodic_task [req-9f5300e3-0a33-4a60-ab59-6585ae982fd2 - - - - -] Running periodic task ShareManager._publish_service_capabilities run_periodic_tasks /usr/local/lib/python2.7/dist-packages/oslo_service/periodic_task.py:215
2017-03-01 07:04:36.232 28938 DEBUG oslo_service.periodic_task [req-9f5300e3-0a33-4a60-ab59-6585ae982fd2 - - - - -] Running periodic task ShareManager.migration_driver_continue run_periodic_tasks /usr/local/lib/python2.7/dist-packages/oslo_service/periodic_task.py:215
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task [req-9f5300e3-0a33-4a60-ab59-6585ae982fd2 - - - - -] Error during ShareManager.migration_driver_continue
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task Traceback (most recent call last):
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task File "/usr/local/lib/python2.7/dist-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task task(self, context)
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task File "/opt/stack/new/manila/manila/utils.py", line 518, in wrapper
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task raise exception.DriverNotInitialized(driver=driver_name)
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task DriverNotInitialized: Share driver 'EMCShareDriver' not initialized.
2017-03-01 07:04:36.232 28938 ERROR oslo_service.periodic_task
=====

detailed logs can be found at http://publiclogs.emc.com/vnx_ostack/EMC_VNX_Manila/5/logs/screen-m-shr.txt.gz

Tags:

Peter Wang (peter.wang) on 2017-03-02

Changed in manila:
status:	New → Confirmed

Peter Wang (peter.wang) on 2017-03-03

Changed in manila:
assignee:	nobody → Peter Wang (peter.wang)

OpenStack Infra (hudson-openstack) on 2017-03-10

Changed in manila:
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-16: Fix merged to manila (master)

Reviewed: https://review.openstack.org/440364
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=8b6868aaaec9058ef01f42cf4c3dc9825efa79b2
Submitter: Jenkins
Branch: master

commit 8b6868aaaec9058ef01f42cf4c3dc9825efa79b2
Author: Peter Wang <email address hidden>
Date: Thu Mar 2 16:39:26 2017 +0800

Handle ssl for VNX manila driver

    From python 2.7.9 on, the ssl verification is enabled by default,
    This commit adds 2 options for Manila, so user is able to control the
    ssl verification.

The 2 options are ported from cinder/driver.py, these can be used by any
vendor driver which needs to handle ssl verification.

    Closes-bug: 1669202
    Implements: blueprint add-ssl-verification-options
    Change-Id: Ia9a488cab9d4d2d25c5ab534dbf4d61e930cfd7f

Changed in manila:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-17: Fix proposed to manila (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/446798

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-24: Change abandoned on manila (stable/ocata)

Change abandoned by Peter Wang (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/446798
Reason: Move options to driver now

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-28: Fix merged to manila (master)

Reviewed: https://review.openstack.org/449408
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=bf0f42858b0f57aaf08630cda3514c822b21db09
Submitter: Jenkins
Branch: master

commit bf0f42858b0f57aaf08630cda3514c822b21db09
Author: Peter Wang <email address hidden>
Date: Fri Mar 24 10:10:05 2017 +0800

Handle SSL from VNX driver

    From python 2.7.9 on, the ssl verification is enabled by default,
    This commit adds 2 options for VNX Manila driver, so user is able
    to control the ssl verification.

    Closes-bug: 1669202
    Implements: blueprint vnx-ssl-verification-options
    Change-Id: Iedad15e077c0537e133f81e6d537798482789160

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-29: Fix proposed to manila (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/451130

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-04-13: Fix included in openstack/manila 5.0.0.0b1

This issue was fixed in the openstack/manila 5.0.0.0b1 development milestone.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-05-25: Fix merged to manila (stable/ocata)

Reviewed: https://review.openstack.org/451130
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=b3f7f12fcf11aa07cb1b4e97bc11fdb248ba2233
Submitter: Jenkins
Branch: stable/ocata

commit b3f7f12fcf11aa07cb1b4e97bc11fdb248ba2233
Author: Peter Wang <email address hidden>
Date: Fri Mar 24 10:10:05 2017 +0800

Handle SSL from VNX driver

    From python 2.7.9 on, the ssl verification is enabled by default,
    This commit adds 2 options for VNX Manila driver, so user is able
    to control the ssl verification.

    Closes-bug: 1669202
    Implements: blueprint vnx-ssl-verification-options
    Change-Id: Iedad15e077c0537e133f81e6d537798482789160
    (cherry picked from commit bf0f42858b0f57aaf08630cda3514c822b21db09)