cinderclient service catalog lookup fails

Bug #1048798 reported by clayg
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
clayg

Bug Description

If your auth system isn't adding the service_catalog in the request headers, you get an unhandled exception in cinderclient.

2012-09-10 14:44:55 INFO nova.api.openstack.wsgi [req-3ab60e8c-888d-4d63-b857-1de0db0e0ac8 account1 account1] POST http://localhost:8774/v1.1/account1/servers/5b92fa58-ab9c-4140-b841-64d0c41786c9/os-volume_attachments
2012-09-10 14:44:55 AUDIT nova.api.openstack.compute.contrib.volumes [req-3ab60e8c-888d-4d63-b857-1de0db0e0ac8 account1 account1] Attach volume d7f55869-e888-4e1a-94f3-561725a4def6 to instance 5b92fa58-ab9c-4140-b841-64d0c41786c9 at /dev/vdf
2012-09-10 14:44:55 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous call on compute.nova ... from (pid=29457) multicall /opt/nova/nova/openstack/common/rpc/amqp.py:351
2012-09-10 14:44:55 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is e1918d3455b94332b78b55b5eea4fa10 from (pid=29457) multicall /opt/nova/nova/openstack/common/rpc/amqp.py:354
2012-09-10 14:44:55 ERROR nova.api.openstack.wsgi [req-3ab60e8c-888d-4d63-b857-1de0db0e0ac8 account1 account1] Exception handling resource: 'NoneType' object is not iterable
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi Traceback (most recent call last):
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/api/openstack/wsgi.py", line 925, in _process_stack
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/api/openstack/wsgi.py", line 1013, in dispatch
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi return method(req=request, **action_args)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/api/openstack/compute/contrib/volumes.py", line 392, in create
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi volume_id, device)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/compute/api.py", line 114, in wrapped
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi return func(self, context, target, *args, **kwargs)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/compute/api.py", line 104, in inner
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/compute/api.py", line 1750, in attach_volume
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi context, instance['uuid'], device)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi self.gen.next()
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/compute/api.py", line 1742, in attach_volume
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi volume = self.volume_api.get(context, volume_id)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/volume/cinder.py", line 135, in get
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi item = cinderclient(context).volumes.get(volume_id)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/nova/nova/volume/cinder.py", line 59, in cinderclient
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi endpoint_type=endpoint_type)
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi File "/opt/python-cinderclient/cinderclient/service_catalog.py", line 53, in url_for
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi for service in catalog:
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi TypeError: 'NoneType' object is not iterable
2012-09-10 14:44:55 TRACE nova.api.openstack.wsgi
2012-09-10 14:44:55 INFO nova.api.openstack.wsgi [req-3ab60e8c-888d-4d63-b857-1de0db0e0ac8 account1 account1] http://localhost:8774/v1.1/account1/servers/5b92fa58-ab9c-4140-b841-64d0c41786c9/os-volume_attachments returned with HTTP 400
2012-09-10 14:44:55 INFO nova.osapi_compute.wsgi.server [req-3ab60e8c-888d-4d63-b857-1de0db0e0ac8 account1 account1] 127.0.0.1 - - [10/Sep/2012 14:44:55] "POST /v1.1/account1/servers/5b92fa58-ab9c-4140-b841-64d0c41786c9/os-volume_attachments HTTP/1.1" 400 338 0.274771

I'd actually prefer to be able to set the cinder endpoint via a flag for two reasons.

1) if you're using a reverse proxy, or another auth system, it may not be convenient or even make sense to add a copy of the service catalog to the request headers/context - the KeystoneContext middleware certainly seems to support the idea that it may not always be there. Plus, glance uses flags afaict, and that works ok.
2) There may be good reasons for the management endpoint that nova uses to talk to cinder to be different than the public endpoint, so the users catalog may not really help. (e.g. public endpoint has os-volume_actions ext disabled to prevent user calling detach directly without updating nova)

The attached patch also happens to support noauth just cause it was "right there"

Tags: cinder
Revision history for this message
clayg (clay-gerrard) wrote :
Changed in nova:
milestone: none → folsom-rc1
assignee: nobody → clayg (clay-gerrard)
importance: Undecided → High
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/12748

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/12748
Committed: http://github.com/openstack/nova/commit/0fa231f718662f2b83c11ffc39f977fdc8941107
Submitter: Jenkins
Branch: master

commit 0fa231f718662f2b83c11ffc39f977fdc8941107
Author: Clay Gerrard <email address hidden>
Date: Mon Sep 10 19:05:10 2012 -0500

    Add flag cinder_endpoint_template to volume.cinder

    Add optional flag to allow cinder endpoint to be defined by flag rather
    than extracted from the request context's service catalog. No change to
    default behavior.

    This flag allows deployers to use a seperate management endpoint for
    cinder which compute can use exclusively to send messages to cinder, or
    just use an auth system that doesn't add/forward the catalog along with
    the request (e.g. noauth).

    Add python-cinderclient to tools/test-requires

    fix lp bug #1048798

    Change-Id: Icb416bf4df2a6e37024f1fbc866006d46d30bcf2

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: folsom-rc1 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.