OpenStack Compute (nova)

Xenserver: If a VDI is missing, an active instance cannot be deleted

Bug #1030143 reported by Kevin L. Mitchell
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Renuka Apte
OpenStack Compute (nova) 2012.2 "folsom"

Bug Description

When deleting an active instance, the following traceback was observed:

    2012-07-27 19:17:12 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/openstack/common/rpc/amqp.py", line 275, in _process_data
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/exception.py", line 117, in wrapped
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp temp_level, payload)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/contextlib.py", line 23, in __exit__
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp self.gen.next()
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/exception.py", line 92, in wrapped
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 187, in decorated_function
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp function(self, context, instance_uuid, *args, **kwargs)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 211, in decorated_function
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp sys.exc_info())
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/contextlib.py", line 23, in __exit__
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp self.gen.next()
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 205, in decorated_function
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp return function(self, context, instance_uuid, *args, **kwargs)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 914, in terminate_instance
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp do_terminate_instance()
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/utils.py", line 680, in inner
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp retval = f(*args, **kwargs)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 907, in do_terminate_instance
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp self._delete_instance(context, instance)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 884, in _delete_instance
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp self._shutdown_instance(context, instance)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 849, in _shutdown_instance
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp block_device_info)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/driver.py", line 218, in destroy
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp self._vmops.destroy(instance, network_info, block_device_info)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/vmops.py", line 1005, in destroy
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp block_device_info=block_device_info)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/vmops.py", line 1025, in _destroy
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp self._destroy_vdis(instance, vm_ref, block_device_info)
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/vmops.py", line 923, in _destroy_vdis
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp nodestroy.append(bdm['connection_info']['data']['vdi_uuid'])
    2012-07-27 19:17:12 TRACE nova.openstack.common.rpc.amqp KeyError: 'vdi_uuid'

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/10436

Changed in nova:
assignee: nobody → Kevin L. Mitchell (klmitch)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/10436
Committed: http://github.com/openstack/nova/commit/3f315000755732c4780dae03ef65f67d3f87e8f7
Submitter: Jenkins
Branch: master

commit 3f315000755732c4780dae03ef65f67d3f87e8f7
Author: Kevin L. Mitchell <email address hidden>
Date: Fri Jul 27 16:26:24 2012 -0500

    Allow _destroy_vdis if a mapping has no VDI

    In _destroy_vdis() in the xenapi vmops.py, a list of VDIs to not
    destroy is collected. If one of the block device mappings does
    not have a corresponding VDI, however, _destroy_vdis() fails
    (KeyError('vdi_uuid') is raised). This interferes with
    instance deletion. The problem is corrected by skipping mappings
    for which the connection_info data does not contain 'vdi_uuid'.

    Fixes bug 1030143.

    Change-Id: I5125d557ab799fcf572f665523cd07a893ad3b90

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/10574

Changed in nova:
assignee: Kevin L. Mitchell (klmitch) → Renuka Apte (renukaapte)
status: Fix Committed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/10574
Committed: http://github.com/openstack/nova/commit/cf8b2a2026cf8606de0f9474bd2e1563d662b4ab
Submitter: Jenkins
Branch: master

commit cf8b2a2026cf8606de0f9474bd2e1563d662b4ab
Author: Renuka Apte <email address hidden>
Date: Mon Jul 30 14:56:24 2012 -0700

    xenapi: Tag nova volumes during attach_volume

    1. Set other_config:'osvol':'True' on VBD during attach.
    2. Use this (instead of nodestroys list) to determine which volumes
    should not be destroyed on instance terminate.
    3. Ensure that the SRs for the attached volumes are forgotten when
    instance is terminated.

    In the virt/xenapi layer, there is no way to determine which of the
    VDIs are nova volumes. This information is required when terminating
    and instance, to ensure the volumes are kept intact, as well as other
    operations like migrate.

    Fixes bug 1030143.

    Change-Id: Id1717e64bc29092ce9ffe13b7c254a3867fe8342

Changed in nova:
status: In Progress → Fix Committed
John Garbutt (johngarbutt)
Changed in nova:
importance: Undecided → Medium
Thierry Carrez (ttx)
Changed in nova:
milestone: none → folsom-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: folsom-rc1 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.