OpenStack Compute (nova)

Volume auto attach can fail if there is a swap or ephemeral device

Bug #1055712 reported by Chuck Thier on 2012-09-24

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	High	Vish Ishaya	OpenStack Compute (nova) 2012.2 "folsom"

Bug Description

The following behavior has been reported:

Images are set up so that the root partition is on the /dev/xvda device, and swap is on /dev/xvdc:

root@singularity:~# ll /dev/xv*
brw-rw---- 1 root disk 202, 0 Sep 6 20:03 /dev/xvda
brw-rw---- 1 root disk 202, 1 Sep 6 20:03 /dev/xvda1
brw-rw---- 1 root disk 202, 32 Sep 6 20:03 /dev/xvdc
brw-rw---- 1 root disk 202, 33 Sep 6 20:03 /dev/xvdc1

Attach a volume:

A few moments later, I see this:

root@singularity:~# ll /dev/xv*
brw-rw---- 1 root disk 202, 0 Sep 6 20:03 /dev/xvda
brw-rw---- 1 root disk 202, 1 Sep 6 20:03 /dev/xvda1
brw-rw---- 1 root disk 202, 16 Sep 21 15:46 /dev/xvdb <=== new volume
brw-rw---- 1 root disk 202, 32 Sep 6 20:03 /dev/xvdc
brw-rw---- 1 root disk 202, 33 Sep 6 20:03 /dev/xvdc1

Cool beans. Now to attach another…..

The volume fails to attach, and the following error is in the logs (Ths is Xenserver):

Unable to use SR OpaqueRef:a2d2dea8-eab2-7845-3cf2-969daa9da5a2 for instance instance-38d17e58-ec60-41ca-84ba-e5ea48c9e1ef

It would appear that it is trying to auto-attach to /dev/xvdc even though it is already in use, and just to verify:

Moments later, it showed up on the guest as expected:

root@singularity:~# ll /dev/xv*
brw-rw---- 1 root disk 202, 0 Sep 6 20:03 /dev/xvda
brw-rw---- 1 root disk 202, 1 Sep 6 20:03 /dev/xvda1
brw-rw---- 1 root disk 202, 16 Sep 21 15:46 /dev/xvdb
brw-rw---- 1 root disk 202, 32 Sep 6 20:03 /dev/xvdc
brw-rw---- 1 root disk 202, 33 Sep 6 20:03 /dev/xvdc1
brw-rw---- 1 root disk 202, 48 Sep 21 16:07 /dev/xvdd <=== new volume

Hoping that I have now forced it to start counting from the correct point, I tried to auto-attach another volume:

Nope, generated another fault, with the same error.

It seems that the "auto" method of selecting a guest block device needs to be a little smarter, and take into consideration what the default volumes are called.

Vish Ishaya (vishvananda) on 2012-09-25

Changed in nova:
status:	New → Triaged
importance:	Undecided → High

Vish Ishaya (vishvananda) on 2012-09-25

summary:	- Volume auto attach fails if other devices are already specified + Volume auto attach can fail if there is a swap device
summary:	- Volume auto attach can fail if there is a swap device + Volume auto attach can fail if there is a swap or ephemeral device

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/13649

Changed in nova:
assignee:	nobody → Vish Ishaya (vishvananda)
status:	Triaged → In Progress

Vish Ishaya (vishvananda) on 2012-09-25

tags:

added: folsom-rc-potential

Vish Ishaya (vishvananda) on 2012-09-25

Changed in nova:
milestone:	none → folsom-rc3
tags:	removed: folsom-rc-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix merged to nova (master)

Reviewed: https://review.openstack.org/13649
Committed: http://github.com/openstack/nova/commit/69564763960cddc249138469811a0a771db16e19
Submitter: Jenkins
Branch: master

commit 69564763960cddc249138469811a0a771db16e19
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Sep 25 11:15:14 2012 -0700

Fix issues with device autoassignment in xenapi

    This is a workaround for two issues in xenapi. The first is that
    does not set the instance default_root_device to /dev/xvda so it
    defaults to /dev/sda. The proper fix for this involves setting the
    default_root_device in xenapi and a db migration to set the proper
    default_root_device for existing instances.

This patch works around this issue by explicitly setting the prefix
to /dev/xvd if the compute driver is xenapi.

    The second issue is that xenapi never updates the instance record
    to include default_swap_device and default_ephemeral device. The
    fix for this involes adding the appropriate update to the instance
    record and a migration that sets the proper values for all existing
    instances.

    This patch works around this issue by explicily checking the
    instance_type and removing the devices from the list if the compute
    driver is xenapi.

Fixes bug 1055715 and bug 1055712

Change-Id: I61aa15e69eb0a22430bb22ea5149b1f0735b3328

Changed in nova:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix proposed to nova (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/13674

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-26: Fix merged to nova (milestone-proposed)

Reviewed: https://review.openstack.org/13674
Committed: http://github.com/openstack/nova/commit/076cb9d98330ae1ead94cc4d2c08b27939752648
Submitter: Jenkins
Branch: milestone-proposed

commit 076cb9d98330ae1ead94cc4d2c08b27939752648
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Sep 25 11:15:14 2012 -0700

Fix issues with device autoassignment in xenapi

This patch works around this issue by explicitly setting the prefix
to /dev/xvd if the compute driver is xenapi.

    This patch works around this issue by explicily checking the
    instance_type and removing the devices from the list if the compute
    driver is xenapi.

Fixes bug 1055715 and bug 1055712

Change-Id: I61aa15e69eb0a22430bb22ea5149b1f0735b3328
(cherry picked from commit 69564763960cddc249138469811a0a771db16e19)