libvirt: rebuild is not using kernel and ramdisk associated with the new image

Bug #1060925 reported by Sirisha Devineni
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Sirisha Devineni
Folsom
Fix Released
Low
Vish Ishaya
nova (Ubuntu)
Fix Released
Undecided
Unassigned
Quantal
Fix Released
Undecided
Unassigned

Bug Description

Nova rebuild command works like this:

nova rebuild
 usage: nova rebuild [--rebuild-password <rebuild-password>] [--poll]
                    [--minimal]
                    <server> <image>
Currently the image (rootfs) is replaced but not the kernel (and ramdisk if applicable).
The objective is to make libvirt use the correct kernel and ramdisk associated with the image in glance.

Steps to reproduce:
1. Rebuild a good instance to a new image.
 2. Check that the console output has received the keys (like a normal nova boot).
 3. Use the credentials of the original instance such as the ssh keys and the floating ip and try to ssh into the instance.
 4. Check that the rootfs is correctly rebuilt (cat /etc/issue).
5. Check that the kernel is as expected (uname -r).

I reproduced this issue in devstack installed folsom. Created a Vm with Cirros image and after that tried to rebuild the instance using ami-tty image.
Then the output is as follows:
After first boot with Cirros image:
$ cat /etc/issue
login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root.
$uname -r
3.0.0-12-virtual

After rebuilt with ami-tty image:
# bash
bash-4.1# uname -r
3.0.0-12-virtual
bash-4.1# cat /etc/issue

ttylinux ver 12.1 [atheling]
x86_64 class \s kernel \r (\l)
The initial "root" and "user" password is "password".

The database entries are as follows:
image_ref is pointing to new image ID and kernel and ramdisk are pointing to old ones as follows:
mysql> select id, image_ref, kernel_id, ramdisk_id, server_name, vm_state from instances;
+----+--------------------------------------+--------------------------------------+--------------------------------------+-------------+----------+
| id | image_ref | kernel_id | ramdisk_id | server_name | vm_state |
+----+--------------------------------------+--------------------------------------+--------------------------------------+-------------+----------+
| 1 | c4dd8ed5-9488-44f8-98c3-34bcf6cdd52d | b438e614-630e-4bf7-b907-4e4165b32408 | 04a1326f-0891-4b10-8861-61df279d9b40 | NULL | active |
+----+--------------------------------------+--------------------------------------+--------------------------------------+-------------+----------+
1 row in set (0.00 sec)

mysql> Bye
root@ubuntu28folsom:~# nova image-list
+--------------------------------------+---------------------------------+--------+--------+
| ID | Name | Status | Server |
+--------------------------------------+---------------------------------+--------+--------+
| 583282f0-3f79-49ff-9886-97c2a1a3cd3c | cirros-0.3.0-x86_64-uec | ACTIVE | |
| b438e614-630e-4bf7-b907-4e4165b32408 | cirros-0.3.0-x86_64-uec-kernel | ACTIVE | |
| 04a1326f-0891-4b10-8861-61df279d9b40 | cirros-0.3.0-x86_64-uec-ramdisk | ACTIVE | |
| c4dd8ed5-9488-44f8-98c3-34bcf6cdd52d | disk123 | ACTIVE | |
| 5fab96d5-cc2d-4d81-86cc-825885380c12 | kernel | ACTIVE | |
| 5962333b-376e-4d6f-8938-39fff085f7eb | ram | ACTIVE | |
+--------------------------------------+---------------------------------+--------+--------+
root@ubuntu28folsom:~# nova list
+--------------------------------------+--------+--------+------------------+
| ID | Name | Status | Networks |
+--------------------------------------+--------+--------+------------------+
| 80f72970-7131-4426-9a44-7075b290af3e | imgsrv | ACTIVE | private=10.0.0.2 |
+--------------------------------------+--------+--------+------------------+
root@ubuntu28folsom:~# nova show 80f72970-7131-4426-9a44-7075b290af3e
+------------------------+----------------------------------------------------------+
| Property | Value |
+------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2012-10-03T17:32:06Z |
| flavor | m1.tiny (1) |
| hostId | ebf6015593d45f07c808e7868e03b8a14359f41196b8105f66dbd443 |
| id | 80f72970-7131-4426-9a44-7075b290af3e |
| image | disk123 (c4dd8ed5-9488-44f8-98c3-34bcf6cdd52d) |
| key_name | siri |
| metadata | {} |
| name | imgsrv |
| private network | 10.0.0.2 |
| progress | 0 |
| security_groups | [{u'name': u'default'}] |
| status | ACTIVE |
| tenant_id | 81a600396e5c4d07b07772363d1ddf95 |
| updated | 2012-10-03T17:38:36Z |
| user_id | d4418ce0c6524435ba4901fefe394193 |
+------------------------+----------------------------------------------------------+
root@ubuntu28folsom:~#

Changed in nova:
assignee: nobody → Sirisha Devineni (sirisha-devineni)
description: updated
summary: - Nova rebuild is not using kernel and ramdisk associated with the new
+ libvirt: rebuild is not using kernel and ramdisk associated with the new
image
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/14083

Changed in nova:
status: New → In Progress
tags: added: folsom-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/14083
Committed: http://github.com/openstack/nova/commit/c781e90929a6b3c39c0528a864cf560a5a8999e2
Submitter: Jenkins
Branch: master

commit c781e90929a6b3c39c0528a864cf560a5a8999e2
Author: Sirisha Devineni <email address hidden>
Date: Fri Oct 5 21:50:42 2012 +0530

    Update kernel_id and ramdisk_id while rebuilding instance

    While rebuilding an instances if kernel_id and ramdisk_id associated
    with new image_ref then update instances table for instance_id
    with the new kernel and ramdisk.

    Fixes bug 1060925

    Change-Id: I07dfdbed8580411adea14fd6c5be1a9569a7a021

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
Changed in nova:
importance: Undecided → Low
tags: removed: folsom-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/folsom)

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/16683

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/folsom)

Reviewed: https://review.openstack.org/16683
Committed: http://github.com/openstack/nova/commit/8f6a718c653ff9e058c5dad83a0009f3f14bc074
Submitter: Jenkins
Branch: stable/folsom

commit 8f6a718c653ff9e058c5dad83a0009f3f14bc074
Author: Sirisha Devineni <email address hidden>
Date: Fri Oct 5 21:50:42 2012 +0530

    Update kernel_id and ramdisk_id while rebuilding instance

    While rebuilding an instances if kernel_id and ramdisk_id associated
    with new image_ref then update instances table for instance_id
    with the new kernel and ramdisk.

    Fixes bug 1060925

    Change-Id: I07dfdbed8580411adea14fd6c5be1a9569a7a021
    (cherry picked from commit c781e90929a6b3c39c0528a864cf560a5a8999e2)

Changed in nova (Ubuntu):
status: New → Fix Released
Changed in nova (Ubuntu Quantal):
status: New → Confirmed
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello Sirisha, or anyone else affected,

Accepted nova into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/2012.2.1+stable-20121212-a99a802e-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Quantal):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (8.3 KiB)

This bug was fixed in the package nova - 2012.2.1+stable-20121212-a99a802e-0ubuntu1

---------------
nova (2012.2.1+stable-20121212-a99a802e-0ubuntu1) quantal-proposed; urgency=low

  * Ubuntu updates:
    - debian/control: Ensure novaclient is upgraded with nova,
      require python-keystoneclient >= 1:2.9.0. (LP: #1073289)
    - d/p/avoid_setuptools_git_dependency.patch: Refresh.
  * Dropped patches, applied upstream:
    - debian/patches/CVE-2012-5625.patch: [a99a802]
  * Resynchronize with stable/folsom (b55014ca) (LP: #1085255):
    - [a99a802] create_lvm_image allocates dirty blocks (LP: #1070539)
    - [670b388] RPC exchange name defaults to 'openstack' (LP: #1083944)
    - [3ede373] disassociate_floating_ip with multi_host=True fails
      (LP: #1074437)
    - [22d7c3b] libvirt imagecache should handle shared image storage
      (LP: #1075018)
    - [e787786] Detached and deleted RBD volumes remain associated with insance
      (LP: #1083818)
    - [9265eb0] live_migration missing migrate_data parameter in Hyper-V driver
      (LP: #1066513)
    - [3d99848] use_single_default_gateway does not function correctly
      (LP: #1075859)
    - [65a2d0a] resize does not migrate DHCP host information (LP: #1065440)
    - [102c76b] Nova backup image fails (LP: #1065053)
    - [48a3521] Fix config-file overrides for nova-dhcpbridge
    - [69663ee] Cloudpipe in Folsom: no such option: cnt_vpn_clients
      (LP: #1069573)
    - [6e47cc8] DisassociateAddress can cause Internal Server Error
      (LP: #1080406)
    - [22c3d7b] API calls to dis-associate an auto-assigned floating IP should
      return proper warning (LP: #1061499)
    - [bd11d15] libvirt: if exception raised during volume_detach, volume state
      is inconsistent (LP: #1057756)
    - [dcb59c3] admin can't describe all images in ec2 api (LP: #1070138)
    - [78de622] Incorrect Exception raised during Create server when metadata
      over 255 characters (LP: #1004007)
    - [c313de4] Fixed IP isn't released before updating DHCP host file
      (LP: #1078718)
    - [f4ab42d] Enabling Return Reservation ID with XML create server request
      returns no body (LP: #1061124)
    - [3db2a38] 'BackupCreate' should accept rotation parameter greater than or
      equal to zero (LP: #1071168)
    - [f7e5dde] libvirt reboot sometimes fails to reattach volumes
      (LP: #1073720)
    - [ff776d4] libvirt: detaching volume may fail while terminating other
      instances on the same host concurrently (LP: #1060836)
    - [85a8bc2] Used instance uuid rather than id in remove-fixed-ip
    - [42a85c0] Fix error on invalid delete_on_termination value
    - [6a17579] xenapi migrations fail w/ swap (LP: #1064083)
    - [97649b8] attach-time field for volumes is not updated for detach volume
      (LP: #1056122)
    - [8f6a718] libvirt: rebuild is not using kernel and ramdisk associated with
      the new image (LP: #1060925)
    - [fbe835f] live-migration and volume host assignement (LP: #1066887)
    - [c2a9150] typo prevents volume_tmp_dir flag from working (LP: #1071536)
    - [93efa21] Instances deleted during spawn leak network allocations
      (LP: #1068716)
    - [ebabd02] After restartin...

Read more...

Changed in nova (Ubuntu Quantal):
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.