Ubuntu
nova package

nova-api reporting missing images after upgrade

Bug #932991 reported by Andrew Glen-Young
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
New
Undecided
Unassigned
nova (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

After upgrading Openstack from the packaged version in Oneiric to the packaged version in Precise I have found that some images are being reported as missing, but not all.

I am using nova-api, glance and swift with deprecated_auth.

Please let me know if you need any further information?

Software versions:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu precise (development branch)"

$ dpkg-query --show nova-*
nova-api 2012.1~e4~20120210.12574-0ubuntu1
nova-common 2012.1~e4~20120210.12574-0ubuntu1
nova-doc 2012.1~e4~20120210.12574-0ubuntu1
nova-network 2012.1~e4~20120210.12574-0ubuntu1
nova-objectstore 2012.1~e4~20120210.12574-0ubuntu1
nova-scheduler 2012.1~e4~20120210.12574-0ubuntu1

Swift reports the image exists:

$ swift -A https://localhost/auth/v1.0 -U glance:glance -K $KEY stat glance $((0xd1))
       Account: AUTH_glance
     Container: glance
        Object: 209
  Content Type: text/plain
Content Length: 13
 Last Modified: Wed, 12 Oct 2011 19:41:42 GMT
          ETag: b9886512afafc0d1598bc7874bccf4b1
 Accept-Ranges: bytes
          Vary: Accept-Encoding

glance db shows that it exists:

sqlite> select * from images where location like '%/209';
c9d4eed5-2ecc-4fa5-9ae1-0da4a0f0c066||1476395008|active|1|swift+https://glance:glance:XXXXXXXXXX@127.0.0.1/auth/v1.0//glance/209|2011-10-12 19:41:25.082058|2011-10-12 19:49:06.521850||0|ami|ami|b9886512afafc0d1598bc7874bccf4b1||0|0|0

glance client:

$ glance show c9d4eed5-2ecc-4fa5-9ae1-0da4a0f0c066
URI: http://0.0.0.0:9292/v1/images/c9d4eed5-2ecc-4fa5-9ae1-0da4a0f0c066
Id: c9d4eed5-2ecc-4fa5-9ae1-0da4a0f0c066
Public: Yes
Protected: No
Name: None
Status: active
Size: 1476395008
Disk format: ami
Container format: ami
Minimum Ram Required (MB): 0
Minimum Disk Required (GB): 0
Property 'image_location': xxxxxx-cloud-images/ubuntu-lucid-10.04-amd64-server-20110930.man
Property 'image_state': available
Property 'kernel_id': 158
Property 'min_ram': 0
Property 'ramdisk_id': 159
Property 'owner': None
Property 'architecture': x86_64
Property 'min_disk': 0
Property 'project_id': xxxxxx_project

euca2ools reports the image doesn't exist:

$ euca-describe-images ami-000000d1
ImageNotFound: Image ami-000000d1 could not be found.

Revision history for this message
Andrew Glen-Young (aglenyoung) wrote : Re: [Bug 932991] [NEW] nova-api reporting missing images after upgrade

On Wed 15-02-2012 19:02 ›, Andrew Glen-Young wrote:
>
> glance client:
>
> $ glance show c9d4eed5-2ecc-4fa5-9ae1-0da4a0f0c066
> URI: http://0.0.0.0:9292/v1/images/c9d4eed5-2ecc-4fa5-9ae1-0da4a0f0c066
> Id: c9d4eed5-2ecc-4fa5-9ae1-0da4a0f0c066
> Public: Yes
> Protected: No
> Name: None
> Status: active
> Size: 1476395008
> Disk format: ami
> Container format: ami
> Minimum Ram Required (MB): 0
> Minimum Disk Required (GB): 0
> Property 'image_location': xxxxxx-cloud-images/ubuntu-lucid-10.04-amd64-server-20110930.man
> Property 'image_state': available
> Property 'kernel_id': 158
> Property 'min_ram': 0
> Property 'ramdisk_id': 159
> Property 'owner': None
> Property 'architecture': x86_64
> Property 'min_disk': 0
> Property 'project_id': xxxxxx_project
>

A quick update.

It looks like the 'kernel_id' and 'ramdisk_id' properties for the image were
not updated to use UUIDs. Once we updated these IDs to the correct UUIDs for
the relevant images, euca-describe-images seems to work.

This workaround is non-optimal for someone upgrading their Openstack
cluster.

--
Regards.
Andrew Glen-Young

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in nova (Ubuntu):
status: New → Confirmed
James Troup (elmo)
tags: added: canonistack
Adam Gandelman (gandelman-a)
tags: added: ubuntu-openstack-upgrade
Revision history for this message
Adam Gandelman (gandelman-a) wrote :

@Andrew-

To be clear, you had to 'update kernel_id' and 'ramdisk_id' columns of the image_properties table in Glance's database, not on any table in Nova's database? I believe I'm able to reproduce this, but it looks like a glance bug, not nova.

Revision history for this message
Adam Gandelman (gandelman-a) wrote :

I've opened Bug #975651 in Glance to track this there.

I should note that this doesn't affect the functionality of euca-describe-instances, but now seems to manifest itself during instance launch/euca-run-instances. Perhaps some work has been done around EC2 -> Glance UUID translation in the months since this bug was opened.

Revision history for this message
Andrew Glen-Young (aglenyoung) wrote :

@Adam:

I definitely had to update the Glance database and update the image_properties table. My memory is a little fuzzy on the exact details, but I believe that the nova.s3_images table did not have the correct UUID for ramdisks and kernels as well. I only noticed this much later however and I am unsure if it is related.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.