OpenStack Compute (nova)

meta data of nova instance with case sensitive has problem when deleting

Bug #1535224 reported by hgangwx
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
Medium
Rajesh Tailor

Bug Description

[Summary]
meta data of nova instance with case sensitive has problem when deleting

[Topo]
devstack all-in-one node

[Description and expect result]
can delete case sensitive meta one by one

[Reproduceable or not]
reproduceable

[Recreate Steps]
1) set 4 metas for an instance, if ignore case, they have same string:
stack@45-59:~/devstack$ nova meta inst set abc=1 Abc=2 ABc=3 ABC=4
stack@45-59:~/devstack$ nova show inst
+--------------------------------------+----------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | 45-59 |
| OS-EXT-SRV-ATTR:hostname | inst |
| OS-EXT-SRV-ATTR:hypervisor_hostname | 45-59 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-SRV-ATTR:kernel_id | 13716aa6-6110-4b26-9eb4-71f053c14b3d |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | 4c1870c0-520c-4423-8a6b-72e1839eeb76 |
| OS-EXT-SRV-ATTR:reservation_id | r-wc29hykh |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-01-18T15:34:42.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | True |
| created | 2016-01-18T15:34:31Z |
| flavor | m1.tiny (1) |
| hostId | 8b5dd2a2c67716f40b65b8796dcb25c62f7522764e2dc7f9ff88ea7b |
| id | c2f19b32-cf99-4f6f-b9aa-fcf7246ff189 |
| image | cirros-0.3.4-x86_64-uec (03ada996-9688-471f-ab8c-a9cc18175e2e) |
| key_name | - |
| locked | False |
| metadata | {"abc": "1", "Abc": "2", "ABC": "3", "ABc": "4"} |
| name | inst |
| net1 network | 1.0.0.3 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | ACTIVE |
| tenant_id | e88840d981e2457595baed077ed2aac8 |
| updated | 2016-01-18T15:43:48Z |
| user_id | 9d619f44512a4289a670332300f1089c |
+--------------------------------------+----------------------------------------------------------------+

2) if delete one of them, all are deleted: >>>>>>>>ISSUE
stack@45-59:~/devstack$ nova meta inst delete abc
stack@45-59:~/devstack$ nova show inst
+--------------------------------------+----------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | 45-59 |
| OS-EXT-SRV-ATTR:hostname | inst |
| OS-EXT-SRV-ATTR:hypervisor_hostname | 45-59 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-SRV-ATTR:kernel_id | 13716aa6-6110-4b26-9eb4-71f053c14b3d |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | 4c1870c0-520c-4423-8a6b-72e1839eeb76 |
| OS-EXT-SRV-ATTR:reservation_id | r-wc29hykh |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-01-18T15:34:42.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | True |
| created | 2016-01-18T15:34:31Z |
| flavor | m1.tiny (1) |
| hostId | 8b5dd2a2c67716f40b65b8796dcb25c62f7522764e2dc7f9ff88ea7b |
| id | c2f19b32-cf99-4f6f-b9aa-fcf7246ff189 |
| image | cirros-0.3.4-x86_64-uec (03ada996-9688-471f-ab8c-a9cc18175e2e) |
| key_name | - |
| locked | False |
| metadata | {} |
| name | inst |
| net1 network | 1.0.0.3 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | ACTIVE |
| tenant_id | e88840d981e2457595baed077ed2aac8 |
| updated | 2016-01-18T15:43:48Z |
| user_id | 9d619f44512a4289a670332300f1089c |
+--------------------------------------+----------------------------------------------------------------+
stack@45-59:~/devstack$

[Configration]
reproduceable bug, no need

[logs]
reproduceable bug, no need

[Root cause anlyze or debug inf]
reproduceable bug

[Attachment]
None

Tags: db
Pallavi (p-pallavi)
Changed in nova:
assignee: nobody → Pallavi (p-pallavi)
Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

Bug skimming result
===================
I could reproduce it on master (mitaka) with a devstack single node.
=> CONFIRMED

Nova version
------------
    $ git log --oneline -1
    8ffca40 Imported Translations from Zanata

Steps to reproduce
------------------
   $ nova boot bug1535224 --image cirros-0.3.4-x86_64-uec --flavor m1.tiny
   $ nova meta bug1535224 set abc=1 Abc=2 ABc=3 ABC=4 foo=bar
   $ nova meta bug1535224_2 delete abc

Preanalysis
-----------
The REST API gets the correct info to delete the metadata:
    [...] INFO nova.osapi_compute.wsgi.server [...]
    192.168.122.214 "DELETE
    /v2.1/2cec396e517f4fddb2837eadd824f866/servers/
    faa394f5-7890-44ae-a729-4823106cc7df/metadata/abc HTTP/1.1"
    status: 204 len: 274 time: 0.2725191

The controllers below use case sensitive dicts as data structures.

Therefore I assume the issue on the database layer.

If you have questions, I'm available in the IRC channel #openstack-nova
under the name "markus_z".

Changed in nova:
status: New → Confirmed
tags: added: db
Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

Correction to #1:

> $ nova meta bug1535224_2 delete abc

should be:

$ nova meta bug1535224 delete abc

Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote :

very similar to this one is bug 1538011

Revision history for this message
Sheel Rana (ranasheel2000) wrote :

Dear Pallavi,

Could I take this bug ahead?

I am proposing fix for this bug, in case you want to work on this bug, feel free to -1 my patch.
Thank you!!

Revision history for this message
Sheel Rana (ranasheel2000) wrote :
Download full text (6.1 KiB)

This issue is fixed by changing way of deleting metadata entry form database.
Currently it uses key to delete database entry but as key can be duplicate.
For ex, abc and ABC.

So, I am implementing id instead of key to fix this issue as id will always be unique.

Refer below for use case run after fix:

[root@localhost sqlalchemy(keystone_admin)]# nova show c943dfed-8572-4591-a0fd-58e87bd8c92e
+--------------------------------------+----------------------------------------------------------+
| Property | Value |
 +--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | compute2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute2 |
| OS-EXT-SRV-ATTR:instance_name | instance-0000000c |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-02-26T09:50:16.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2016-02-26T09:04:10Z |
| flavor | m1.tiny (1) |
| hostId | 290549584beced55981bf511efdd80645db00691d2f03078c45af029 |
| id | c943dfed-8572-4591-a0fd-58e87bd8c92e |
| image | cirros (26923212-117c-4abb-9364-97e6535d99c6) |
| key_name | - |
| metadata | {"abc": "123", "ABC": "123"} |
| mynet network | 192.168.56.6 |
| name | test10 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
|...

Read more...

Changed in nova:
assignee: Pallavi (p-pallavi) → Sheel Rana (ranasheel2000)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/285222

Revision history for this message
Augustina Ragwitz (auggy) wrote :

I've -1'ed this proposed fix because this is part of a larger issue of how to handle metadata keys across the board. I've also marked this as a duplicate to the bug filed where we are currently attempting to figure out how to solve this problem.

Revision history for this message
Augustina Ragwitz (auggy) wrote :

Removed duplicate status to the other bug as that was specifically with aggregate metadata, although the root problem is the same. The fix for this will be similar, but the tests will need to cover different scenarios. The current solution is to case fold in the sqlachemy api so you would only ever create "abc" even if you tried to create multiple metadata keys, ABC aBc abc, etc.

Revision history for this message
Sheel Rana (ranasheel2000) wrote :

Dear Augustina Ragwitz,

Are we done with approach finalization..(I did not see any final confirmation on http://lists.openstack.org/pipermail/openstack-dev/2016-February/087404.html)

As per above comment it seems we are following API folding approach during metadata creation time itself..
Could you please confirm so that i could go ahead with fix?

Also are we planning micro version for this?

Revision history for this message
Sheel Rana (ranasheel2000) wrote :

Ohk got confirmation on https://bugs.launchpad.net/nova/+bug/1538011 - comment#14

"Augustina Ragwitz (auggy) wrote on 2016-02-29:
Per discussion, current fix strategy is to casefold in sqlalchemy/api.py and force keys to all lower case."

I will share patch same way for this bug...

Thanks!!

Sean Dague (sdague)
Changed in nova:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Sheel Rana (<email address hidden>) on branch: master
Review: https://review.openstack.org/285222
Reason: Its been decided not to fix this..so closing this...

Anusha Unnam (anusha-unnam)
Changed in nova:
assignee: Sheel Rana (ranasheel2000) → nobody
status: In Progress → Confirmed
Revision history for this message
Anusha Unnam (anusha-unnam) wrote :

@Sheel Rana,
Can you give more details on why it is decided not to fix this?
I went through the comments on the patch and didn't find the reason.

sudhakar kumar srivastava (sudhakar.srivastava)
Changed in nova:
assignee: nobody → sudhakar kumar srivastava (sudhakar.srivastava)
surbhi sarda (surbhisarda)
Changed in nova:
assignee: sudhakar kumar srivastava (sudhakar.srivastava) → surbhi sarda (surbhisarda)
Revision history for this message
surbhi sarda (surbhisarda) wrote :

is this bug closed or can i work on it??

SREELAKSHMI PENTA (sreelakshmi-penta)
Changed in nova:
assignee: surbhi sarda (surbhisarda) → SREELAKSHMI PENTA (sreelakshmi-penta)
prameela kapuganti (prameela)
Changed in nova:
assignee: SREELAKSHMI PENTA (sreelakshmi-penta) → prameela kapuganti (prameela)
Nazeema Begum (nazeema123)
Changed in nova:
assignee: prameela kapuganti (prameela) → Nazeema Begum (nazeema123)
Sean Dague (sdague)
Changed in nova:
assignee: Nazeema Begum (nazeema123) → nobody
Rajesh Tailor (ratailor)
Changed in nova:
assignee: nobody → Rajesh Tailor (ratailor)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.