oslo.versionedobjects

[SRU] Fixing UUID coerce function for unicode non uuid form

Bug #1760918 reported by Seyeong Kim
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Fix Released
Undecided
Unassigned
Mitaka
Won't Fix
High
Unassigned
Ocata
Fix Released
High
Unassigned
Pike
Fix Released
High
Unassigned
Queens
Fix Released
Undecided
Unassigned
oslo.versionedobjects
Fix Released
Undecided
Unassigned
python-oslo.versionedobjects (Ubuntu)
Fix Released
High
Unassigned
Xenial
Won't Fix
High
Unassigned
Artful
Fix Released
High
Unassigned
Bionic
Fix Released
High
Unassigned

Bug Description

[Impact]

In Kilo, with ldap integrated, user_id can have non uuid format ( e.g. Gāo )
Then after upgrading kilo to mitaka ( in trusty env )

Gāo user can't create cinder volume with unicode error.

Affected Xenial, UCA Mitaka ( this is modified after i re-testing all )

Please refer to error code like below [1]

[Test Case]

1. deploy openstack env
2. source novarc ( as admin )
3. keystone user-create --name test
4. openstack user set --password-prompt test
4. log into keystone node and install mysql-client
5. connect to mysql with keystone user
6. do below query
- set foreign_key_checks = 0;
- update local_user set user_id='Gāo' where user_id='UUID';
- update user set id='Gāo' where id='UUID';
- set foreign_key_checks = 1;
7. keystone user-role-add --user test --tenant admin --role Admin
8. source novarc ( as test )
9. cinder create --name test 1

got unicode error on cinder-api node logs

[Regression Potential]
This patch make code handles unicode non uuid form user id. Basically this has little risk. but cinder-api service need to be restarted.

[Others]

[1] ###########

2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault six.reraise(*self._exc_info)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 82, in _execute_task
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault result = task.execute(**arguments)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/api/create_volume.py", line 527, in execute
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault volume = objects.Volume(context=context, **volume_properties)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/cinder/objects/volume.py", line 180, in __init__
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault super(Volume, self).__init__(*args, **kwargs)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 304, in __init__
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault setattr(self, key, kwargs[key])
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 72, in setter
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault field_value = field.coerce(self, name, value)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 192, in coerce
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault return self._type.coerce(obj, attr, value)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 325, in coerce
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault "for further details" % value, FutureWarning)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/logging/__init__.py", line 1709, in _showwarning
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault s = warnings.formatwarning(message, category, filename, lineno, line)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault File "/usr/lib/python2.7/warnings.py", line 38, in formatwarning
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault s = "%s:%s: %s: %s\n" % (filename, lineno, category.__name__, message)
2018-03-14 10:58:56.932 2198 ERROR cinder.api.middleware.fault UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 9: ordinal not in range(128)

See original description
OpenStack Infra (hudson-openstack)
Changed in oslo.versionedobjects:
assignee: nobody → Seyeong Kim (xtrusia)
status: New → In Progress
Seyeong Kim (seyeongkim)
tags: added: sts
Seyeong Kim (seyeongkim)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.versionedobjects (master)

Reviewed: https://review.openstack.org/557778
Committed: https://git.openstack.org/cgit/openstack/oslo.versionedobjects/commit/?id=b1d0b5d886afef8c08330bf3c2291e180aa1f534
Submitter: Zuul
Branch: master

commit b1d0b5d886afef8c08330bf3c2291e180aa1f534
Author: Seyeong Kim <email address hidden>
Date: Thu Mar 29 10:03:51 2018 -0700

    Fixing UUID coerce function for unicode non uuid form

    If we use non uuid unicode user id in py27 env,
    unicode error pop up when function is called.

    Closes-bug: #1760918

    Change-Id: Ic6b6308fb1960ec40407e6efde30137b64543e72

Changed in oslo.versionedobjects:
status: In Progress → Fix Released
Seyeong Kim (seyeongkim)
description: updated
summary: - Fixing UUID coerce function for unicode non uuid form
+ [SRU] Fixing UUID coerce function for unicode non uuid form
Revision history for this message
Seyeong Kim (seyeongkim) wrote :
description: updated
tags: added: sts-sru-needed
Revision history for this message
Seyeong Kim (seyeongkim) wrote :
Revision history for this message
Seyeong Kim (seyeongkim) wrote :
Revision history for this message
Seyeong Kim (seyeongkim) wrote :
Revision history for this message
Seyeong Kim (seyeongkim) wrote :
Revision history for this message
Seyeong Kim (seyeongkim) wrote :
Revision history for this message
Seyeong Kim (seyeongkim) wrote :
Changed in cloud-archive:
assignee: nobody → Seyeong Kim (xtrusia)
Changed in python-oslo.versionedobjects (Ubuntu):
assignee: nobody → Seyeong Kim (xtrusia)
Changed in cloud-archive:
status: New → In Progress
Changed in python-oslo.versionedobjects (Ubuntu):
status: New → In Progress
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "lp1760918_xenial.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.versionedobjects (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/559815

Corey Bryant (corey.bryant)
Changed in python-oslo.versionedobjects (Ubuntu Xenial):
importance: Undecided → High
status: New → Triaged
Changed in python-oslo.versionedobjects (Ubuntu Artful):
importance: Undecided → High
status: New → Triaged
Changed in python-oslo.versionedobjects (Ubuntu Bionic):
importance: Undecided → High
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hi Seyeong,

Thanks for the bug and patches. In the future it would be nice to include the exception in the bug for others who search for it.

Thanks,
Corey

Revision history for this message
Corey Bryant (corey.bryant) wrote :

I've uploaded this for all affected releases. Most are awaiting review by the SRU team.
Note that I updated the ocata patch to add "# -*- coding: utf-8 -*-" to test_fields.py

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-oslo.versionedobjects - 1.31.2-0ubuntu2

---------------
python-oslo.versionedobjects (1.31.2-0ubuntu2) bionic; urgency=medium

  * Fixing UUID coerce function for unicode non uuid form id (LP: #1760918)
    - d/p/Fixing-uuid-coerce-function-for-unicode-non-uuid-form.patch

 -- Seyeong Kim <email address hidden> Sun, 08 Apr 2018 18:05:44 -0700

Changed in python-oslo.versionedobjects (Ubuntu Bionic):
status: In Progress → Fix Released
Revision history for this message
Corey Bryant (corey.bryant) wrote : Please test proposed package

Hello Seyeong, or anyone else affected,

Accepted python-oslo.versionedobjects into ocata-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ocata-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ocata-needed to verification-ocata-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ocata-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-ocata-needed
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello Seyeong, or anyone else affected,

Accepted python-oslo.versionedobjects into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-queens-needed
Revision history for this message
Seyeong Kim (seyeongkim) wrote :

Hello Corey

Im sorry for making confusion on this case.

I thought I tested all of releases previously.
so i uploaded patch for all of them

but during verification, I found some of them are not affected.

- not affected.
UCA ocata, pike, queens
artful,

- affected
UCA mitaka
xenial

for mitaka and xenial, my patch is working fine and simple. though there is the other patch can fix this.

I need discussion about this with you.

I'll ping you in irc.

Thanks a lot

tags: added: verification-queens-failed
removed: verification-queens-needed
tags: added: verification-ocata-failed
removed: verification-ocata-needed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.versionedobjects 1.33.0

This issue was fixed in the openstack/oslo.versionedobjects 1.33.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.versionedobjects (stable/pike)

Reviewed: https://review.openstack.org/559820
Committed: https://git.openstack.org/cgit/openstack/oslo.versionedobjects/commit/?id=5931129fc998f0ef48f39b0c6e84faa564670f1d
Submitter: Zuul
Branch: stable/pike

commit 5931129fc998f0ef48f39b0c6e84faa564670f1d
Author: Seyeong Kim <email address hidden>
Date: Thu Mar 29 10:03:51 2018 -0700

    Fixing UUID coerce function for unicode non uuid form

    If we use non uuid unicode user id in py27 env,
    unicode error pop up when function is called.

    Closes-bug: #1760918

    Change-Id: Ic6b6308fb1960ec40407e6efde30137b64543e72
    (cherry picked from commit b1d0b5d886afef8c08330bf3c2291e180aa1f534)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.versionedobjects (stable/queens)

Reviewed: https://review.openstack.org/559815
Committed: https://git.openstack.org/cgit/openstack/oslo.versionedobjects/commit/?id=e918eb976fb5a6f9fa7b7644d5a10d383fcfcf21
Submitter: Zuul
Branch: stable/queens

commit e918eb976fb5a6f9fa7b7644d5a10d383fcfcf21
Author: Seyeong Kim <email address hidden>
Date: Thu Mar 29 10:03:51 2018 -0700

    Fixing UUID coerce function for unicode non uuid form

    If we use non uuid unicode user id in py27 env,
    unicode error pop up when function is called.

    Closes-bug: #1760918

    Change-Id: Ic6b6308fb1960ec40407e6efde30137b64543e72
    (cherry picked from commit b1d0b5d886afef8c08330bf3c2291e180aa1f534)

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Seyeong, If I understand correctly, you're saying this patch isn't required on any release other than mitaka, however it doesn't break the other releases. Is that correct? The patch has already landed upstream in master (rocky), stable/queens, and stable/pike. If it doesn't regress those releases then I don't think it's worth reverting upstream. As for the SRU for those releases, we don't need to make the extra effort to SRU it. Please confirm before we move ahead. Thanks.

Revision history for this message
Sean McGinnis (sean-mcginnis) wrote :

This fix was not correct and will need to be backported again.

https://bugs.launchpad.net/oslo.versionedobjects/+bug/1763179

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Does comment #21 mean that the fix is introducing regressions? Should I reject the current python-oslo.versionedobjects from the artful and xenial queues?

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Lukasz, yes please reject and we'll get the new fix backported. Thanks Sean.

Seyeong Kim (seyeongkim)
Changed in oslo.versionedobjects:
assignee: Seyeong Kim (xtrusia) → nobody
Changed in python-oslo.versionedobjects (Ubuntu Bionic):
assignee: Seyeong Kim (xtrusia) → nobody
Revision history for this message
Seyeong Kim (seyeongkim) wrote :

corey, sil2100

I was off for 2 days.

Thanks a lot and sorry for making confusions.

as upstream, i need to find the other patch not this for the customer's issue.

changed patch ( i tested before, sean fixed ) doesn't work for xenial, mitaka for the customer's issue.

I'll find it.

Please reject those.

Thanks again

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This is being reverted via https://bugs.launchpad.net/cloud-archive/+bug/1763179.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

I'm not sure why pike was marked as fix committed. Moved back to Triaged as nothing was uploaded to the UCA for pike.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

I've re-uploaded this for bionic (queens), artful (pike), and ocata along with the fix for LP: #1763179. Bionic and Artful are awaiting SRU team review.

Seyeong Kim (seyeongkim)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Seyeong, or anyone else affected,

Accepted python-oslo.versionedobjects into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-oslo.versionedobjects/1.26.0-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-oslo.versionedobjects (Ubuntu Artful):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-artful
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello Seyeong, or anyone else affected,

Accepted python-oslo.versionedobjects into pike-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:pike-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-pike-needed to verification-pike-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-pike-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-pike-needed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.versionedobjects 1.31.3

This issue was fixed in the openstack/oslo.versionedobjects 1.31.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/oslo.versionedobjects 1.26.2

This issue was fixed in the openstack/oslo.versionedobjects 1.26.2 release.

Revision history for this message
Seyeong Kim (seyeongkim) wrote :

for artful

tested TEST CASE

p.s. artful was fine but upgraded pkg also fine.

dpkg -l | grep python-oslo.versionedobjects

ii python-oslo.versionedobjects 1.26.0-0ubuntu2 all deals with DB schema versions and code expectations - Python 2.x

tags: added: verification-done-artful
removed: verification-needed-artful
Revision history for this message
Seyeong Kim (seyeongkim) wrote :

for pike

tested TEST CASE

dpkg -l | grep python-oslo.versionedobjects

ii python-oslo.versionedobjects 1.26.0-0ubuntu2~cloud0 all deals with DB schema versions and code expectations - Python 2.x

tags: added: verification-pike-done
removed: verification-pike-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-oslo.versionedobjects - 1.26.0-0ubuntu2

---------------
python-oslo.versionedobjects (1.26.0-0ubuntu2) artful; urgency=medium

  [ Corey Bryant ]
  * d/p/dont-force-unicode-strings-for-UUID-coercion.patch:
    This reverts and changes the fix provided by
    d/p/Fixing-uuid-coerce-function-for-unicode-non-uuid-form.patch
    (LP: #1763179).

  [ Seyeong Kim ]
  * Fixing UUID coerce function for unicode non uuid form id (LP: #1760918)
    - d/p/Fixing-uuid-coerce-function-for-unicode-non-uuid-form.patch

  [ Corey Bryant ]
  * d/gbp.conf: Create stable/pike branch.

 -- Corey Bryant <email address hidden> Tue, 17 Apr 2018 08:47:39 -0400

Changed in python-oslo.versionedobjects (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for python-oslo.versionedobjects has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for python-oslo.versionedobjects has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package python-oslo.versionedobjects - 1.26.0-0ubuntu2~cloud0
---------------

 python-oslo.versionedobjects (1.26.0-0ubuntu2~cloud0) xenial-pike; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 python-oslo.versionedobjects (1.26.0-0ubuntu2) artful; urgency=medium
 .
   [ Corey Bryant ]
   * d/p/dont-force-unicode-strings-for-UUID-coercion.patch:
     This reverts and changes the fix provided by
     d/p/Fixing-uuid-coerce-function-for-unicode-non-uuid-form.patch
     (LP: #1763179).
 .
   [ Seyeong Kim ]
   * Fixing UUID coerce function for unicode non uuid form id (LP: #1760918)
     - d/p/Fixing-uuid-coerce-function-for-unicode-non-uuid-form.patch
 .
   [ Corey Bryant ]
   * d/gbp.conf: Create stable/pike branch.

Revision history for this message
Seyeong Kim (seyeongkim) wrote :

@corey
we can do this sru for xenial and mitaka as well?

Thanks

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hi Seoyeong, since this patch was reverted in favor of https://bugs.launchpad.net/oslo.versionedobjects/+bug/1763179, and never uploaded for Xenial/Mitaka, can we just fix this via LP#1763179? Marking as Won't Fix for xenial/mitaka for now.

Changed in python-oslo.versionedobjects (Ubuntu Xenial):
status: Triaged → Won't Fix
Revision history for this message
Corey Bryant (corey.bryant) wrote :

To be clear, this patch wasn't completely reverted. Rather, LP:1763179 was fixed on top of this patch. Nonetheless, since nothing's been uploaded yet for xenial/mitaka, let's plan on just fixing this via a single bug in LP:1763179.

Revision history for this message
Seyeong Kim (seyeongkim) wrote :

got it,

thanks corey

Edward Hope-Morley (hopem)
tags: added: sts-sru-done
removed: sts-sru-needed
Chris MacNaughton (chris.macnaughton)
Changed in cloud-archive:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.