[SRU] Keystone user tenant membership not always removed

Bug #998137 reported by Adam Gandelman
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
High
Adam Gandelman
Essex
Fix Released
Undecided
Unassigned
keystone (Ubuntu)
Fix Released
Undecided
Unassigned
Precise
Fix Released
Undecided
Unassigned
Quantal
Fix Released
Undecided
Unassigned

Bug Description

The fix for Bug #959294 removes tenant association before attempting to delete the user to avoid a FK constraint error. On some configurations, this is not always reliable. The integirty error still happens, although subsequent attempts to delete user eventually succeed.

See shell log at http://paste.ubuntu.com/980372/

Changed in keystone:
assignee: nobody → Adam Gandelman (gandelman-a)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/7353

Joseph Heck (heckj)
tags: added: essex-backport-potential
Changed in keystone:
milestone: none → folsom-1
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/7353
Committed: http://github.com/openstack/keystone/commit/2e912f4b91b57c060c2363c067125276a92457d2
Submitter: Jenkins
Branch: master

commit 2e912f4b91b57c060c2363c067125276a92457d2
Author: Adam Gandelman <email address hidden>
Date: Fri May 11 09:49:39 2012 -0700

    Flush tenant membership deletion before user

    Ensure user tenant membership is *actually* removed before deleting
    user.

    Fixes bug 998137. This should be proposed to stable/essex as well.

    Change-Id: I097456e603177646d62ebf08ca71af1fd7f787bf

Changed in keystone:
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote : Re: Keystone user tenant membership not always removed

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in keystone (Ubuntu):
status: New → Confirmed
Revision history for this message
Rohit Karajgi (rohitk) wrote :

Can this be back ported to stable/essex?

Revision history for this message
Jay Pipes (jaypipes) wrote :

Rohit, bring this bug to the attention of the stable/essex branch maintainers for Keystone (I believe they are Adam Gandelman -- who reported this bug -- and Mark McLoughlin)

Revision history for this message
Adam Gandelman (gandelman-a) wrote :

Proposed to stable/essex here: https://review.openstack.org/#/c/7676/

Assuming its merged, it should be updated in Ubuntu Precise with the first set of stable release updates of the Openstack packages

Revision history for this message
Rohit Karajgi (rohitk) wrote :

Thanks Adam!

Thierry Carrez (ttx)
Changed in keystone:
status: Fix Committed → Fix Released
Changed in keystone (Ubuntu Precise):
status: New → Confirmed
Revision history for this message
Adam Gandelman (gandelman-a) wrote :

Fixed upstream and released into Quantal with 2012.2~f1-0ubuntu1.

Changed in keystone (Ubuntu Quantal):
status: Confirmed → Fix Released
Revision history for this message
Alan Pevec (apevec) wrote :

> Proposed to stable/essex here: https://review.openstack.org/#/c/7676/

Is this still Essex backport candidate? If yes, above review should be resubmitted.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (stable/essex)

Fix proposed to branch: stable/essex
Review: https://review.openstack.org/8403

Revision history for this message
Adam Gandelman (gandelman-a) wrote : Re: Keystone user tenant membership not always removed

Resubmitted here, https://review.openstack.org/#/c/8403/ Thanks, Alan.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (stable/essex)

Reviewed: https://review.openstack.org/8403
Committed: http://github.com/openstack/keystone/commit/d0a73669369d86ff4c7b9de715fa4eec9bc58b59
Submitter: Jenkins
Branch: stable/essex

commit d0a73669369d86ff4c7b9de715fa4eec9bc58b59
Author: Adam Gandelman <email address hidden>
Date: Mon Jun 11 10:35:16 2012 -0700

    Flush tenant membership deletion before user

    Ensure user tenant membership is *actually* removed before deleting
    user.

    Applied to 'stable/essex', originally committed to trunk via
    https://review.openstack.org/#/c/7353/

    Fixes bug 998137.

    Change-Id: Ib52970d68f288b8742c3e060c7040838a1c738c2

tags: added: in-stable-essex
Chuck Short (zulcss)
summary: - Keystone user tenant membership not always removed
+ [SRU] Keystone user tenant membership not always removed
Revision history for this message
Chuck Short (zulcss) wrote :

** Impact **

The fix for Bug #959294 removes tenant association before attempting to delete the user to avoid a FK constraint error. On some configurations, this is not always reliable. The integirty error still happens, although subsequent attempts to delete user eventually succeed.

See shell log at http://paste.ubuntu.com/980372/

** Development Fix **

This has been addressed in https://review.openstack.org/7353 and fixed in quantal

** Stable Fix **

This has been address in https://review.openstack.org/8403.

** Test case **

Please see http://paste.ubuntu.com/980372/

** Regression Potential **

Minimal this has been tested in the openstack-ci lab

Revision history for this message
Brian Murray (brian-murray) wrote :

Putting everything in one place, here is the test case:

ubuntu@ip-10-252-87-70:~$ tenant_id=$(keystone tenant-create --name testingtenant | grep id | awk '{ print $4 }')
ubuntu@ip-10-252-87-70:~$ user_id=$(keystone user-create --name testinguser --pass testing --tenant_id $tenant_id | grep id | awk '{ print $4 }')
ubuntu@ip-10-252-87-70:~$ echo $user_id
e5e045e898694f46a55fb9483a1c03c4
ubuntu@ip-10-252-87-70:~$ keystone user-delete $user_id
No handlers could be found for logger "keystoneclient.client"
An unexpected error prevented the server from fulfilling your request. (IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`keystone`.`user_tenant_membership`, CONSTRAINT `user_tenant_membership_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`))') 'DELETE FROM user WHERE user.id = %s' ('e5e045e898694f46a55fb9483a1c03c4',) (HTTP 500)
ubuntu@ip-10-252-87-70:~$ keystone user-delete $user_id
No handlers could be found for logger "keystoneclient.client"
An unexpected error prevented the server from fulfilling your request. (IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`keystone`.`user_tenant_membership`, CONSTRAINT `user_tenant_membership_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`))') 'DELETE FROM user WHERE user.id = %s' ('e5e045e898694f46a55fb9483a1c03c4',) (HTTP 500)
ubuntu@ip-10-252-87-70:~$ keystone user-delete $user_id
No handlers could be found for logger "keystoneclient.client"
An unexpected error prevented the server from fulfilling your request. (IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`keystone`.`user_tenant_membership`, CONSTRAINT `user_tenant_membership_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`))') 'DELETE FROM user WHERE user.id = %s' ('e5e045e898694f46a55fb9483a1c03c4',) (HTTP 500)
ubuntu@ip-10-252-87-70:~$ keystone user-delete $user_id
No handlers could be found for logger "keystoneclient.client"
An unexpected error prevented the server from fulfilling your request. (IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`keystone`.`user_tenant_membership`, CONSTRAINT `user_tenant_membership_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `user` (`id`))') 'DELETE FROM user WHERE user.id = %s' ('e5e045e898694f46a55fb9483a1c03c4',) (HTTP 500)
ubuntu@ip-10-252-87-70:~$ keystone user-delete $user_id
ubuntu@ip-10-252-87-70:~$

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Adam, or anyone else affected,

Accepted keystone into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in keystone (Ubuntu Precise):
status: Confirmed → Fix Committed
tags: added: verification-needed
James Page (james-page)
Changed in keystone (Ubuntu Precise):
milestone: none → ubuntu-12.04.1
Revision history for this message
Sebastien Bacher (seb128) wrote :

This update is in proposed for 26 days, could somebody using that software verify the fix so the SRU can be moved to updates?

Revision history for this message
Adam Gandelman (gandelman-a) wrote : Verification report.

Please find the attached Jenkins job results from the Ubuntu Server Team's CI infrastructure. As part of the verification process for this bug, Keystone has been deployed and configured across multiple nodes using precise-proposed as an installation source. After successful bring-up and configuration of the cluster, a number of exercises and smoke tests have be invoked to ensure the updated package did not introduce any regressions. A number of test iterations were carried out to catch any possible transient errors.

Please Note the list of installed packages at the top and bottom of the report.

For records of upstream test coverage of this update, please see the Jenkins links in the comments of the relevant upstream code-review(s):

Trunk review: https://review.openstack.org/7353
Stable review: https://review.openstack.org/8403

As per the provisional Micro Release Exception granted to this package by the Technical Board, we hope this contributes toward verification of this update.

Revision history for this message
Adam Gandelman (gandelman-a) wrote :

Test coverage log.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package keystone - 2012.1+stable~20120608-aff45d6-0ubuntu1

---------------
keystone (2012.1+stable~20120608-aff45d6-0ubuntu1) precise-proposed; urgency=low

  * New usptream snapshot. (LP: #1010473)
  * Resynchronize with stable/essex:
    - aff45d6 - Make import_nova_auth only create roles which don't already exist
      (LP: #959294)
  * debian/patches/0013-Flush-tenant-membership-deletion-before-user.patch: Backported
    fix for "Flush tenant membership deletion before user." (LP: #998137)
 -- Chuck Short <email address hidden> Tue, 05 Jun 2012 11:24:02 -0400

Changed in keystone (Ubuntu Precise):
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in keystone:
milestone: folsom-1 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.