[SRU] Fix None comparison when sorting by `updated_at`

Bug #2019175 reported by Andreas Hamacher
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Fix Released
Undecided
Unassigned
Ussuri
Fix Released
Undecided
Unassigned
Victoria
Fix Released
Undecided
Unassigned
Wallaby
Fix Released
Undecided
Unassigned
Xena
Fix Released
Undecided
Unassigned
Yoga
Fix Released
Undecided
Unassigned
Zed
Fix Released
Undecided
Unassigned
heat (Ubuntu)
Fix Released
Undecided
Unassigned
Focal
Fix Released
Undecided
Unassigned
Jammy
Fix Released
Undecided
Unassigned

Bug Description

StoryBoard link: https://storyboard.openstack.org/#!/story/2009653

[Impact]

If resource candidates were never updated before then their `updated_at` value in the db remain `None`.
This prevents future resource updates when any of of the resources in db has been replaced without being updated before.

[Test Plan]

A. Reproducer steps:
  1. create a project and nova instances using > 1 vcpus in that project
  2. create a stack with a OS::Nova::Quota resource managing the project's quotas
  3. change the vcpu quota to an invalid value (i.e. 0) and update the stack three times
  4. set a valid quota and update the stack again

Then stack update will fail with the message:
"'<' not supported between instances of 'datetime.datetime' and 'NoneType'"
(openstack stack show <stackname>)

B. Install the heat package with fix, restart heat service
C. Re-run step 4 from reproducer steps again (update the stack once again)
D. Confirm that resource update succeeds and the stack update shows 'UPDATE_COMPLETE' with no errors (can repeat several times as well) or fails with a different message for another reason

[Where problems could occur]

If the `created_at` value happens to `None` as well for some reason then this will end up failing in the same manner.
Newer OpenStack versions already contain this fix and released packages (Wallaby, Yoga, and Zed) work as expected.

[Other info]

The fix was backported to Ussuri after the last release of Heat (14.2).
Upstream PR: https://review.opendev.org/c/openstack/heat/+/837697

This was impacting a user who is using Ussuri. Latest releases already contain the fix in the following packages:
Wallaby: heat_16.1.0-0ubuntu1~cloud0
Yoga: heat_18.0.0-0ubuntu1.2~cloud0
Zed: heat_19.0.0-0ubuntu1.2~cloud0

Related branches

Revision history for this message
Andreas Hamacher (andreashamacher) wrote :

really sorry... this belongs into python-heat, not puppet-heat

affects: puppet-heat → cloud-archive
summary: - bug fixed and backported but not released in package
+ [SRU] Fix None comparision when sorting by `updated_at`
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote : Re: [SRU] Fix None comparision when sorting by `updated_at`

New OpenStack releases have the fix already in the following package versions:

Wallaby - heat_16.1.0-0ubuntu1~cloud0
Yoga - heat_18.0.0-0ubuntu1.2~cloud0
Zed - heat_19.0.0-0ubuntu1.2~cloud0

description: updated
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote (last edit ):

A new Xena release containing this fix is currently in -proposed.

https://bugs.launchpad.net/cloud-archive/+bug/2019763

That'll certainly supersede this one, so I'll wait on its release and do SRU for Ussuri & Victoria only.

Changed in heat (Ubuntu):
status: New → Fix Released
Changed in cloud-archive:
status: New → Fix Released
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Thanks Pon. Merged and uploaded to victoria-staging and the focal unapproved queue [1].

[1] https://launchpad.net/ubuntu/focal/+queue?queue_state=1&queue_text=heat

Revision history for this message
Corey Bryant (corey.bryant) wrote : Please test proposed package

Hello Andreas, or anyone else affected,

Accepted heat into victoria-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:victoria-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-victoria-needed to verification-victoria-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-victoria-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-victoria-needed
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote (last edit ): Re: [SRU] Fix None comparision when sorting by `updated_at`

Xena has fix in 1:17.0.2-0ubuntu1~cloud0 (released via https://bugs.launchpad.net/cloud-archive/+bug/2019763).

summary: - [SRU] Fix None comparision when sorting by `updated_at`
+ [SRU] Fix None comparison when sorting by `updated_at`
Revision history for this message
Andreas Hasenack (ahasenack) wrote : Please test proposed package

Hello Andreas, or anyone else affected,

Accepted heat into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/heat/1:14.2.0-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in heat (Ubuntu Focal):
status: New → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Can you please clarify the error in the [test plan]? I suppose at some point you will get something like
"'<' not supported between instances of 'datetime.datetime' and 'NoneType'"? Just so that someone following the test plan knows what to expect.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello Andreas, or anyone else affected,

Accepted heat into ussuri-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ussuri-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ussuri-needed to verification-ussuri-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ussuri-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-ussuri-needed
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Following the reproducer steps:
 - create a project and nova instances using > 1 vcpus in that project
 - create a stack with a OS::Nova::Quota resource managing the project's quotas
 - change the vcpu quota to an invalid value (i.e. 0) and update the stack three times
 - set a valid quota and update the stack again

was able to reproduce the issue with heat 1:14.2.0-0ubuntu1.1 and verified that the heat 1:15.1.0-0ubuntu1~cloud1 fixes it.

Attached the logs and test deplyoyment for Victoria.

tags: added: verification-victoria-done
removed: verification-victoria-needed
description: updated
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Similarly completed testing for Ussuri too.

Commands, stack template, etc can be seen in the attached log.

tags: added: verification-done verification-done-focal verification-ussuri-done
removed: verification-needed verification-needed-focal verification-ussuri-needed
Revision history for this message
Corey Bryant (corey.bryant) wrote : Update Released

The verification of the Stable Release Update for heat has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package heat - 1:15.1.0-0ubuntu1~cloud1
---------------

 heat (1:15.1.0-0ubuntu1~cloud1) focal-victoria; urgency=medium
 .
   [ Ponnuvel Palaniyappan ]
   * d/p/lp2019175.patch: Fix None comparision when sorting
     by `updated_at` (LP: #2019175)
 .
   [ Corey Bryant ]
   * d/p/lp2019175.patch: Updated patch from upstream stable/victoria branch.

Revision history for this message
Chris Halse Rogers (raof) wrote :

Has this actually been verified for focal? The comment only claims to be testing for ussuri, and doesn't include any version numbering we could use to verify what packages were actually tested.

Changed in heat (Ubuntu Focal):
status: Fix Committed → Incomplete
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

@raof, The victoria verification was done on Focal. As there's only two new packages were created (ussuri & victoria UCA packages), I assumed there's no separate focal release and marked it complete based on focal-victoria verification. Corey can confirm whether a separate Ubuntu package for Focal will be created & released.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hi Pon, The package that Andreas accepted into focal-proposed is the one to test for focal (https://launchpad.net/ubuntu/+source/heat/1:14.2.0-0ubuntu2). That was also backported to ussuri-proposed (on bionic) for testing.

Changed in heat (Ubuntu Jammy):
status: New → Fix Released
Revision history for this message
Edward Hope-Morley (hopem) wrote :

@pponnuvel the package in focal-proposed needs to be verified and the package in bionic-ussuri/proposed also needs to be verified. Once both are verified you can set verification-done-focal verification-ussuri-done respectively. It seems that you have verified bionic-ussuri/proposed so i will leave that tag as-is but not focal-proposed so I will set that tag back to verification-needed-focal.

tags: added: verification-needed verification-needed-focal
removed: verification-done verification-done-focal
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote (last edit ):

Just to clarify, the previous two verification I did were using:

1:14.2.0-0ubuntu2~cloud0 (on Bionic, ussuri-proposed)
1:15.1.0-0ubuntu1~cloud1 (on Focal, victoria-proposed)

Corey confirmed (also clarified by Ed) a separate SRU verification for Focal will be needed.
So I'll do another verification using 1:14.2.0-0ubuntu2 on Focal. Thanks all for clearing it up!

Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Attached SRU logs using 1:14.2.0-0ubuntu2

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Marked all verifications as complete.

Done 3 for:

1:14.2.0-0ubuntu2~cloud0 (on Bionic, ussuri-proposed)
1:15.1.0-0ubuntu1~cloud1 (on Focal, victoria-proposed)
1:14.2.0-0ubuntu2 (on Focal, focal-proposed)

Revision history for this message
Ponnuvel Palaniyappan (pponnuvel) wrote :

Looks like I forgot to attach the Focal verification log. Attaching here.

Changed in heat (Ubuntu Focal):
status: Incomplete → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package heat - 1:14.2.0-0ubuntu2

---------------
heat (1:14.2.0-0ubuntu2) focal; urgency=medium

  * d/p/lp2019175.patch: Fix None comparision when sorting
    by `updated_at` (LP: #2019175)

 -- Ponnuvel Palaniyappan <email address hidden> Mon, 15 May 2023 17:37:30 +0100

Changed in heat (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for heat has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package heat - 1:14.2.0-0ubuntu2~cloud0
---------------

 heat (1:14.2.0-0ubuntu2~cloud0) bionic-ussuri; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 heat (1:14.2.0-0ubuntu2) focal; urgency=medium
 .
   * d/p/lp2019175.patch: Fix None comparision when sorting
     by `updated_at` (LP: #2019175)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.