do-release-upgrade fails when ESM packages are installed

FYI - PR being discussed in https://github.com/canonical/ubuntu-advantage-client/pull/2261

This fix is urgently needed in the stable series but Kinetic is in beta freeze. Therefore I'm accepting the SRUs even though Kinetic isn't fixed yet. The Kinetic upload is ready in the linked MP and can be uploaded as soon as Kinetic is open for uploads again. We tagged block-proposed "just in case" but didn't want to even upload to kinetic-proposed without the approval of the release team, so are staging it prior to upload instead.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.1~22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.1~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.1~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.2~22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.2~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.2~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Renan, or anyone else affected,

Accepted ubuntu-advantage-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.11.2~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Integration test results after fixing this bug:

Loading config defaults from "./tox.ini"
Using defaults:
          color True
  show_snippets True
   show_skipped False
        dry_run False
    show_source True
   show_timings True
 stdout_capture False
 stderr_capture False
    log_capture False
 logging_format %(asctime)s:%(levelname)s:%(name)s:%(message)s
  logging_level info
  steps_catalog False
        summary True
          junit True
          stage None
       userdata {}
 default_format pretty
   default_tags
scenario_outline_annotation_schema {name} -- @{row.id} {examples.name}
more_formatters {}
Using default path "./features"
Trying base directory: /home/renan/work/ubuntu-advantage-client/features
--- job suffix: None
Config options:
  image_clean = True
  destroy_instances = True
  ephemeral_instance = False
  snapshot_strategy = False
  cloud_credentials_path = None
  contract_token = <REDACTED>
  contract_token_staging = <REDACTED>
  contract_token_staging_expired = <REDACTED>
  machine_type = lxd.container
  private_key_file = None
  private_key_name = uaclient-integration
  reuse_image = None
  debs_path = None
  artifact_dir = None
  install_from = InstallationSource.PROPOSED
  custom_ppa = None
  custom_ppa_keyid = None
  userdata_file = None
  check_version = 27.11.2~16.04.1
  sbuild_chroot = None

Feature: Upgrade between releases when uaclient is unattached # features/ubuntu_upgrade_unattached.feature:2

  @slow @series.all @uses.config.machine_type.lxd.container @upgrade
  Scenario Outline: Unattached upgrade -- @1.1 ubuntu release # features/ubuntu_upgrade_unattached.feature:41
    Given a `xenial` machine with ubuntu-advantage-tools installed # features/steps/steps.py:61
    When I prepare the local PPAs to upgrade from `xenial` to `bionic` # features/steps/steps.py:1132
    And I run `apt-get dist-upgrade --assume-yes` with sudo # features/steps/steps.py:373
    And I reboot the machine # features/steps/steps.py:574
    And I create the file `/etc/update-manager/release-upgrades.d/ua-test.cfg` with the following # features/steps/steps.py:557
      """
      [Sources]
      AllowThirdParty=yes
      """
    And I run `sed -i 's/Prompt=lts/Prompt=lts/' /etc/update-manager/release-upgrades` with sudo # features/steps/steps.py:373
    And I run `do-release-upgrade --frontend DistUpgradeViewNonInteractive` `with sudo` and stdin `y\n` # features/steps/steps.py:300
    And I reboot the machine # features/steps/steps.py:574
    And I run `lsb_release -cs` as non-root # features/steps/steps.py:373
    Then I will see the following on stdout # features/steps/steps.py:611
      """
      bionic
      """
    And I verify that running `egrep "xenial|disabled" /etc/apt/sour...

This bug was fixed in the package ubuntu-advantage-tools - 27.11.2~22.10.1

---------------
ubuntu-advantage-tools (27.11.2~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.2: (LP: #1991173)
    - esm: add the --beta flag back to esm-apps
    - messaging: show Ubuntu Pro beta message in apt output
    - security-status: don't show esm-apps information when the service is not
      enabled
    - ros: add the --beta flag back to ros and ros-updates

ubuntu-advantage-tools (27.11.1~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.1: (LP: #1990907)
    - Fix release upgrade when ESM packages are installed
      + d/postinst: remove series information from the APT preferences template
      + esm: remove series information from the APT preferences file

 -- Renan Rodrigo <email address hidden> Thu, 29 Sep 2022 16:20:42 -0300

This bug was fixed in the package ubuntu-advantage-tools - 27.11.2~16.04.1

---------------
ubuntu-advantage-tools (27.11.2~16.04.1) xenial; urgency=medium

  * Backport new upstream release: (LP: #1991173) to xenial

ubuntu-advantage-tools (27.11.2~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.2: (LP: #1991173)
    - esm: add the --beta flag back to esm-apps
    - messaging: show Ubuntu Pro beta message in apt output
    - security-status: don't show esm-apps information when the service is not
      enabled
    - ros: add the --beta flag back to ros and ros-updates

ubuntu-advantage-tools (27.11.1~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.1: (LP: #1990907)
    - Fix release upgrade when ESM packages are installed
      + d/postinst: remove series information from the APT preferences template
      + esm: remove series information from the APT preferences file

ubuntu-advantage-tools (27.11~22.10.1) kinetic; urgency=medium

  * d/control:
    - Update VCS references
  * d/links:
    - add usr/bin/pro as an alias to ubuntu-advantage
  * d/postinst:
    - include root_mode parameter when creating UAConfig instances
    - change calls to add_notice to notice_file.add
    - create public machine-token file if it does not exist
  * New upstream release 27.11 (LP: #1989279)
    - api:
      + new `pro api` command to access the public client API
      + 'version' endpoint returning version information
      + 'should auto attach' endpoint informing if a system should run
        auto-attach on startup
      + 'full auto attach' endpoint performing auto-attach
      + 'magic attach' endpoints for the Magic Attach flow
    - auto-attach:
      + better errors for invalid pro images (GH: #2180, #1833)
      + don't detach on already auto-attached instances
      + no-op when ubuntu-advantage information is present on cloud-init
        userdata
      + change systemd unit to run after cloud-config
    - cli:
      + cli: better error message on unrecognized flags (GH: #672)
    - collect-logs:
      + can now be executed as a non-root user
      + is executed automatically and result is appended when using apport to
        report a bug
    - docs: now formatted to be built with sphinx, and published in readthedocs
    - enable:
      + new access-only flag for usecases where auto-install is undesired
      + fix apt auth line replacement (LP: #1985863)
    - esm-apps: generally available as non-beta as part of Ubuntu Pro
    - fix: check if livepatch has already fixed a CVE before attempting a fix
    - jobs: new timer job to check if the release reached end of support
    - pro:
      + Ubuntu Pro is released as a product
      + make `pro` the recommended executable for the client
      + client, apt and motd messages updated/rewritten to show Pro
        information
      + base URL changed from /advantage to /pro
      + ESM services renamed as part of Pro
    - ros: released as a non-beta entitlement
    - security-status
      + does not require the --format flag anymore
      + human readable output added based on ubuntu-security-status
      + machine readable output contains CVEs fixed by Livepatch
...

The verification of the Stable Release Update for ubuntu-advantage-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package ubuntu-advantage-tools - 27.11.2~18.04.1

---------------
ubuntu-advantage-tools (27.11.2~18.04.1) bionic; urgency=medium

  * Backport new upstream release: (LP: #1991173) to bionic

ubuntu-advantage-tools (27.11.2~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.2: (LP: #1991173)
    - esm: add the --beta flag back to esm-apps
    - messaging: show Ubuntu Pro beta message in apt output
    - security-status: don't show esm-apps information when the service is not
      enabled
    - ros: add the --beta flag back to ros and ros-updates

ubuntu-advantage-tools (27.11.1~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.1: (LP: #1990907)
    - Fix release upgrade when ESM packages are installed
      + d/postinst: remove series information from the APT preferences template
      + esm: remove series information from the APT preferences file

ubuntu-advantage-tools (27.11~22.10.1) kinetic; urgency=medium

  * d/control:
    - Update VCS references
  * d/links:
    - add usr/bin/pro as an alias to ubuntu-advantage
  * d/postinst:
    - include root_mode parameter when creating UAConfig instances
    - change calls to add_notice to notice_file.add
    - create public machine-token file if it does not exist
  * New upstream release 27.11 (LP: #1989279)
    - api:
      + new `pro api` command to access the public client API
      + 'version' endpoint returning version information
      + 'should auto attach' endpoint informing if a system should run
        auto-attach on startup
      + 'full auto attach' endpoint performing auto-attach
      + 'magic attach' endpoints for the Magic Attach flow
    - auto-attach:
      + better errors for invalid pro images (GH: #2180, #1833)
      + don't detach on already auto-attached instances
      + no-op when ubuntu-advantage information is present on cloud-init
        userdata
      + change systemd unit to run after cloud-config
    - cli:
      + cli: better error message on unrecognized flags (GH: #672)
    - collect-logs:
      + can now be executed as a non-root user
      + is executed automatically and result is appended when using apport to
        report a bug
    - docs: now formatted to be built with sphinx, and published in readthedocs
    - enable:
      + new access-only flag for usecases where auto-install is undesired
      + fix apt auth line replacement (LP: #1985863)
    - esm-apps: generally available as non-beta as part of Ubuntu Pro
    - fix: check if livepatch has already fixed a CVE before attempting a fix
    - jobs: new timer job to check if the release reached end of support
    - pro:
      + Ubuntu Pro is released as a product
      + make `pro` the recommended executable for the client
      + client, apt and motd messages updated/rewritten to show Pro
        information
      + base URL changed from /advantage to /pro
      + ESM services renamed as part of Pro
    - ros: released as a non-beta entitlement
    - security-status
      + does not require the --format flag anymore
      + human readable output added based on ubuntu-security-status
      + machine readable output contains CVEs fixed by Livepatch
...

This bug was fixed in the package ubuntu-advantage-tools - 27.11.2~20.04.1

---------------
ubuntu-advantage-tools (27.11.2~20.04.1) focal; urgency=medium

  * Backport new upstream release: (LP: #1991173) to focal

ubuntu-advantage-tools (27.11.2~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.2: (LP: #1991173)
    - esm: add the --beta flag back to esm-apps
    - messaging: show Ubuntu Pro beta message in apt output
    - security-status: don't show esm-apps information when the service is not
      enabled
    - ros: add the --beta flag back to ros and ros-updates

ubuntu-advantage-tools (27.11.1~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.1: (LP: #1990907)
    - Fix release upgrade when ESM packages are installed
      + d/postinst: remove series information from the APT preferences template
      + esm: remove series information from the APT preferences file

ubuntu-advantage-tools (27.11~22.10.1) kinetic; urgency=medium

  * d/control:
    - Update VCS references
  * d/links:
    - add usr/bin/pro as an alias to ubuntu-advantage
  * d/postinst:
    - include root_mode parameter when creating UAConfig instances
    - change calls to add_notice to notice_file.add
    - create public machine-token file if it does not exist
  * New upstream release 27.11 (LP: #1989279)
    - api:
      + new `pro api` command to access the public client API
      + 'version' endpoint returning version information
      + 'should auto attach' endpoint informing if a system should run
        auto-attach on startup
      + 'full auto attach' endpoint performing auto-attach
      + 'magic attach' endpoints for the Magic Attach flow
    - auto-attach:
      + better errors for invalid pro images (GH: #2180, #1833)
      + don't detach on already auto-attached instances
      + no-op when ubuntu-advantage information is present on cloud-init
        userdata
      + change systemd unit to run after cloud-config
    - cli:
      + cli: better error message on unrecognized flags (GH: #672)
    - collect-logs:
      + can now be executed as a non-root user
      + is executed automatically and result is appended when using apport to
        report a bug
    - docs: now formatted to be built with sphinx, and published in readthedocs
    - enable:
      + new access-only flag for usecases where auto-install is undesired
      + fix apt auth line replacement (LP: #1985863)
    - esm-apps: generally available as non-beta as part of Ubuntu Pro
    - fix: check if livepatch has already fixed a CVE before attempting a fix
    - jobs: new timer job to check if the release reached end of support
    - pro:
      + Ubuntu Pro is released as a product
      + make `pro` the recommended executable for the client
      + client, apt and motd messages updated/rewritten to show Pro
        information
      + base URL changed from /advantage to /pro
      + ESM services renamed as part of Pro
    - ros: released as a non-beta entitlement
    - security-status
      + does not require the --format flag anymore
      + human readable output added based on ubuntu-security-status
      + machine readable output contains CVEs fixed by Livepatch
  ...

This bug was fixed in the package ubuntu-advantage-tools - 27.11.2~22.04.1

---------------
ubuntu-advantage-tools (27.11.2~22.04.1) jammy; urgency=medium

  * Backport new upstream release: (LP: #1991173) to jammy

ubuntu-advantage-tools (27.11.2~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.2: (LP: #1991173)
    - esm: add the --beta flag back to esm-apps
    - messaging: show Ubuntu Pro beta message in apt output
    - security-status: don't show esm-apps information when the service is not
      enabled
    - ros: add the --beta flag back to ros and ros-updates

ubuntu-advantage-tools (27.11.1~22.10.1) kinetic; urgency=medium

  * New upstream release 27.11.1: (LP: #1990907)
    - Fix release upgrade when ESM packages are installed
      + d/postinst: remove series information from the APT preferences template
      + esm: remove series information from the APT preferences file

ubuntu-advantage-tools (27.11~22.10.1) kinetic; urgency=medium

  * d/control:
    - Update VCS references
  * d/links:
    - add usr/bin/pro as an alias to ubuntu-advantage
  * d/postinst:
    - include root_mode parameter when creating UAConfig instances
    - change calls to add_notice to notice_file.add
    - create public machine-token file if it does not exist
  * New upstream release 27.11 (LP: #1989279)
    - api:
      + new `pro api` command to access the public client API
      + 'version' endpoint returning version information
      + 'should auto attach' endpoint informing if a system should run
        auto-attach on startup
      + 'full auto attach' endpoint performing auto-attach
      + 'magic attach' endpoints for the Magic Attach flow
    - auto-attach:
      + better errors for invalid pro images (GH: #2180, #1833)
      + don't detach on already auto-attached instances
      + no-op when ubuntu-advantage information is present on cloud-init
        userdata
      + change systemd unit to run after cloud-config
    - cli:
      + cli: better error message on unrecognized flags (GH: #672)
    - collect-logs:
      + can now be executed as a non-root user
      + is executed automatically and result is appended when using apport to
        report a bug
    - docs: now formatted to be built with sphinx, and published in readthedocs
    - enable:
      + new access-only flag for usecases where auto-install is undesired
      + fix apt auth line replacement (LP: #1985863)
    - esm-apps: generally available as non-beta as part of Ubuntu Pro
    - fix: check if livepatch has already fixed a CVE before attempting a fix
    - jobs: new timer job to check if the release reached end of support
    - pro:
      + Ubuntu Pro is released as a product
      + make `pro` the recommended executable for the client
      + client, apt and motd messages updated/rewritten to show Pro
        information
      + base URL changed from /advantage to /pro
      + ESM services renamed as part of Pro
    - ros: released as a non-beta entitlement
    - security-status
      + does not require the --format flag anymore
      + human readable output added based on ubuntu-security-status
      + machine readable output contains CVEs fixed by Livepatch
  ...