Users are prompted by ucf on upgrade from Trusty to Xenial

I will further increase this SRU bug with information that is lacking on from the original bug proposal:

[Impact]
During an upgrade from trusty to xenial, users will be prompted to make a decision regarding the diff on unattended-upgrades. This is not a good user experience, specially because the user can make an uninformed decision of keeping the old config file, which will make unattended-upgrades to not work as we expect.

[Test case]

To reproduce the isse, you can:

1. Launch a trusty vm
2. Perform a do-release-upgrade and observe that you will be prompted with the 50unattende-upgrades change

To verify that the error is fixed:

1. Launch a trusty vm
2. Import this ppa into the system:
   https://launchpad.net/~lamoura/+archive/ubuntu/unattended-upgrades-ppa
3. Configure do-release-upgrade to allow using third parties during upgrade
4. Run a do-release-upgrade
5. Verify the prompt is no longer there and that we end up with the expected 50unattended-upgrades
   config file

[Where problems could occur]

The changes is this package should only be surface during an upgrade operation. Because we are now delivering a new file to the system and configuring postinst to use it. Because of that, we believe this is the only scenario that could be affect in case of a regression is discovered in the package.

-------------------

Finally, if there is any additional information that I need to add to this SRU, just let me know and I will provide it

Fixed in 2.8.

Imagine a package shipping config file with contents "foo: true; bar: false". You might put the md5 of that in the ucf history, so that packaging that later changes to "foo: true; bar: true" knows that if the user has anything other than "foo: true; bar: false" then it is a customisation that should be preserved.

However, if packaging has iterated between all four possibilities, then list of all possible hashes defeats the purpose. Packaging will believe that whatever the user changed, it isn't a customisation because it was shipped previously by the packaging. Customisations will therefore get lost, contrary to our intention.

Therefore, it follows that the list of hashes shipped should be minimally what packaging did actually previously ship; anything more unnecessarily increases the risk of a collision with a user customisation. Since we don't support upgrade leaps beyond LTS-to-LTS, historical hashes can therefore be dropped to keep this minimal.

For this SRU, I'd therefore expect to see being added only the hashes involved in the specific issue being fixed - presumably only one - rather than 117. Usually the above never comes up, but I think it's relevant when there are 117, rather than the one or two entries that are common.

Similarly, for Hirsute, I'd have expected only the hashes since Focal and onwards to be included.

I did ask on IRC how to reproduce the hashes, and Balint pointed out that an explanation exists in README.source. Sorry I didn't see this before - the explanation is in Hirsute but not in this SRU upload. I think this reference to that from this bug comment is sufficient for reproducibility.

Perhaps I'm completely wrong here, in which case I'd appreciate a correction. Otherwise, please adjust the upload to minimally fix the specific issue here by only including the required hashes. I also suggest rethinking the Hirsute upload.

> For this SRU, I'd therefore expect to see being added only the hashes involved in the specific issue being fixed - presumably only one - rather than 117. Usually the above never comes up, but I think it's relevant when there are 117, rather than the one or two entries that are common.

> Similarly, for Hirsute, I'd have expected only the hashes since Focal and onwards to be included.

Agreed Robie, I think this project needs that release-specific set of md5sums to add. Lucas will limit this SRU to just the needed md5sum for trusty -> xenial to avoid prompting.

Ultimately I'd like to track an upstream bug for the md5sum filtering per release because there are 5 different versions of 50unattended-upgrades for Debian/Raspbian/SteamOS/Devuan and the tooling to generate these md5sum files should be distribution and release aware. This is risk that we don't want to inadvertently add to this SRU.

So, let's track that upstream feature as LP:# 1917677

I think you are right that a subset of md5sums should be

Thank you for this review Robie, we have now updated the package to only ship two md5sum. The latest version of 50unattended-upgrades of Trusty and Xenial.

I have already updated the MR with those changes and I Bryce is reviewing that changes before uploading the new package.

Hello Lucas, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Thanks for the review Robie,

I have tested the package on the proposed pocket by using this script:

------------
#!/bin/sh
set -x

series=trusty
name=$series-upgrade

multipass delete $name
multipass purge
multipass launch $series --name $name

multipass exec $name -- sudo sh -c "cat <<EOF >/etc/apt/sources.list.d/ubuntu-trusty-proposed.list
# Enable Ubuntu proposed archive
deb http://archive.ubuntu.com/ubuntu/ trusty-proposed restricted main multiverse universe
EOF"
multipass exec $name -- sudo apt-get update
multipass exec $name -- sudo apt-get upgrade -y
multipass exec $name -- sudo apt-get dist-upgrade -y
multipass exec $name -- do-release-upgrade

set +x
-------------

Using it, I can confirm that the unattended-upgrades prompt no longer appears.

If there is more testing that should be performed regarding this package, please let me know and I can perform them as well

Hi Robie, I just realized that I forgot to change the tag when I posted my test process :(

This is the reason why this package is not on the -updates pocket yet, right ?

The verification of the Stable Release Update for unattended-upgrades has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package unattended-upgrades - 1.1ubuntu1.18.04.7~16.04.7

---------------
unattended-upgrades (1.1ubuntu1.18.04.7~16.04.7) xenial; urgency=medium

  * data: add md5sum history file on the data folder (LP: #1915547)
    - This file contains the md5sum of the current xenial and trusty
      version of 50unattended-upgrades. This file will be used to avoid
      prompting users about unattended-upgrade changes when upgrading
      from trusty to xenial.
  * debian/postint: make ucf command reference the md5sum history file

 -- Lucas Moura <email address hidden> Wed, 10 Feb 2021 17:37:18 -0300