Ubuntu
unattended-upgrades package

[SRU] Unattended-upgrades silently does not apply updates when MinimalSteps is disabled and there are autoremovable kernels

Bug #1853861 reported by Balint Reczey
268
This bug affects 2 people
Affects Status Importance Assigned to Milestone
unattended-upgrades (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Disco
Fix Released
Undecided
Unassigned
Eoan
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

 * When autoremovable kernel packages are present on the system, there are updates to apply and Unattended-Upgrade::MinimalSteps is set to "false", the autoremovable kernel packages are not removed and the updates are not applied.
 * The root cause is u-u not cleaning the dirty cache between operations and also relying on having a cache with packages marked to be installed when applying updates in one shot.
 * The fix is clearing the cache between operations and marking packages before installing them in one shot.

[Test Case]

 * Install kernel-related packages, mark them as automatically installed to make them auto-removable ones.
 * Downgrade a few packages to a version lower than what is present in the security pocket.
 * Set Unattended-Upgrade::MinimalSteps to "false":
   # echo 'Unattended-Upgrade::MinimalSteps "false";' > /etc/apt/apt.conf.d/51unattended-upgrades-oneshot

 * Run u-u:
   # unattended-upgrade --verbose --debug

 * Observe fixed versions removing the kernel packages properly and also upgrading packages.

[Regression Potential]

 * The changes introduce marking packages to install/upgrade and clearing the cache more often. The added operations slow down u-u, but clearing the cache adds a few 100 milliseconds on typical hardware and marking upgradable packages is also in the same range.
 * Functional regressions are unlikely due to those changes since the fixes are present in 19.04 and later releases and the extensive autopkgtest also covers when upgrades are performed in minimal steps.

[Other Info]

 * While this bug has a security impact by holding back installation of security updates I don't recommend releasing the fix via the security pocket because this bug occurs only when the local configuration file of u-u is changed and u-u does not hold back upgrades with UCF-managed config file conflicts.
  See: https://github.com/mvo5/unattended-upgrades/issues/168

See original description
Balint Reczey (rbalint)
description: updated
summary: - [SRU] Unattended-upgrades does not apply updates when MinimalSteps is
- disabled and there are autoremovable kernels
+ [SRU] Unattended-upgrades silently does not apply updates when
+ MinimalSteps is disabled and there are autoremovable kernels
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in unattended-upgrades (Ubuntu):
status: New → Confirmed
Balint Reczey (rbalint)
information type: Public → Public Security
Balint Reczey (rbalint)
Changed in unattended-upgrades (Ubuntu):
status: Confirmed → Fix Released
Changed in unattended-upgrades (Ubuntu Disco):
status: New → Fix Released
Changed in unattended-upgrades (Ubuntu Eoan):
status: New → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Balint, or anyone else affected,

Accepted unattended-upgrades into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.13 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in unattended-upgrades (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed verification-needed-bionic
Changed in unattended-upgrades (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Balint, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Balint Reczey (rbalint) wrote :
Download full text (18.1 KiB)

Tested 1.1ubuntu1.18.04.13 on Bionic:

root@uu-non-minimal-lp-1838917:~# echo 'Unattended-Upgrade::MinimalSteps "false";' > /etc/apt/apt.conf.d/51unattended-upgrades-oneshot
root@uu-non-minimal-lp-1838917:~# yes no | apt autoremove
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  libdw1 libfreetype6 libmaxminddb0 linux-hwe-tools-4.18.0-13 linux-hwe-tools-4.18.0-14 linux-hwe-tools-4.18.0-15 linux-hwe-tools-4.18.0-16 linux-hwe-tools-4.18.0-17
  linux-hwe-tools-4.18.0-18 linux-tools-4.18.0-13-generic linux-tools-4.18.0-14-generic linux-tools-4.18.0-15-generic linux-tools-4.18.0-16-generic
  linux-tools-4.18.0-17-generic linux-tools-4.18.0-18-generic linux-tools-common
0 upgraded, 0 newly installed, 16 to remove and 18 not upgraded.
After this operation, 133 MB disk space will be freed.
Do you want to continue? [Y/n] Abort.
root@uu-non-minimal-lp-1838917:~# apt list --upgradable
Listing... Done
libnss-systemd/bionic-updates 237-3ubuntu10.33 amd64 [upgradable from: 237-3ubuntu10.31]
libpam-systemd/bionic-updates 237-3ubuntu10.33 amd64 [upgradable from: 237-3ubuntu10.31]
libpython3.6/bionic-updates 3.6.9-1~18.04 amd64 [upgradable from: 3.6.8-1~18.04.3]
libpython3.6-minimal/bionic-updates 3.6.9-1~18.04 amd64 [upgradable from: 3.6.8-1~18.04.3]
libpython3.6-stdlib/bionic-updates 3.6.9-1~18.04 amd64 [upgradable from: 3.6.8-1~18.04.3]
libsystemd0/bionic-updates 237-3ubuntu10.33 amd64 [upgradable from: 237-3ubuntu10.31]
libudev1/bionic-updates 237-3ubuntu10.33 amd64 [upgradable from: 237-3ubuntu10.31]
linux-tools-common/bionic-updates,bionic-security 4.15.0-72.81 all [upgradable from: 4.15.0-70.79]
python3-distupgrade/bionic-updates 1:18.04.36 all [upgradable from: 1:18.04.34]
python3.6/bionic-updates 3.6.9-1~18.04 amd64 [upgradable from: 3.6.8-1~18.04.3]
python3.6-minimal/bionic-updates 3.6.9-1~18.04 amd64 [upgradable from: 3.6.8-1~18.04.3]
snapd/bionic-updates 2.42.1+18.04 amd64 [upgradable from: 2.40+18.04]
sosreport/bionic-updates 3.6-1ubuntu0.18.04.4 amd64 [upgradable from: 3.6-1ubuntu0.18.04.3]
systemd/bionic-updates 237-3ubuntu10.33 amd64 [upgradable from: 237-3ubuntu10.31]
systemd-sysv/bionic-updates 237-3ubuntu10.33 amd64 [upgradable from: 237-3ubuntu10.31]
tshark/bionic-updates,bionic-security 2.6.10-1~ubuntu18.04.0 amd64 [upgradable from: 2.4.5-1]
ubuntu-release-upgrader-core/bionic-updates 1:18.04.36 all [upgradable from: 1:18.04.34]
udev/bionic-updates 237-3ubuntu10.33 amd64 [upgradable from: 237-3ubuntu10.31]
wireshark-common/bionic-updates,bionic-security 2.6.10-1~ubuntu18.04.0 amd64 [upgradable from: 2.4.5-1]
root@uu-non-minimal-lp-1838917:~# unattended-upgrade --verbose --debug
Initial blacklisted packages:
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Using (^linux-image-[0-9]+\.[0-9\.]+-.*|^linux-headers-[0-9]+\.[0-9\.]+-.*|^linux-image-extra-[0-9]+\.[0-9\.]+-.*|^linux-modules-[0-9]+\.[0-9\.]+-.*|^linux-modules-extra-[0-9]+\.[0-9\.]+-.*|^linux-signed-image-[0-9]+\.[0-9\.]+-.*|^linux-image-unsigned-[0-9]+\.[0-9\.]+-.*|^kfreebsd-image-[0-9...

tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
Balint Reczey (rbalint) wrote :
Download full text (13.4 KiB)

Verified 1.1ubuntu1.18.04.7~16.04.5 on Xenial:

root@uu-x-non-minimal-lp-1838917:~# apt list --upgradable
Listing... Done
dbus/xenial-proposed 1.10.6-1ubuntu3.5 amd64 [upgradable from: 1.10.6-1ubuntu3.4]
libdbus-1-3/xenial-proposed 1.10.6-1ubuntu3.5 amd64 [upgradable from: 1.10.6-1ubuntu3.4]
libglib2.0-0/xenial-proposed 2.48.2-0ubuntu4.5 amd64 [upgradable from: 2.48.2-0ubuntu4.4]
libglib2.0-data/xenial-proposed 2.48.2-0ubuntu4.5 all [upgradable from: 2.48.2-0ubuntu4.4]
libpam-modules/xenial-proposed 1.1.8-3.2ubuntu2.2 amd64 [upgradable from: 1.1.8-3.2ubuntu2.1]
libpam-modules-bin/xenial-proposed 1.1.8-3.2ubuntu2.2 amd64 [upgradable from: 1.1.8-3.2ubuntu2.1]
libpam-runtime/xenial-proposed 1.1.8-3.2ubuntu2.2 all [upgradable from: 1.1.8-3.2ubuntu2.1]
libpam-systemd/xenial-proposed 229-4ubuntu21.23 amd64 [upgradable from: 229-4ubuntu21.22]
libpam0g/xenial-proposed 1.1.8-3.2ubuntu2.2 amd64 [upgradable from: 1.1.8-3.2ubuntu2.1]
libsqlite3-0/xenial-updates,xenial-security 3.11.0-1ubuntu1.3 amd64 [upgradable from: 3.11.0-1ubuntu1.2]
libsystemd0/xenial-proposed 229-4ubuntu21.23 amd64 [upgradable from: 229-4ubuntu21.22]
libudev1/xenial-proposed 229-4ubuntu21.23 amd64 [upgradable from: 229-4ubuntu21.22]
linux-tools-common/xenial-updates,xenial-proposed,xenial-security 4.4.0-170.199 all [upgradable from: 4.4.0-169.198]
login/xenial-proposed 1:4.2-3.1ubuntu5.5 amd64 [upgradable from: 1:4.2-3.1ubuntu5.4]
passwd/xenial-proposed 1:4.2-3.1ubuntu5.5 amd64 [upgradable from: 1:4.2-3.1ubuntu5.4]
python3-distupgrade/xenial-updates 1:16.04.29 all [upgradable from: 1:16.04.27]
python3-update-manager/xenial-updates 1:16.04.17 all [upgradable from: 1:16.04.16]
snapd/xenial-proposed 2.42.1 amd64 [upgradable from: 2.40]
sosreport/xenial-updates 3.6-1ubuntu0.16.04.4 amd64 [upgradable from: 3.6-1ubuntu0.16.04.3]
systemd/xenial-proposed 229-4ubuntu21.23 amd64 [upgradable from: 229-4ubuntu21.22]
systemd-sysv/xenial-proposed 229-4ubuntu21.23 amd64 [upgradable from: 229-4ubuntu21.22]
tshark/xenial-updates,xenial-security 2.6.10-1~ubuntu16.04.0 amd64 [upgradable from: 2.0.2+ga16e22e-1]
ubuntu-core-launcher/xenial-proposed 2.42.1 amd64 [upgradable from: 2.40]
ubuntu-release-upgrader-core/xenial-updates 1:16.04.29 all [upgradable from: 1:16.04.27]
udev/xenial-proposed 229-4ubuntu21.23 amd64 [upgradable from: 229-4ubuntu21.22]
uidmap/xenial-proposed 1:4.2-3.1ubuntu5.5 amd64 [upgradable from: 1:4.2-3.1ubuntu5.4]
update-manager-core/xenial-updates 1:16.04.17 all [upgradable from: 1:16.04.16]
wireshark-common/xenial-updates,xenial-security 2.6.10-1~ubuntu16.04.0 amd64 [upgradable from: 2.0.2+ga16e22e-1]
root@uu-x-non-minimal-lp-1838917:~# apt autoremove
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  libdw1 libunwind8 linux-hwe-edge-tools-4.13.0-16 linux-hwe-edge-tools-4.13.0-17 linux-hwe-edge-tools-4.13.0-19 linux-tools-4.13.0-16-generic linux-tools-4.13.0-17-generic
  linux-tools-4.13.0-19-generic linux-tools-common
0 upgraded, 0 newly installed, 9 to remove and 27 not upgraded.
After this operation, 14.1 MB disk space will be freed.
Do you want to continue? [Y/n] n
Abort.
root@uu-x-non-min...

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Daniel Richard G. (skunk) wrote :

Thanks Balint. I've installed the bionic-proposed package, and have not observed any silently-failed upgrades as before (but of course verifying it in my use case is tantamount to proving a negative).

Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for unattended-upgrades has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.1ubuntu1.18.04.7~16.04.5

---------------
unattended-upgrades (1.1ubuntu1.18.04.7~16.04.5) xenial; urgency=medium

  * Fix non-minimal upgrades (LP: #1853861)
    - Mark packages for performing non-minimal upgrades
    - Clear cache after checking upgrades against the blacklist again.
      This fixes the issue when the dirty cache caused all packages to be
      upgraded in the first "minimal" step.
      Thanks to Paul Wise
    - Leave the cache clean when returning from calculate_upgradable_pkgs()
      When collecting upgradable packages the upgradable ones stayed in the
      cache and they were upgraded together even when unattended-upgrades
      was configured to perform upgrades in minimal steps.
      Thanks to Paul Wise
    - Clear cache after downloading packages

 -- Balint Reczey <email address hidden> Mon, 25 Nov 2019 16:23:06 +0100

Changed in unattended-upgrades (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.1ubuntu1.18.04.13

---------------
unattended-upgrades (1.1ubuntu1.18.04.13) bionic; urgency=medium

  * Fix non-minimal upgrades (LP: #1853861)
    - Mark packages for performing non-minimal upgrades
    - Clear cache after checking upgrades against the blacklist again.
      This fixes the issue when the dirty cache caused all packages to be
      upgraded in the first "minimal" step.
      Thanks to Paul Wise
    - Leave the cache clean when returning from calculate_upgradable_pkgs()
      When collecting upgradable packages the upgradable ones stayed in the
      cache and they were upgraded together even when unattended-upgrades
      was configured to perform upgrades in minimal steps.
      Thanks to Paul Wise
    - Clear cache after downloading packages

 -- Balint Reczey <email address hidden> Mon, 25 Nov 2019 16:23:06 +0100

Changed in unattended-upgrades (Ubuntu Bionic):
status: Fix Committed → Fix Released
Francis Ginther (fginther)
tags: added: id-5ddbf97425c5f4266c2b848a
Revision history for this message
sacarde (sacarde) wrote :

hi,
   in my kubuntu-18.04 I have:
"unattended-upgrades 1.1ubuntu1.18.04.14"

but when it upgrade some packages, it remove packages .deb
even if I entered "APT:Periodic::AutocleanInterval"0";" into
/etc/apt/apt.conf.d/20auto-upgrades

is this the same bug?

thank you

To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.