[SRU] upgrade from 14.04 to 16.04 fails with: ca-certificates 20170717~16.04.1 failed to install/upgrade: triggers looping, abandoned

Status changed to 'Confirmed' because the bug affects multiple users.

I'm new to ubuntu and just getting started.

dist-upgrade logs from 14.04 to 16.04

term log with dpkg debug mode enabled.

The root cause of the removal is ca-certificates-java in xenial Depends: openjdk-7-jre-headless (>= 7~u3-2.1.1~pre1-1) | java7-runtime-headless; java7-runtime-headless is provided by openjdk-8-jre-headless, but this will not cause apt to install openjdk-8-jre-headless on upgrade when openjdk-7-jre-headless is already installed.

openjdk-7-jre-headless is only available in Trusty, and has a dependency chain on an obsolete version of tzdata, so apt forces removal of tzdata-java -> openjdk-7-jre-headless -> ca-certificates-java

Switching ca-certificates-java in Xenial to explicitly prefer openjdk-8-jre-headless as the first alternative may be sufficient hint to the package manager for this upgrade to work correctly.

- Eric

I have tested an upgrade with the new Xenial test package of "ca-certificates-java" that I have prepare using a local repository approach (since the pkg is not yet in Ubuntu archive). So far switching to "openjdk-8-jre-headless" has good result after 3 different upgrade attempt. (trusty->xenial)

I'll talk to foundation team tomorrow for them to sign-off on the proposal change before I proceed with the final upload.

* Before upgrade (Trusty)
--
ii ca-certificates 20170717~14.04.1 all Common CA certificates
ii ca-certificates-java 20130815ubuntu1 all Common CA certificates (JKS keystore)
ii man-db 2.6.7.1-1ubuntu1 amd64 on-line manual pager
--

* After upgrade (xenial)
--
ii ca-certificates 20170717~16.04.1 all Common CA certificates
ii ca-certificates-java 20160321ubuntu1~16.04.1testb2 all Common CA certificates (JKS keystore)
ii man-db 2.7.5-1 amd64 on-line manual pager
--

"ca-certificates-java 20160321ubuntu1~16.04.1testb2" being my test package.

Regards,
Eric

Output of "/var/log/dist-upgrade/screenlog.0"

-----------------------------------
Preparing to unpack .../ca-certificates-java_20160321ubuntu1~16.04.1testb2_all.deb ...^M
Unpacking ca-certificates-java (20160321ubuntu1~16.04.1testb2) over (20130815ubuntu1) ...^M
dpkg: openjdk-7-jre-headless:amd64: dependency problems, but removing anyway as you requested:^M
 ca-certificates-java depends on openjdk-8-jre-headless | java7-runtime-headless; however:^M
  Package openjdk-8-jre-headless is not installed.^M
  Package java7-runtime-headless is not installed.^M
  Package openjdk-7-jre-headless:amd64 which provides java7-runtime-headless is to be removed.^M
^M
(Reading database ... ^M(Reading database ... 5%^M(Reading database ... 10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading database ... 25%^M(Reading database ... 30%^M(Reading database ... 35%^M(Reading database ... 40%^M(Reading database ... 45%^M(Reading database ... 50%^M(Reading database ... 55%^M(Reading database ... 60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Reading database ... 75%^M(Reading database ... 80%^M(Reading database ... 85%^M(Reading database ... 90%^M(Reading database ... 95%^M(Reading database ... 100%^M(Reading database ... 53541 files and directories currently installed.)^M
Removing openjdk-7-jre-headless:amd64 (7u151-2.6.11-2ubuntu0.14.04.1) ...^M
Removing tzdata-java (2017c-0ubuntu0.14.04) ...^M
(Reading database ... ^M(Reading database ... 5%^M(Reading database ... 10%^M(Reading database ... 15%^M(Reading database ... 20%^M(Reading database ... 25%^M(Reading database ... 30%^M(Reading database ... 35%^M(Reading database ... 40%^M(Reading database ... 45%^M(Reading database ... 50%^M(Reading database ... 55%^M(Reading database ... 60%^M(Reading database ... 65%^M(Reading database ... 70%^M(Reading database ... 75%^M(Reading database ... 80%^M(Reading database ... 85%^M(Reading database ... 90%^M(Reading database ... 95%^M(Reading database ... 100%^M(Reading database ... 52927 files and directories currently installed.)^M
-----------------------------------

- Eric

Would not it make sense to depend on default-jre-headless instead of a specific version of the jdk? This way you always depend on the jdk available on the release.

The "default-jre-headless" might be a good approach to look at to avoid to have to maintain Debian/Ubuntu ca-certificates-java package for situation like this or others possible scenarios.

IMHO, I think it's better to stick with the actual fix for now and maybe submitting a bug to Debian (if no already existing) to consider the approach mentioned by jibel in comment #14.

Thoughts ?

- Eric

If I compare a pure installation via apt-get install ca-certificates-java in Xenial (no upgrade involve) I see a behaviour change between the package currently in the archive and the one I have fix in a PPA.

--------------------------------
# Current ca-certificates-java (from archive) #
It is installaing "openjdk-9-jre-headless"

sudo apt-get install ca-certificates-java
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpcsclite1 libxi6 libxrender1 libxtst6 openjdk-9-jre-headless
  x11-common

# ca-certificates-java fix to depend on openjdk-8
It is now installing "openjdk-8-jre-headless"

sudo apt-get install ca-certificates-java
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libfontconfig1 libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpcsclite1
  libxi6 libxrender1 libxtst6 openjdk-8-jre-headless x11-common

# rmadison openjdk-9-jre-headless --suite=xenial
 openjdk-9-jre-headless | 9~b114-0ubuntu1 | xenial/universe | amd64, i386

# rmadison openjdk-8-jre-headless --suite=xenial
 openjdk-8-jre-headless | 8u77-b03-3ubuntu3 | xenial | amd64, arm64, armhf, i386, powerpc, ppc64el, s390x
--------------------------------

This might have an impact on user already having ca-certificates-java with openjdk-9 at next package upgrade.

I consider that using the package found in main (openjdk-8) is IMHO a better approach than using the universe (openjdk-9), but while this may affect current ca-certificates-java user. I wonder if we should put the xenial package to depen on openjdk-9.

I'll look with foundation team.

- Eric

I have quickly test, and haven't had any problem to upgrade from actual package of "ca-certificates-java" from the archive to the "ca-certificates-java" (including the openjdk-8 fix) on my PPA.

Seems like in this case openjdk-9 remain and openjdk-8 is discard. Keeping the most recent openjdk version installed.

But I would gladly appreciate foundation team review to double-check.

# sudo apt-get install ca-certificates-java -y
--------------------------
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
  ca-certificates-java
1 upgraded, 0 newly installed, 0 to remove and 9 not upgraded.
Need to get 14.5 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://ppa.launchpad.net/slashd/sf164025/ubuntu xenial/main amd64 ca-certificates-java all 20160321ubuntu1~16.04.1ppab1 [14.5 kB]
Fetched 14.5 kB in 0s (50.9 kB/s)
(Reading database ... 54312 files and directories currently installed.)
Preparing to unpack .../ca-certificates-java_20160321ubuntu1~16.04.1ppab1_all.deb ...
Unpacking ca-certificates-java (20160321ubuntu1~16.04.1ppab1) over (20160321) ...
Processing triggers for ca-certificates (20170717~16.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...

done.
done.
Setting up ca-certificates-java (20160321ubuntu1~16.04.1ppab1) ...
Processing triggers for ca-certificates (20170717~16.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...

done.
done.

--------------------------

#dpkg
--------------------------
ii ca-certificates 20170717~16.04.1 all Common CA certificates
ii ca-certificates-java 20160321ubuntu1~16.04.1ppab1 all Common CA certificates (JKS keystore)
ii openjdk-9-jre-headless:amd64 9~b114-0ubuntu1 amd64 OpenJDK Java runtime, using Hotspot JIT (headless)
--------------------------

- Eric

# Discussion with foundation team.

<slangasek> slashd: interesting that this is a behavior difference; we actually *would* want it to pull in openjdk-8-jre-headless by default instead of -9- in xenial, since 9 is in universe and 8 is in main. It is a behavior change that we should call out as part of the SRU (i.e. "regression potential"), but I don't consider it a blocker for the SRU.

<slangasek> The debdiff for xenial looks fine, the question is the test case. Did you verify already that having this package available in apt sources fixes the upgrade failure?

<slangasek> then +1 from me

Uploaded for Xenial and Zesty.

Hello MadCatmkII, or anyone else affected,

Accepted ca-certificates-java into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ca-certificates-java/20160321ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello MadCatmkII, or anyone else affected,

Accepted ca-certificates-java into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ca-certificates-java/20161107ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[JUSTIFICATION XENIAL]
----------------------

The upgrade has been tested in 2 ways :
- Release upgrade using the fix package via a local archive.
- Release upgrade using the fix package via xenial-proposed.

In both cases, the upgrade went well installing "ca-certificates-java" version "20160321ubuntu1" which now depend on openjdk-8 instead of -7-, fixing the trigger failure.

* [Trusty]
ii ca-certificates 20170717~14.04.1
ii ca-certificates-java 20130815ubuntu1
ii openjdk-7-jre-headless:amd64 7u151-2.6.11-2ubuntu0.14.04.1

* [Xenial]
ii ca-certificates-java 20160321ubuntu1
rc openjdk-7-jre-headless:amd64 7u151-2.6.11-2ubuntu0.14.04.1
ii openjdk-8-jre-headless:amd64 8u151-b12-0ubuntu0.16.04.2

- Eric

[JUSTIFICATION ZESTY]
----------------------

Release upgrade using "ca-certificates-java" package from xenial-proposed (part of the current SRU) and upgraded to "ca-certificates-java" from zesty-proposed during the release upgrade.
To simulate the situation once both packages will land in $RELEASE-updates.

The upgrade went well installing "ca-certificates-java" version "20161107ubuntu1" which now depend on openjdk-8 instead of -7-, fixing the trigger failure.

[Xenial]
ii ca-certificates-java 20160321ubuntu1 <== Currently in xenial-proposed
ii openjdk-8-jre-headless:amd64 8u151-b12-0ubuntu0.16.04.2

[Zesty]
ii ca-certificates-java 20161107ubuntu1 <== Currently in zesty-proposed
ii openjdk-8-jre-headless:amd64 8u151-b12-0ubuntu0.17.04.2

- Eric

[JUSTIFICATION ZESTY] (Part 2)
----------------------
Release upgrade using "ca-certificates-java" package currently found in -updates and upgraded to zesty-proposed during the release upgrade.

The upgrade went well installing "ca-certificates-java" version "20161107ubuntu1" which now depend on openjdk-8 instead of -7-, fixing the trigger failure.

[Xenial]

ii ca-certificates-java 20160321 <== Currently in xenial-updates
ii openjdk-9-jre-headless:amd64 9~b114-0ubuntu1

(Note: This "ca-certificates-java" package version installs openjdk-9 part of 'universe' (as mentionned in [Regression potential] section) as oppose to its successor that will install openjdk-8 part of 'main'. This is not considered a blocker after a conversation with slangasek (SRU verification team).

[Zesty]

ii ca-certificates-java 20161107ubuntu1 <== Currently in zesty-proposed
ii openjdk-9-jre-headless:amd64 9~b161-1

To summarize the possible scenarios:

* If Xenial users has "ca-certificates-java" package lower than "20160321ubuntu1" installed, they will have openjdk-9- and it will follow during the release upgrade to Zesty.
* If Xenial users installs "ca-certificates-java" for the first time with version "20160321ubuntu1", they will have openjdk-8- and it will follow during the release upgrade to Zesty.
* If Xenial users already have "ca-certificates-java" intalled and upgrade to version "20160321ubuntu1", they will keep openjdk-9- and it will follow during the release upgrade to Zesty.

- Eric

This bug was fixed in the package ca-certificates-java - 20161107ubuntu1

---------------
ca-certificates-java (20161107ubuntu1) zesty; urgency=medium

  * Depend on openjdk-8 instead of openjdk-7. (LP: #1723198)

 -- Eric Desrochers <email address hidden> Fri, 15 Dec 2017 00:12:19 -0500

The verification of the Stable Release Update for ca-certificates-java has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package ca-certificates-java - 20160321ubuntu1

---------------
ca-certificates-java (20160321ubuntu1) xenial; urgency=medium

  * Depend on openjdk-8 instead of openjdk-7. (LP: #1723198)

 -- Eric Desrochers <email address hidden> Thu, 14 Dec 2017 15:38:26 -0500

To re-enforce the early release of the package, here's a comment that has been brought to my attention from a user impacted by this issue : "I have 3 VMs that were reproducing this 100%, but now they no longer can. This does appear to be fixed. Thanks for the diligence and quick turnaround!".