focal chroots broken due to change in sources.list

Bug #1872163 reported by Tim Lunn
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
sbuild-launchpad-chroot (Ubuntu)
Fix Released
High
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Eoan
Won't Fix
Undecided
Unassigned
Focal
Fix Released
High
Unassigned
Groovy
Fix Released
High
Unassigned

Bug Description

----- Begin SRU info -----
[Impact]
Users of sbuild-launchpad-chroot cannot use chroots created with
sbuild-launchpad-chroot with a build-release of focal or groovy.
An attempt to do so fails during 'apt-get update' as the
/etc/apt/sources.list file that is created is not valid.

To recreate failure just try build and usage.

  $ sudo apt-get update
  $ sudo apt-get install sbuild-launchpad-chroot
  $ sudo sbuild-launchpad-chroot create --architecture=amd64 \
     --name=focal-amd64 --series=focal

Then get a dsc file to build. Any dsc will show the issue, this is only
to have a complete example.

  $ dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/cloud-utils/0.31-7-gd99b2d76-0ubuntu1/cloud-utils_0.31-7-gd99b2d76-0ubuntu1.dsc
  $ sbuild --dist=focal --arch=amd64 --arch-all cloud-utils_0.31-7-gd99b2d76-0ubuntu1.dsc
  ...
  +------------------------------------------------------------------------------+
  | Update chroot |
  +------------------------------------------------------------------------------+

  E: Type 'main' is not known on line 5 in source list /etc/apt/sources.list
  E: The list of sources could not be read.
  E: apt-get update failed

[Test Case]
To test:

 * install sbuild-launchpad-chroot.
 * Create chroots for bionic (old format) and focal (new format).
 * build a package in the chroot.

Because the mirror differs between amd64 and other arches, you should
also build for a non-amd64 arch.

The attached script will do all of the above.

I suggest running like:

  * manually enable -proposed
  * get a .dsc and associated files to build into current working dir.
  * ./test-sbuild-launchpad-chroot

Note that sbuild likely wont work properly in a container, so the
script will exit failure if detected.

[Regression Potential]
The most likely scenario for regression is in uers with custom scripts in
/etc/schroot.d/setup.d that make assumptions about the format of
/etc/apt/sources.list.

A user could have:
 1. added a script that ran *before* 90apt-sources that wrote or edited
    the /etc/apt/sources.list to modify a mirror or make other changes.

    Since we have changed the code to no longer attempt to edit
    sources.list the changes in 1 would likely be destroyed.

 2. added a script that ran *after* 90apt-sources that made similar changes
    and made assumption of the state of that file.

    Assumptions made about the format of this file may be broken and cause
    breakage of such scripts.

Lastly, a user could have relied on 90apt-sources to edit
/etc/apt/sources.list of schroots that were not created by
sbuild-launchpad-schroot. It seems unlikely to me that a user would have
desired this, but the change will no longer modify those schroots at all.
That is generally a safer/focused behavior. sbuild-launchpad-chroot
will now only modify schroots that it created (determined by
presense of launchpad.series in the schroot config).

Ultimately... it is somewhat unlikely that user-added scripts would have
worked with focal or groovy schroots.

[Other Info]
The change that caused this failure was in the /etc/apt/sources.list file that
is present in the launchpad schroot.

A summary of how sbuild-launchpad-chroot works. When a build is requested, the
following happens:
 1. The packagd file /etc/schroot/setup.d/01launchpad-chroot executes. It
    will query launchpad for a tarball to use as chroot, and possibly update
    the existing one.

 2. /etc/schroot/setup.d/90apt-sources executes and updates the
    exixting /etc/apt/sources.list inside the chroot. The goal of this is
    to enable the requested combination of
    suite (release, security, ... ) and
    component (main, restricted, universe ...)

The existing code is only able to handle a "one line" version of
/etc/apt/sources.list, that looks like the following:

   deb http://archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse

Focal and groovy chroots now a have a "complete" /etc/apt/sources.list
file as you would see in a lxc container or an installed system.

The change made was to more fully "own" the writing of
/etc/apt/sources.list. The previous code made a large assumption on the
format of /etc/apt/sources.list in the chroot, and tried to edit it based
on that assumption. When the format of the file changed, its ability to
edit the file failed.

The new code just declares the settings, which makes it much simpler.
The new code is not expected to be "general purpose". In order to
avoid breaking other schroot's, it is only enabled if this schroot
was set up by launchpad-schroot.

-- precise support --
The change upstream also dropped support for *running on* precise.
Ability to build for precise should not be changed.

-- feature added --
There is one feature added in this patch, now the user can
set the mirror that they would like to use by adding a setting:
 apt.default.mirror
To the /etc/schroot/chroot.d/name

For example:
  $ grep apt /etc/schroot.d/focal-amd64
  apt.enable=true
  apt.default.mirror=http://us.archive.ubuntu.com/ubuntu

If no value is set there, then code will use the mirror specified in the
first 'deb' line of /etc/apt/sources.list.
----- End SRU info -----

At some stage fairly recently my Focal schroots have broken due to the sources.list file that is now included. I will attach this file, however it looks like the standard distro source.list file vs the single liner version I have in my eoan schroots.

Reverting to a single liner similar to Eoan fixes this issue.

Not sure if this is an issue with this tool or with the downloaded chroot from launchpad changing

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: sbuild-launchpad-chroot 0.16
ProcVersionSignature: Ubuntu 5.4.0-21.25-generic 5.4.27
Uname: Linux 5.4.0-21-generic x86_64
ApportVersion: 2.20.11-0ubuntu26
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: GNOME
Date: Sat Apr 11 15:58:45 2020
InstallationDate: Installed on 2016-05-27 (1414 days ago)
InstallationMedia: Ubuntu-GNOME 16.04 LTS "Xenial Xerus" - Release amd64 (20160509)
PackageArchitecture: all
SourcePackage: sbuild-launchpad-chroot
UpgradeStatus: Upgraded to focal on 2019-11-18 (145 days ago)

Related branches

Revision history for this message
Tim Lunn (darkxst) wrote :
Revision history for this message
Tim Lunn (darkxst) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in sbuild-launchpad-chroot (Ubuntu):
status: New → Confirmed
Revision history for this message
Scott Moser (smoser) wrote :

Here is *a* fix/workaround that I'd expect to work for existing releases and focal

$ diff -u /etc/schroot/setup.d/90apt-sources.dist /etc/schroot/setup.d/90apt-sources
--- /etc/schroot/setup.d/90apt-sources.dist 2020-04-17 14:26:50.510749407 -0400
+++ /etc/schroot/setup.d/90apt-sources 2020-04-17 14:27:40.814787763 -0400
@@ -113,6 +113,13 @@
         APT_COMPONENTS="$comps"
     fi

+ # LAUNCHPAD_SERIES comes from schroot config 'launchpad.series'
+ if [ -n "$LAUNCHPAD_SERIES" ] &&
+ [ -n "$APT_COMPONENTS" -o -z "$APT_POCKETS" ]; then
+ echo "Setting one-line sources.list (LP: #1873507)"
+ echo "deb http://archive.ubuntu.com/ubuntu/ ${LAUNCHPAD_SERIES} main restricted universe multiverse" > "$sources"
+ fi
+
     if [ -n "$APT_POCKETS" ]; then
         echo "setting apt pockets to 'release $APT_POCKETS' in sources.list"
         awk -v pockets="$APT_POCKETS" '

Revision history for this message
Scott Moser (smoser) wrote :

(updated from the comment)
A workaround patch changing 90apt-sources.

tags: added: patch
Revision history for this message
Chad Smith (chad.smith) wrote :

Just put up a slightly different branch for review that should fix this. The issue on focal is that now we have empty lines in the full /etc/apt/sources.list. I don't think we have to limit it to one line do we?

Here is a changeset that ignores those empty lines, and still appends a new apt source line for each pocket defined.

https://code.launchpad.net/~chad.smith/ubuntu/+source/sbuild-launchpad-chroot/+git/sbuild-launchpad-chroot/+merge/382529

Scott Moser (smoser)
Changed in sbuild-launchpad-chroot (Ubuntu Bionic):
status: New → Confirmed
Changed in sbuild-launchpad-chroot (Ubuntu Eoan):
status: New → Confirmed
Changed in sbuild-launchpad-chroot (Ubuntu Focal):
status: New → Confirmed
Changed in sbuild-launchpad-chroot (Ubuntu Groovy):
status: Confirmed → In Progress
importance: Undecided → High
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sbuild-launchpad-chroot - 0.17

---------------
sbuild-launchpad-chroot (0.17) groovy; urgency=medium

  * Fix updating of apt sources.list for focal and groovy
    chroots (LP: #1872163)

 -- Scott Moser <email address hidden> Fri, 08 May 2020 10:10:53 -0400

Changed in sbuild-launchpad-chroot (Ubuntu Groovy):
status: In Progress → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :
description: updated
description: updated
Scott Moser (smoser)
description: updated
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I was seeing this in focal-unapproved and can confirm that the change (manually applied to my /etc/schroot/setup.d/90apt-sources) fixes the issue. Thanks!

I'd expect many people to be blocked by this, let's bump at least the focal prio up a bit to reflect that.

Changed in sbuild-launchpad-chroot (Ubuntu Focal):
importance: Undecided → High
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Tim, or anyone else affected,

Accepted sbuild-launchpad-chroot into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.16ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sbuild-launchpad-chroot (Ubuntu Focal):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-focal
Changed in sbuild-launchpad-chroot (Ubuntu Eoan):
status: Confirmed → Fix Committed
tags: added: verification-needed-eoan
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Tim, or anyone else affected,

Accepted sbuild-launchpad-chroot into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.15ubuntu0.19.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sbuild-launchpad-chroot (Ubuntu Bionic):
status: Confirmed → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Tim, or anyone else affected,

Accepted sbuild-launchpad-chroot into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sbuild-launchpad-chroot/0.14ubuntu0.18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Proposed package removed from archive

The version of sbuild-launchpad-chroot in the proposed pocket of Eoan that was purported to fix this bug report has been removed because the bugs that were to be fixed by the upload were not verified in a timely (105 days) fashion.

tags: removed: verification-needed-eoan
Changed in sbuild-launchpad-chroot (Ubuntu Eoan):
status: Fix Committed → Won't Fix
Scott Moser (smoser)
tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Scott Moser (smoser) wrote :
Revision history for this message
Scott Moser (smoser) wrote :

Verification log of focal.

Note 2 things:
 a.) recent update to the test script
 b.) if you look closely at the log you'll see '91smoser-schroot-setup' output.
That is from https://gist.github.com/smoser/14df5f0cd621e10d2282d7c90345e322
It should not affect the verification of this bug.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for sbuild-launchpad-chroot has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sbuild-launchpad-chroot - 0.16ubuntu0.20.04.1

---------------
sbuild-launchpad-chroot (0.16ubuntu0.20.04.1) focal; urgency=medium

  * Fix updating of apt sources.list for focal and groovy
    chroots (LP: #1872163)

 -- Chad Smith <email address hidden> Mon, 11 May 2020 17:33:46 -0600

Changed in sbuild-launchpad-chroot (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

This failed to build from source for bionic and subsequently has been removed from -proposed.

Changed in sbuild-launchpad-chroot (Ubuntu Bionic):
status: Fix Committed → Won't Fix
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sbuild-launchpad-chroot - 0.17ubuntu0.20.10.1~18.04.1

---------------
sbuild-launchpad-chroot (0.17ubuntu0.20.10.1~18.04.1) bionic; urgency=medium

  * Backport to bionic. (LP: #1905634)

sbuild-launchpad-chroot (0.17ubuntu0.20.10.1) groovy; urgency=medium

  * Add direct (Build-)Dependency on python3-ubuntutools instead of
    relying on the indirect dependency via ubuntu-dev-tools.
    (LP: #1852677)

 -- dann frazier <email address hidden> Wed, 25 Nov 2020 17:07:26 -0700

Changed in sbuild-launchpad-chroot (Ubuntu Bionic):
status: Won't Fix → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.