Add ubuntu-oci project for building OCI-ready tarballs with livecd-rootfs

Bug #1926732 reported by Thomas Bechtold
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
livecd-rootfs (Ubuntu)
Fix Released
Undecided
Thomas Bechtold
Bionic
Fix Released
Undecided
Thomas Bechtold
Focal
Fix Released
Undecided
Thomas Bechtold
Groovy
Fix Released
Undecided
Thomas Bechtold
Hirsute
Fix Released
Undecided
Thomas Bechtold

Bug Description

[Impact]

Currently the ubuntu-base livecd-rootfs project is used to build tarballs that are the base for building docker/OCI images.
The tarballs produced with the ubuntu-base project are modified externally (see https://github.com/tianon/docker-brew-ubuntu-core/blob/master/update.sh ) to create the "official" ubuntu images on dockerhub.

When including the ubuntu-oci project into livecd-rootfs, we can build tarballs that already contain the changes that are currently done externally. This has multiple advantages:

1) a Dockerfile using that tarball would no longer have to modify anything (means less layers)
2) publishing new OCI images no longer depends on the external dockerhub image creation. Currently eg. the AWS ECR ubuntu containers depend on the containers from dockerhub. That would be no longer the case with this change
3) Possible faster reaction on CVEs. no longer depending on external processes.

[Test Plan]

1) build ubuntu-oci project
a) Build a test ubuntu-oci tarball with ubuntu-old-fashioned:

./scripts/ubuntu-bartender/ubuntu-bartender --no-cleanup -- --series impish --image-format plain --project ubuntu-oci --subproject minimized

b) create a Dockerfile using the tarball created in step 1):

cat << EOF > Dockerfile
FROM scratch
ADD ubuntu-impish-oci-cloudimg-amd64-root.tar.gz /
CMD ["/bin/bash"]
EOF

c) build the container

docker build .

d) run something in the container:

docker run -it $container-id uname -a

e) check the size of the docker image and compare with the image for the same series available on dockerhub

2) build another project (ubuntu-base minimized)
a) Build a test ubuntu-base minimized project with ubuntu-old-fashioned to make sure the build does not break

[Where problems could occur]

This change could break other livecd-rootfs projects which might lead to failed builds. But beside that, there is nothing I can think of that would be affected by this change.

[Other Info]

When this is accepted, there will be livefs builds available at https://launchpad.net/~cloud-images-release-managers/+livefs/ubuntu/$SERIES/ubuntu-oci

Related branches

Changed in livecd-rootfs (Ubuntu Hirsute):
status: New → In Progress
assignee: nobody → Thomas Bechtold (toabctl)
Changed in livecd-rootfs (Ubuntu Groovy):
status: New → In Progress
assignee: nobody → Thomas Bechtold (toabctl)
Changed in livecd-rootfs (Ubuntu Focal):
status: New → In Progress
Changed in livecd-rootfs (Ubuntu Bionic):
status: New → In Progress
Changed in livecd-rootfs (Ubuntu Focal):
assignee: nobody → Thomas Bechtold (toabctl)
Changed in livecd-rootfs (Ubuntu Bionic):
assignee: nobody → Thomas Bechtold (toabctl)
description: updated
Changed in livecd-rootfs (Ubuntu Bionic):
status: In Progress → Fix Committed
Changed in livecd-rootfs (Ubuntu Focal):
status: In Progress → Fix Committed
Changed in livecd-rootfs (Ubuntu Groovy):
status: In Progress → Fix Committed
Changed in livecd-rootfs (Ubuntu Hirsute):
status: In Progress → Fix Committed
Changed in livecd-rootfs (Ubuntu):
status: New → In Progress
Changed in livecd-rootfs (Ubuntu):
status: In Progress → Fix Released
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Thomas, or anyone else affected,

Accepted livecd-rootfs into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.719.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed verification-needed-hirsute
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Thomas, or anyone else affected,

Accepted livecd-rootfs into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.694.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed-groovy
tags: added: verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Thomas, or anyone else affected,

Accepted livecd-rootfs into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.664.21 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (livecd-rootfs/2.664.21)

All autopkgtests for the newly accepted livecd-rootfs (2.664.21) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

livecd-rootfs/2.664.21 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#livecd-rootfs

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

description: updated
Revision history for this message
Thomas Bechtold (toabctl) wrote :

hirsute verification successful.

I changed the testing plan given that using the package from proposed with ubuntu-old-fashioned is not that easy.

I did now use launchpad itself to build (see https://code.launchpad.net/~toabctl/+livefs/ubuntu/hirsute/proposed):

- hirsute ubuntu-oci build: https://launchpadlibrarian.net/538316605/buildlog_ubuntu_hirsute_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz
- hirsute ubuntu-base build: https://launchpadlibrarian.net/538319522/buildlog_ubuntu_hirsute_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz

Testing the docker container tarball:

$ cat << EOF > Dockerfile
FROM scratch
ADD hirsute-minimal-cloudimg-amd64-root.tar.gz /
CMD ["/bin/bash"]
EOF
$ docker build .
Sending build context to Docker daemon 87.33MB
Step 1/3 : FROM scratch
 --->
Step 2/3 : ADD hirsute-minimal-cloudimg-amd64-root.tar.gz /
 ---> ba9d835e917e
Step 3/3 : CMD ["/bin/bash"]
 ---> Running in ca61b1822d7e
Removing intermediate container ca61b1822d7e
 ---> 62da9ad33be3
Successfully built 62da9ad33be3

$ docker run -it 62da9ad33be3 uname -a
Linux 2a455f31570a 5.11.0-16-generic #17-Ubuntu SMP Wed Apr 14 20:12:43 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Size of hirsute-minimal-cloudimg-amd64-root.tar.gz is 28449136 which is good.

tags: added: verification-done-hirsute
removed: verification-needed-hirsute
Revision history for this message
Thomas Bechtold (toabctl) wrote :

groovy verification successful.

I changed the testing plan given that using the package from proposed with ubuntu-old-fashioned is not that easy.

I did now use launchpad itself to build (see https://code.launchpad.net/~toabctl/+livefs/ubuntu/groovy/proposed):

- hirsute ubuntu-oci build: https://launchpadlibrarian.net/538319776/buildlog_ubuntu_groovy_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz
- hirsute ubuntu-base build: https://launchpadlibrarian.net/538330773/buildlog_ubuntu_groovy_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz

Testing the docker container tarball:

$ cat << EOF > Dockerfile
FROM scratch
ADD groovy-minimal-cloudimg-amd64-root.tar.gz /
CMD ["/bin/bash"]
EOF
$ docker build .
Sending build context to Docker daemon 87.33MB
Step 1/3 : FROM scratch
 --->
Step 2/3 : ADD groovy-minimal-cloudimg-amd64-root.tar.gz /
 ---> e188f5d7b5cf
Step 3/3 : CMD ["/bin/bash"]
 ---> Running in 739610b46528
Removing intermediate container 739610b46528
 ---> 564bda04f793
Successfully built 564bda04f793
$ docker run -it 564bda04f793 uname -a
Linux a7ca765289f1 5.11.0-16-generic #17-Ubuntu SMP Wed Apr 14 20:12:43 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Size of groovy-minimal-cloudimg-amd64-root.tar.gz is 30441216 which is good.

tags: added: verification-done-groovy
removed: verification-needed-groovy
Revision history for this message
Thomas Bechtold (toabctl) wrote :

focal verification successful.

I changed the testing plan given that using the package from proposed with ubuntu-old-fashioned is not that easy.

I did now use launchpad itself to build (see https://code.launchpad.net/~toabctl/+livefs/ubuntu/focal/proposed):

- focal ubuntu-oci build: https://code.launchpad.net/~toabctl/+livefs/ubuntu/focal/proposed/+build/274969/+files/buildlog_ubuntu_focal_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz
- focal ubuntu-base build: https://code.launchpad.net/~toabctl/+livefs/ubuntu/focal/proposed/+build/274966/+files/buildlog_ubuntu_focal_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz
Testing the docker container tarball:

$ cat << EOF > Dockerfile
FROM scratch
ADD focal-minimal-cloudimg-amd64-root.tar.gz /
CMD ["/bin/bash"]
EOF

$ docker build .
Sending build context to Docker daemon 27.73MB
Step 1/3 : FROM scratch
 --->
Step 2/3 : ADD focal-minimal-cloudimg-amd64-root.tar.gz /
 ---> ec62f2828d83
Step 3/3 : CMD ["/bin/bash"]
 ---> Running in 9b9bdbbe6cf9
Removing intermediate container 9b9bdbbe6cf9
 ---> 2f88a0922525
Successfully built 2f88a0922525

$ docker run -it 2f88a0922525 uname -a
Linux 1086ac358e49 5.11.0-16-generic #17-Ubuntu SMP Wed Apr 14 20:12:43 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Size of focal-minimal-cloudimg-amd64-root.tar.gz is 27730133 which is good.

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Thomas, or anyone else affected,

Accepted livecd-rootfs into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.525.53 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed-bionic
Revision history for this message
Thomas Bechtold (toabctl) wrote :

bionic verification successful.

I changed the testing plan given that using the package from proposed with ubuntu-old-fashioned is not that easy.

I did now use launchpad itself to build (see https://code.launchpad.net/~toabctl/+livefs/ubuntu/bionic/proposed):

- bionic ubuntu-oci build: https://code.launchpad.net/~toabctl/+livefs/ubuntu/bionic/proposed/+build/275398/+files/buildlog_ubuntu_bionic_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz
- bionic ubuntu-base build: https://code.launchpad.net/~toabctl/+livefs/ubuntu/bionic/proposed/+build/275412/+files/buildlog_ubuntu_bionic_amd64_amd64-minimized-all_proposed_BUILDING.txt.gz

Testing the docker container tarball:

$ cat << EOF > Dockerfile
FROM scratch
ADD bionic-minimal-cloudimg-amd64-root.tar.gz /
CMD ["/bin/bash"]
EOF

$ docker build .
Sending build context to Docker daemon 26.05MB
Step 1/3 : FROM scratch
 --->
Step 2/3 : ADD bionic-minimal-cloudimg-amd64-root.tar.gz /
 ---> 77389ea55ba2
Step 3/3 : CMD ["/bin/bash"]
 ---> Running in ed79500badff
Removing intermediate container ed79500badff
 ---> 5cd1b3e0d127
Successfully built 5cd1b3e0d127
$ docker run -it 5cd1b3e0d127 uname -a
Linux b4ecacff15c4 5.11.0-16-generic #17-Ubuntu SMP Wed Apr 14 20:12:43 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Size of bionic-minimal-cloudimg-amd64-root.tar.gz is 26047357 which is good.

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Changed in livecd-rootfs (Ubuntu):
assignee: nobody → Thomas Bechtold (toabctl)
Revision history for this message
Steve Langasek (vorlon) wrote : Update Released

The verification of the Stable Release Update for livecd-rootfs has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.719.1

---------------
livecd-rootfs (2.719.1) hirsute; urgency=medium

  [ Thomas Bechtold ]
  * Add a new ubuntu-oci project that contains the customizations currently
    performed downstream for the official Ubuntu images on dockerhub.
    (LP: #1926732)

 -- Michael Hudson-Doyle <email address hidden> Tue, 11 May 2021 11:34:58 +1200

Changed in livecd-rootfs (Ubuntu Hirsute):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.694.5

---------------
livecd-rootfs (2.694.5) groovy; urgency=medium

  [ Thomas Bechtold ]
  * Add a new ubuntu-oci project that contains the customizations currently
    performed downstream for the official Ubuntu images on dockerhub.
    (LP: #1926732)

 -- Michael Hudson-Doyle <email address hidden> Tue, 11 May 2021 11:37:12 +1200

Changed in livecd-rootfs (Ubuntu Groovy):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.664.21

---------------
livecd-rootfs (2.664.21) focal; urgency=medium

  [ Thomas Bechtold ]
  * Add a new ubuntu-oci project that contains the customizations currently
    performed downstream for the official Ubuntu images on dockerhub.
    (LP: #1926732)

 -- Michael Hudson-Doyle <email address hidden> Tue, 11 May 2021 11:38:16 +1200

Changed in livecd-rootfs (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.525.53

---------------
livecd-rootfs (2.525.53) bionic; urgency=medium

  [ Thomas Bechtold ]
  * Add a new ubuntu-oci project that contains the customizations currently
    performed downstream for the official Ubuntu images on dockerhub.
    (LP: #1926732)

 -- Michael Hudson-Doyle <email address hidden> Tue, 11 May 2021 11:39:46 +1200

Changed in livecd-rootfs (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.