Only "main" component enabled after install

I don't think is the intention, so thanks for the bug.

Curtin should be updating the default source.list in the image with any configuration changes. In this case, we're updating the URI to use a mirror if one is selected.

Can you collect two things?

1) During the install, switch to the debugging screen and capture the /etc/apt/sources.list of the installer environment

2) After install capture the /etc/apt/sources.list file

Please attach both.

during install:
deb cdrom:[Ubuntu-Server 18.04 LTS _Bionic Beaver_ - Release amd64 (20180722)]/ bionic main restricted
deb http://archive.ubuntu.com/ubuntu/ bionic main
deb http://security.ubuntu.com/ubuntu/ bionic-security main
deb http://archive.ubuntu.com/ubuntu/ bionic-updates main

After install, on the installed system:
deb http://br.archive.ubuntu.com/ubuntu bionic main
deb http://br.archive.ubuntu.com/ubuntu bionic-security main
deb http://br.archive.ubuntu.com/ubuntu bionic-updates main

I'm also going to attach the installer logs.

Summary:
 Subiquity can feed curtin a sources.list template and curtin will do the
 right thing. config for that is 'apt/config/sources_list' which then
 will then have rendered '$MIRROR', and '$RELEASE'
   https://git.launchpad.net/curtin/tree/examples/apt-source.yaml#n93

More info:
Curtin is working as designed here, and replacing the ubuntu mirrors
it finds in the /etc/apt/sources.list with the requested mirrors. Other
than that, it does not change the content.

You can see in the provided curtin.log:
   No custom template provided, fall back to modifymirrors in /etc/apt/sources.list on the target system

To see what curtin got *in*, I make an educated guess that the 'filesystem.squashfs' image contains the relevant /etc/apt/sources.list.

   $ iso=ubuntu-18.04-live-server-amd64.iso
   $ isoinfo -i "$iso" -J -R -x /casper/filesystem.squashfs > fs.squashfs
   $ unsquashfs -dest squashfs-root/ fs.squashfs /etc/apt/sources.list
   $ cat squashfs-root/etc/apt/sources.list
   deb http://archive.ubuntu.com/ubuntu/ bionic main
   deb http://security.ubuntu.com/ubuntu/ bionic-security main
   deb http://archive.ubuntu.com/ubuntu/ bionic-updates main

That differs from a ubuntu server squash image.

   $ wget http://cloud-images.ubuntu.com/daily/server/bionic/current/bionic-server-cloudimg-amd64.squashfs
   $ unsquashfs -dest server-squashfs-root/ \
       bionic-server-cloudimg-amd64.squashfs /etc/apt/sources.list
   $ wc -l server-squashfs-root/etc/apt/sources.list
   49 server-squashfs-root/etc/apt/sources.list
   $ grep -v "^#" server-squashfs-root/etc/apt/sources.list

   $ pastebinit server-squashfs-root/etc/apt/sources.list
   http://paste.ubuntu.com/p/nQvtTksHRs/

   $ grep -v "^#" server-squashfs-root/etc/apt/sources.list | grep .
   deb http://archive.ubuntu.com/ubuntu/ bionic main restricted
   deb http://archive.ubuntu.com/ubuntu/ bionic-updates main restricted
   deb http://archive.ubuntu.com/ubuntu/ bionic universe
   deb http://archive.ubuntu.com/ubuntu/ bionic-updates universe
   deb http://archive.ubuntu.com/ubuntu/ bionic multiverse
   deb http://archive.ubuntu.com/ubuntu/ bionic-updates multiverse
   deb http://archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse
   deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted
   deb http://security.ubuntu.com/ubuntu/ bionic-security universe
   deb http://security.ubuntu.com/ubuntu/ bionic-security multiverse

If you're wondering why curtin updates the existing sources.list, that
is to avoid having to know what the right content is. See bug 1513529
or bug 1177432 for examples of why we are not interested in knowing the
right content and maintaining that knowledge.

What I suggest is to have the images built with the correct data,
Then it will all "just work".

Yes, I guess this should be fixed in livecd-rootfs to make sure that filesystem.squashfs has the right thing.

i think this can be done with the change on how livecd rootfs is built - with or without universe.

Please make sure that you end up with the same content that exists in the cloud images.
We don't want arbitrary differences among ways of installing Ubuntu.

Status changed to 'Confirmed' because the bug affects multiple users.

This is blocking the daily install test: in only having main it prevents the install of the utah-client on the system and hence the post-install tests fail.

The last Cosmic ISO to pass and get published was from June 10, 2018. Since then the daily tests have all failed.

sources.list from June 10, 2018 ISO:
https://paste.ubuntu.com/p/FDdQTfhj88/

sources.list from Aug 10, 2018 ISO:
https://paste.ubuntu.com/p/63hGdmB7Ks/

(sorry for attachment noise had wrong bug number while working on a 'lp-attach-file' https://github.com/CanonicalLtd/uss-tableflip/blob/master/scripts/lp-attach-file).

There are several things I don't understand here.

One of them is what changed after June 10. The sources.list you pasted was clearly generated by cloud-init, not by livecd-rootfs or subiquity or curtin or anything else. So for some reason, on June 10, cloud-init rewrote sources.list either when the ISO booted or when the installed system booted.

Another thing I don't understand is why this breaks the daily install test for cosmic only; only main is enabled in bionic too but the tests pass there. Have some of utah's deps dropped out to universe in cosmic?

A final thing I don't understand is how to fix this. I think we need to set PREINSTALLED=true during the livefs build's config step, but I don't know what other consequences this would have.

Utah is a Python 2, with zero plan to bring it to 3, which has deps in universe.

This is what a sources.list looks like from the d-i installer:
https://paste.ubuntu.com/p/6m4ghXv6Gs/

This just happened to me, and reading through, I noticed this is triggered by cloud-init.

Is it possible that it's only biting people who install SSH keys with cloud-init, as I was mildly surprised to see it running as part of boot.

(Screenshot: https://s3.amazonaws.com/uploads.hipchat.com/190176/1597809/1iuhppYcWkjRHm7/upload.png )

For people looking for a one-line fix, here it is:

curl https://pastebin.freepbx.org/view/raw/fca35292 > /etc/apt/sources.list && apt-get update

That paste is a mirror of the paste in #16 without the requirements to login.

Note, that pastebin is for 18.10 - Not 18.04.1 LTS. To fix 18.04.1 LTS installation issues, use this sources.list:

curl https://pastebin.freepbx.org/view/raw/136f66ca > /etc/apt/sources.list && apt-get update

That is a mirror of https://gist.githubusercontent.com/jackw1111/d31140946901fab417131ff4d9ae92e3/raw/480bc68edae51e704114b0f4e256f543f25961af/sources.list which I shortened for ease of pasting.

This bug was fixed in the package livecd-rootfs - 2.535

---------------
livecd-rootfs (2.535) cosmic; urgency=medium

  * Disentangle enabling universe in the final image a little from having
    PREINSTALLED=true set and enable it for a live-server build.
    (LP: #1783129)

 -- Michael Hudson-Doyle <email address hidden> Fri, 24 Aug 2018 13:04:51 +1200

Hello Andreas, or anyone else affected,

Accepted livecd-rootfs into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.525.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

i also had this happen to my sever install. this is what was in sources.list by default:
https://paste.ubuntu.com/p/n8w6PfkQ3R/

i used post #27: https://bugs.launchpad.net/subiquity/+bug/1783129/comments/27 to add to my list:
https://paste.ubuntu.com/p/6KHg4QPXvF/

I have tested that this change works as designed, a system installed with the current bionic live server daily has the expected contents of /etc/apt/sources.list.

I haven't explicitly tested that all other bionic images are still working as designed and I'm not really sure how to. All the automated tests still seem to be passing.

Bionic live server dailies:
http://cdimage.ubuntu.com/ubuntu-server/bionic/daily-live/current/

Confirmed fixed in today's bionic live server image. I have universe and others enabled by default.

The verification of the Stable Release Update for livecd-rootfs has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package livecd-rootfs - 2.525.8

---------------
livecd-rootfs (2.525.8) bionic; urgency=medium

  * Disentangle enabling universe in the final image a little from having
    PREINSTALLED=true set and enable it for a live-server build.
    (LP: #1783129)
  * Fix live-server journald config snippet to actually disable journald rate
    limiting.

livecd-rootfs (2.525.7) bionic; urgency=medium

  * Disable journald rate limiting in the live-server live session.
    (LP: #1776891)

 -- Michael Hudson-Doyle <email address hidden> Tue, 28 Aug 2018 11:03:37 +1200

Brand new to the site so unsure if I am posting correctly

Having the same problem as reported by Kevin Golding on 2018-08-24 (Bug #1788835)

The message following "apt install krb5-user" is:
Package krb5-user is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

The file used is ubuntu-18.04.1-live-server-amd64.iso and was downloaded Thursday 20th September 2018

Is there something still to be done in the "subiquity" task of this bug? Sorry if I missed it in the comments above.

@at-fresh, please try the ISO from http://cdimage.ubuntu.com/ubuntu-server/bionic/daily-live/current/

While I'm not Andrew (@at-fresh), I can confirm that installing using the amd64 image at http://cdimage.ubuntu.com/ubuntu-server/bionic/daily-live/current/ provide the expected sources.list file.

It would be nice to update / refresh / replace the 18.04.1 release images using these, if this is an option (policy-wise), so that the average user will no longer run into this issue.

From the perspective of a long term (and appreciatively so) Ubuntu user, I am under the impression that, had the 18.04(.1) QA / release teams been granted sufficient resources, this and other (even more) serious issues in this LTS release should have been possible to detect and fix pre-release.

Beware that using daily images will also enabled the proposed repository, which some (me included) might find unexpected.

With the release of 18.04.2, we can finally close this bug.