No UID checks on rootfs updates

Keeping a copy on system-image.u.c will not help at all, as the IDs are assigned when building the live filesystem. We'd need the assignments to be actually in a package uploaded to the archive and installed in the image, and for that to be honoured by adduser --system, I think.

Colin, what I'm proposing is to post-process the livefs to reorder UIDs based on the system-image master copy; how would that not work?

Your proposed solution of a list of UIDs kept in a package and used by adduser seems more convergence-friendly.

we will need to use libnss-extrausers to manage the non-system users anyway on the readonly images, i wonder if we cant also use the extrausers password/group files for other users, migrate them from password to extrausers on first boot and watch additions/removals to the readonly file during upgrades (which we process if necessary). that would make the UIDs persistent in the writable space.

Has been discussed on IRC with Stéphane and Oliver. Stéphane and I believe the checking should be done at livefs build time by pre-populating the list of system users with a deterministic order.

This is a lot more reliable than having to fix things up in a boot hook (since in the event of problems we fail the image build, instead of failing the image upgrade), and also lets us fail the image build in the case of uncoordinated introduction of new system users.

My syslog stopped growing, noted owner change:

phablet@ubuntu-phablet:~$ ls -l /var/log
total 129012
-rw-r--r-- 1 root root 0 Apr 30 06:25 alternatives.log
-rw-r--r-- 1 root root 346 Apr 29 14:55 alternatives.log.1
-rw-r--r-- 1 root root 1476 Feb 23 2014 alternatives.log.2.gz
-rw-r----- 1 root adm 0 Sep 15 10:17 apport.log
-rw-r----- 1 root adm 866 Sep 15 09:43 apport.log.1
-rw-r----- 1 root adm 449 Sep 12 10:04 apport.log.2.gz
-rw-r----- 1 root adm 548 Sep 11 19:02 apport.log.3.gz
-rw-r----- 1 root adm 308 Sep 10 09:12 apport.log.4.gz
-rw-r----- 1 root adm 83 Sep 9 16:32 apport.log.5.gz
-rw-r----- 1 root adm 473 Sep 8 17:41 apport.log.6.gz
-rw-r----- 1 root adm 515 Sep 8 09:47 apport.log.7.gz
drwxr-xr-x 2 root root 4096 Sep 1 00:19 apt
-rw-r----- 1 usermetrics adm 599137 Aug 20 09:04 auth.log
-rw-r----- 1 usermetrics adm 143042 Jul 27 06:49 auth.log.1
-rw-r----- 1 usermetrics adm 10891 Jul 21 06:27 auth.log.2.gz
-rw-r----- 1 usermetrics adm 7086 Jul 13 07:08 auth.log.3.gz
-rw-r----- 1 usermetrics adm 2968 Jul 6 06:26 auth.log.4.gz
-rw-r--r-- 1 root root 65466 Feb 23 2014 bootstrap.log
-rw-rw---- 1 root utmp 0 Sep 1 00:19 btmp
-rw-rw---- 1 root utmp 0 Aug 1 06:25 btmp.1
-rw-r----- 1 root adm 68038 Sep 15 14:52 dmesg
-rw-r----- 1 root adm 49406 Sep 15 14:50 dmesg.0
-rw-r----- 1 root adm 17487 Sep 15 09:45 dmesg.1.gz
-rw-r----- 1 root adm 16090 Sep 15 09:43 dmesg.2.gz
-rw-r----- 1 root adm 19710 Sep 12 12:41 dmesg.3.gz
-rw-r----- 1 root adm 15836 Sep 12 12:39 dmesg.4.gz
-rw-r--r-- 1 root root 13392 Sep 5 10:05 dpkg.log
-rw-r--r-- 1 root root 21400 Aug 29 08:56 dpkg.log.1
-rw-r--r-- 1 root root 4885 Jul 25 14:46 dpkg.log.2.gz
-rw-r--r-- 1 root root 1116 Jun 20 14:53 dpkg.log.3.gz
-rw-r--r-- 1 root root 642 Jun 2 17:50 dpkg.log.4.gz
-rw-r--r-- 1 root root 166 Apr 29 14:55 dpkg.log.5.gz
-rw-r--r-- 1 root root 382 Apr 15 16:29 dpkg.log.6.gz
-rw-r--r-- 1 root root 57849 Feb 23 2014 dpkg.log.7.gz
-rw-r--r-- 1 root root 768288 Feb 23 2014 faillog
-rw-r--r-- 1 root root 976 Feb 23 2014 fontconfig.log
drwxr-xr-x 2 root root 4096 Feb 24 2014 fsck
drwxr-xr-x 2 root root 4096 Feb 24 2014 installer
-rw-r----- 1 usermetrics adm 0 Jul 27 07:47 kern.log
-rw-r----- 1 usermetrics adm 61050880 Jul 23 16:01 kern.log.1
-rw-r----- 1 usermetrics adm 13139240 Jul 21 06:35 kern.log.2.gz
-rw-r----- 1 usermetrics adm 10502851 Jul 13 07:22 kern.log.3.gz
-rw-r----- 1 usermetrics adm 6091087 Jul 6 06:27 kern.log.4.gz
-rw-rw-r-- 1 root utmp 9347504 Sep 15 15:12 lastlog
drwxr-xr-x 2 root root 4096 Sep 15 14:53 lightdm
drwxr-xr-x 2 root root 4096 Apr 9 16:30 lxc
-rw-r--r-- 1 root root 53174 May 17 04:34 pm-powersave.log
-rw-r--r-- 1 root root 145082 Apr 30 17:16 pm-powersave.log.1
-rw-r--r-- 1 r...

Status changed to 'Confirmed' because the bug affects multiple users.

The solution to this problem will be implemented through live-build hooks. Sadly the existing set of hooks doesn't quite match what we need, so this adds a new .chroot_early hook type which runs right after deboostrap allowing early mangling of the user/group database.

And now the matching livecd-rootfs change including two hooks, one running right after debootstrap which sets up the user and group database files with the expected values and then a second one making sure nothing got added during package installation.

Packages are up in ppa:stgraber/experimental, please do a test build with those, fix any maintainer scripts blowing up as a result of the pre-existing users/groups and then copy over to the archive.

This bug was fixed in the package livecd-rootfs - 2.245

---------------
livecd-rootfs (2.245) utopic; urgency=medium

  * Add two new hooks for Ubuntu Touch to setup sensible /etc/passwd,
    /etc/shadow, /etc/group and /etc/gshadow PRIOR to package installation
    to guarantee user/group ordering on the image and then to check for any
    unexpected change to those files. (LP: #1332538)

    Any change to either the initial set of users and groups or to the
    post-package-install set will now be fatal to the image and will require
    a manual update of the hardcoded user/group list contained in this new
    chroot_early hook.
  * Bump dependency on live-build accordingly.
  * Update the setup_user hook to also take care of gshadow.
 -- Stephane Graber <email address hidden> Mon, 22 Sep 2014 16:02:58 -0400

This bug was fixed in the package live-build - 3.0~a57-1ubuntu12

---------------
live-build (3.0~a57-1ubuntu12) utopic; urgency=medium

  * Add a new .chroot_early hook type which is run after debootstrap and
    prior to package installation. (LP: #1332538)
 -- Stephane Graber <email address hidden> Mon, 22 Sep 2014 15:53:38 -0400