[FFe] Please accept systemd 242 to Eoan

Bug #1843755 reported by Balint Reczey
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

Eoan currently has 241-7ubuntu1, Debian stable has 241 and Debian testing moved to 242 last week.
While version 241 is the safer choice for Eoan (from v241 and v242) since it is widely tested updating to v242 will allow us to carry fewer patches in Eoan and makes moving to the next release easier.

The proposed package is tested in Bileto and all tests are passing (except for a few unrelated failures):
https://bileto.ubuntu.com/#/ticket/3797

The final version will be 242-6ubuntu1 and I'm tidying up the changelog, too.

I plan merging 242-7, too, going forward but this will not need an FFe.

CHANGES WITH 242:

        * In .link files, MACAddressPolicy=persistent (the default) is changed
          to cover more devices. For devices like bridges, tun, tap, bond, and
          similar interfaces that do not have other identifying information,
          the interface name is used as the basis for persistent seed for MAC
          and IPv4LL addresses. The way that devices that were handled
          previously is not changed, and this change is about covering more
          devices then previously by the "persistent" policy.

          MACAddressPolicy=random may be used to force randomized MACs and
          IPv4LL addresses for a device if desired.

          Hint: the log output from udev (at debug level) was enhanced to
          clarify what policy is followed and which attributes are used.
          `SYSTEMD_LOG_LEVEL=debug udevadm test-builtin net_setup_link /sys/class/net/<name>`
          may be used to view this.

          Hint: if a bridge interface is created without any slaves, and gains
          a slave later, then now the bridge does not inherit slave's MAC.
          To inherit slave's MAC, for example, create the following file:
          ```
          # /etc/systemd/network/98-bridge-inherit-mac.link
          [Match]
          Type=bridge

          [Link]
          MACAddressPolicy=none
          ```

        * The .device units generated by systemd-fstab-generator and other
          generators do not automatically pull in the corresponding .mount unit
          as a Wants= dependency. This means that simply plugging in the device
          will not cause the mount unit to be started automatically. But please
          note that the mount unit may be started for other reasons, in
          particular if it is part of local-fs.target, and any unit which
          (transitively) depends on local-fs.target is started.

        * networkctl list/status/lldp now accept globbing wildcards for network
          interface names to match against all existing interfaces.

        * The $PIDFILE environment variable is set to point the absolute path
          configured with PIDFile= for processes of that service.

        * The fallback DNS server list was augmented with Cloudflare public DNS
          servers. Use `-Ddns-servers=` to set a different fallback.

        * A new special target usb-gadget.target will be started automatically
          when a USB Device Controller is detected (which means that the system
          is a USB peripheral).

        * A new unit setting CPUQuotaPeriodSec= assigns the time period
          relatively to which the CPU time quota specified by CPUQuota= is
          measured.

        * A new unit setting ProtectHostname= may be used to prevent services
          from modifying hostname information (even if they otherwise would
          have privileges to do so).

        * A new unit setting NetworkNamespacePath= may be used to specify a
          namespace for service or socket units through a path referring to a
          Linux network namespace pseudo-file.

        * The PrivateNetwork= setting and JoinsNamespaceOf= dependencies now
          have an effect on .socket units: when used the listening socket is
          created within the configured network namespace instead of the host
          namespace.

        * ExecStart= command lines in unit files may now be prefixed with ':'
          in which case environment variable substitution is
          disabled. (Supported for the other ExecXYZ= settings, too.)

        * .timer units gained two new boolean settings OnClockChange= and
          OnTimezoneChange= which may be used to also trigger a unit when the
          system clock is changed or the local timezone is
          modified. systemd-run has been updated to make these options easily
          accessible from the command line for transient timers.

        * Two new conditions for units have been added: ConditionMemory= may be
          used to conditionalize a unit based on installed system
          RAM. ConditionCPUs= may be used to conditionalize a unit based on
          installed CPU cores.

        * The @default system call filter group understood by SystemCallFilter=
          has been updated to include the new rseq() system call introduced in
          kernel 4.15.

        * A new time-set.target has been added that indicates that the system
          time has been set from a local source (possibly imprecise). The
          existing time-sync.target is stronger and indicates that the time has
          been synchronized with a precise external source. Services where
          approximate time is sufficient should use the new target.

        * "systemctl start" (and related commands) learnt a new
          --show-transaction option. If specified brief information about all
          jobs queued because of the requested operation is shown.

        * systemd-networkd recognizes a new operation state 'enslaved', used
          (instead of 'degraded' or 'carrier') for interfaces which form a
          bridge, bond, or similar, and an new 'degraded-carrier' operational
          state used for the bond or bridge master interface when one of the
          enslaved devices is not operational.

        * .network files learnt the new IgnoreCarrierLoss= option for leaving
          networks configured even if the carrier is lost.

        * The RequiredForOnline= setting in .network files may now specify a
          minimum operational state required for the interface to be considered
          "online" by systemd-networkd-wait-online. Related to this
          systemd-networkd-wait-online gained a new option --operational-state=
          to configure the same, and its --interface= option was updated to
          optionally also take an operational state specific for an interface.

        * systemd-networkd-wait-online gained a new setting --any for waiting
          for only one of the requested interfaces instead of all of them.

        * systemd-networkd now implements L2TP tunnels.

        * Two new .network settings UseAutonomousPrefix= and UseOnLinkPrefix=
          may be used to cause autonomous and onlink prefixes received in IPv6
          Router Advertisements to be ignored.

        * New MulticastFlood=, NeighborSuppression=, and Learning= .network
          file settings may be used to tweak bridge behaviour.

        * The new TripleSampling= option in .network files may be used to
          configure CAN triple sampling.

        * A new .netdev settings PrivateKeyFile= and PresharedKeyFile= may be
          used to point to private or preshared key for a WireGuard interface.

        * /etc/crypttab now supports the same-cpu-crypt and
          submit-from-crypt-cpus options to tweak encryption work scheduling
          details.

        * systemd-tmpfiles will now take a BSD file lock before operating on a
          contents of directory. This may be used to temporarily exclude
          directories from aging by taking the same lock (useful for example
          when extracting a tarball into /tmp or /var/tmp as a privileged user,
          which might create files with really old timestamps, which
          nevertheless should not be deleted). For further details, see:

          https://systemd.io/TEMPORARY_DIRECTORIES

        * systemd-tmpfiles' h line type gained support for the
          FS_PROJINHERIT_FL ('P') file attribute (introduced in kernel 4.5),
          controlling project quota inheritance.

        * sd-boot and bootctl now implement support for an Extended Boot Loader
          (XBOOTLDR) partition, that is intended to be mounted to /boot, in
          addition to the ESP partition mounted to /efi or /boot/efi.
          Configuration file fragments, kernels, initrds and other EFI images
          to boot will be loaded from both the ESP and XBOOTLDR partitions.
          The XBOOTLDR partition was previously described by the Boot Loader
          Specification, but implementation was missing in sd-boot. Support for
          this concept allows using the sd-boot boot loader in more
          conservative scenarios where the boot loader itself is placed in the
          ESP but the kernels to boot (and their metadata) in a separate
          partition.

        * A system may now be booted with systemd.volatile=overlay on the
          kernel command line, which causes the root file system to be set up
          an overlayfs mount combining the root-only root directory with a
          writable tmpfs. In this setup, the underlying root device is not
          modified, and any changes are lost at reboot.

        * Similar, systemd-nspawn can now boot containers with a volatile
          overlayfs root with the new --volatile=overlay switch.

        * systemd-nspawn can now consume OCI runtime bundles using a new
          --oci-bundle= option. This implementation is fully usable, with most
          features in the specification implemented, but since this a lot of
          new code and functionality, this feature should most likely not
          be used in production yet.

        * systemd-nspawn now supports various options described by the OCI
          runtime specification on the command-line and in .nspawn files:
          --inaccessible=/Inaccessible= may be used to mask parts of the file
          system tree, --console=/--pipe may be used to configure how standard
          input, output, and error are set up.

        * busctl learned the `emit` verb to generate D-Bus signals.

        * systemd-analyze cat-config may be used to gather and display
          configuration spread over multiple files, for example system and user
          presets, tmpfiles.d, sysusers.d, udev rules, etc.

        * systemd-analyze calendar now takes an optional new parameter
          --iterations= which may be used to show a maximum number of iterations
          the specified expression will elapse next.

        * The sd-bus C API gained support for naming method parameters in the
          introspection data.

        * systemd-logind gained D-Bus APIs to specify the "reboot parameter"
          the reboot() system call expects.

        * journalctl learnt a new --cursor-file= option that points to a file
          from which a cursor should be loaded in the beginning and to which
          the updated cursor should be stored at the end.

        * ACRN hypervisor and Windows Subsystem for Linux (WSL) are now
          detected by systemd-detect-virt (and may also be used in
          ConditionVirtualization=).

        * The behaviour of systemd-logind may now be modified with environment
          variables $SYSTEMD_REBOOT_TO_FIRMWARE_SETUP,
          $SYSTEMD_REBOOT_TO_BOOT_LOADER_MENU, and
          $SYSTEMD_REBOOT_TO_BOOT_LOADER_ENTRY. They cause logind to either
          skip the relevant operation completely (when set to false), or to
          create a flag file in /run/systemd (when set to true), instead of
          actually commencing the real operation when requested. The presence
          of /run/systemd/reboot-to-firmware-setup,
          /run/systemd/reboot-to-boot-loader-menu, and
          /run/systemd/reboot-to-boot-loader-entry, may be used by alternative
          boot loader implementations to replace some steps logind performs
          during reboot with their own operations.

        * systemctl can be used to request a reboot into the boot loader menu
          or a specific boot loader entry with the new --boot-load-menu= and
          --boot-loader-entry= options to a reboot command. (This requires a
          boot loader that supports this, for example sd-boot.)

        * kernel-install will no longer unconditionally create the output
          directory (e.g. /efi/<machine-id>/<kernel-version>) for boot loader
          snippets, but will do only if the machine-specific parent directory
          (i.e. /efi/<machine-id>/) already exists. bootctl has been modified
          to create this parent directory during sd-boot installation.

          This makes it easier to use kernel-install with plugins which support
          a different layout of the bootloader partitions (for example grub2).

        * During package installation (with `ninja install`), we would create
          symlinks for <email address hidden>, systemd-networkd.service,
          systemd-networkd.socket, systemd-resolved.service,
          remote-cryptsetup.target, remote-fs.target,
          systemd-networkd-wait-online.service, and systemd-timesyncd.service
          in /etc, as if `systemctl enable` was called for those units, to make
          the system usable immediately after installation. Now this is not
          done anymore, and instead calling `systemctl preset-all` is
          recommended after the first installation of systemd.

        * A new boolean sandboxing option RestrictSUIDSGID= has been added that
          is built on seccomp. When turned on creation of SUID/SGID files is
          prohibited.

        * The NoNewPrivileges= and the new RestrictSUIDSGID= options are now
          implied if DynamicUser= is turned on for a service. This hardens
          these services, so that they neither can benefit from nor create
          SUID/SGID executables. This is a minor compatibility breakage, given
          that when DynamicUser= was first introduced SUID/SGID behaviour was
          unaffected. However, the security benefit of these two options is
          substantial, and the setting is still relatively new, hence we opted
          to make it mandatory for services with dynamic users.

Revision history for this message
Steve Langasek (vorlon) wrote :

I do not think this FFe request provides sufficient rationale for why we would want to make an exception. (Neither Debian testing having a newer version, nor "carrying fewer patches", are a compelling reason.)

It also does not include any in-depth analysis of the risks, beyond saying that 241 is "safer". If there is no more detailed analysis of what has changed upstream in 242, then we should default to safer.

Changed in systemd (Ubuntu):
status: New → Incomplete
Revision history for this message
Gianfranco Costamagna (costamagnagianfranco) wrote :

I added in the description, the changelog to know what has been introduced in this release, something I would personally appreciate to have in my dailyjob is:
* systemd.volatile=overlay
* systemd-networkd-wait-online gained a new setting --any for waiting
* networkctl list/status/lldp now accept globbing wildcards for network

description: updated
description: updated
Revision history for this message
Steve Langasek (vorlon) wrote :

cut'n'pasting the changelog (especially with bad word wrapping) does not constitute 'analysis' and is not particularly consumable by the release team.

Revision history for this message
Balint Reczey (rbalint) wrote :

@vorlon: Please see the discussion in LP: #1841790.

Revision history for this message
Balint Reczey (rbalint) wrote :

@vorlon: If you don's see that convincing I don't know what you are looking for. 242 now is as well tested as 241 when it entered Eoan and it is still not too late in the cycle to fix/revert if something breaks.
Imo the most important aspect is preparing for 20.04 LTS and 242 in Eoan serves that better than 241.

Revision history for this message
Steve Langasek (vorlon) wrote :

That bug specifically calls out that 243 was removed because it introduced regressions upstream.

It is left unclear whether these regressions applied to 242.

Were all of the regressions in 243 identified by way of autopkgtests?

Revision history for this message
Balint Reczey (rbalint) wrote :

@vorlon: Those regressions are found in 243 RC1, I'm now preparing 243 final which has new regressions compared to 243 RC1. Many regressions introduced in 243 were reported to upstream and got fixed in 243 final, but not all of them.

There were two regression identified in 242 (regressing network-manager and netplan.io) and I made a minimal revert in 242-6ubuntu1 to avoid those. So regression-wise 242 seems to be clean as far as we can tell and I believe regressions introduced at upstream between v242 and v243 are not a factor in deciding on accepting 242, unless you propose going right to v243, but I would advise against that as it is not ready.

Revision history for this message
Balint Reczey (rbalint) wrote :

@vorlon: You can check in the Bileto ticket that the autopkgtests for 242 are all green (except for a single ostree one).

Revision history for this message
Balint Reczey (rbalint) wrote :

@vorlon: I got reports against v243 via IRC, too, but those were expected since autopkgtests were also failing in releated areas.

Revision history for this message
Balint Reczey (rbalint) wrote :

@vorlon: I agree that upgrading to v242 comes with risks, but the risk fairly hard to assess exactly.

One measure can be the number of upstream stable backports and this is as follows:

rbalint@yogi:~/projects/deb/systemd.git$ git describe github-stable/v242-stable
v242-108-gf875dced33
rbalint@yogi:~/projects/deb/systemd.git$ git describe github-stable/v241-stable
v241-188-g1e19bcd55e

This measure does not give a clear guidance, since lack of backports can mean lack of regressions or the lack of interest to backport as well.

Fedora 30 is also on systemd 241 and Fedora rawhide already moved to 243:
https://pkgs.org/download/systemd

It is still a question of Ubuntu wanting to use Eoan to get feedback on v242 features and regressions in preparing for 20.04 LTS - or we want to have v241 in Eoan to have lower risk of regression and lower required effort to keep systemd in Eoan but having a bigger version bump at the Eoan -> 20.04 upgrade.

If the decision is to keep v241 for Eaon then we can merge changes from Debian stable and upstream v241 stable when needed and I move on to land a good v243 at the beginning of the 20.04 cycle.

If we decide to approve this FFe, I can upload 242 today, we can merge 242 changes from Debian but only in the next few weeks - maybe two months, until Debian moves on to v243 then we are on our own maintaining v243 debs with the help of upstream's v242-stable branch.

I can decrease the unknown risk by merging everything from v242-stable and doing a test round _before_ uploading 242 to Eoan, but I did not want to do that if the decision is sticking to v241 no matter how this testing run goes. So the third option would be conditionally accepting the FFe, only accepting v242 to Eoan if it includes all patches from v242-stable and the autopktests all pass. Picking this I need ~one more day to update the package and run the tests so it could land early next week.

tags: added: id-5d6fb9fb2ac215651795ec78
Revision history for this message
Balint Reczey (rbalint) wrote :

@vorlon Testing 243 goes better than expected, I may able to prepare it for FFe instead of 242 so Eoan would be in sync with Fedora.

Revision history for this message
Steve Langasek (vorlon) wrote :

as commented in person, I'm approving the v242 FFe, but if the target is v243 it is better to not upload 242 to the archive and wait for 243 instead.

Changed in systemd (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Balint Reczey (rbalint) wrote : Re: [FFe] Please accept systemd 243 to Eoan

The 243 package is prepared in https://bileto.ubuntu.com/#/ticket/3801.
According to Steve's out of band approval I'll upload that when every test passes.

summary: - [FFe] Please accept systemd 242 to Eoan
+ [FFe] Please accept systemd 243 to Eoan
Revision history for this message
Balint Reczey (rbalint) wrote :

Unfortunately Bileto reported a few tests as as always failing instead of new regressions. I have checked many, but not all of them before the upload and this is why a few regressions kept systemd in -proposed during the weekend.
https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#systemd

The original test result summary for the Bileto ticket is gone, but the test logs are still available:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-eoan-ci-train-ppa-service-3801/?format=plain

The 243 package in proposed does have only a few regressions and should not break other autopkgtests thus I try to fix the newly observed regressions quickly and if that does not work then ask for the removal of the package from the proposed pocket.

Revision history for this message
Balint Reczey (rbalint) wrote :

Since the time before Eoan Beta is too short I reverted to ship 242, it is now in -proposed.

summary: - [FFe] Please accept systemd 243 to Eoan
+ [FFe] Please accept systemd 242 to Eoan
Balint Reczey (rbalint)
Changed in systemd (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.