Add MODULES=dep initramfs configuration

Bug #1592684 reported by Martin Pitt
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
cloud-images
Invalid
Undecided
Unassigned
cloud-init (Ubuntu)
Triaged
Wishlist
Unassigned

Bug Description

During investigation of low-hanging fruit for improving boot speed, we noticed that we can shave off a large portion of kernel boot time with:

  echo 'MODULES=dep' | sudo tee /etc/initramfs-tools/conf.d/modules-dep.conf

Before:

$ systemd-analyze
Startup finished in 1.435s (kernel) + 1.601s (userspace) = 2.936s

-rw-r--r-- 1 root root 7.4M Jun 15 09:32 /boot/initrd.img-4.4.0-24-generic

After:

$ systemd-analyze
Startup finished in 762ms (kernel) + 1.706s (userspace) = 2.469s

-rw-r--r-- 1 root root 4.2M Jun 15 09:34 initrd.img-4.4.0-24-generic

We don't want to use MODULES=dep by default on real iron (server, desktop) to keep portability of root file systems (plugging them into different machines, changing hardware, etc.). But this is not relevant for virtual environments like QEMU or Hyper-V, so we can remove all these unused drivers from the initrd there. This saves time on unpacking the initrd and also avoids some useless hw probing.

So we propose that the cloud image build system should add this file.

Revision history for this message
Martin Pitt (pitti) wrote :

Adding an Ubuntu task so that this appears on https://bugs.launchpad.net/ubuntu/+bugs?field.tag=bootspeed .

tags: added: bootspeed
Changed in ubuntu:
status: New → Triaged
importance: Undecided → Wishlist
summary: - Use MODULES=dep initramfs configuration
+ Add MODULES=dep initramfs configuration
Revision history for this message
Stefan Bader (smb) wrote :

I would probably rather make it an easy configurable cloud-init option to keep the cloud-images generic until being configured for a specific guest or machine.

Revision history for this message
Martin Pitt (pitti) wrote :

Not a big fan of a configure option, I must say. Adding the MODULES=dep if "systemd-detect-virt --vm" sounds more appropriate to me.

Revision history for this message
Stefan Bader (smb) wrote :

Configure option might be ambiguous. What I meant was being part of the process that sets up an individual guest after it has been cloned of the common cloud-image.
The worry I would have is that the cloud-images themselves are created as some kind of VM which would change the modules option already in the preparation step and by that take away the option of using the images over a wide range of hardware.
To be configurable or not: at least for the time being I would not want to enforce this on all users without some kind of opting in. For that I was thinking of something (either keyword or shell snippet) in the cloud-init user data (not sure I am using the right terminology) which is used to set up the individual instance generated by using the generic image.

Revision history for this message
Martin Pitt (pitti) wrote :

Oh, I see what you mean. Right, this should not already be in the cloud images, but in cloud-init to configure the actual instance appropriate.

affects: ubuntu → cloud-init (Ubuntu)
Changed in cloud-images:
status: New → Invalid
Revision history for this message
Scott Moser (smoser) wrote :

fwiw, a virtual guest (such as cloud image) will only have -virtual kernel, which means loads less modules in the initramfs.

in the bug summary you stated:
"this is not relevant for virtual environments like QEMU or Hyper-V, so we can remove all these unused drivers from the initrd there."

That is not entirely true, as it is a desireable goal to snapshot on EC2 and move to Azure or similar hypervisor change that would be broken by your suggestion. Its possible I've misunderstood something, though.

Revision history for this message
Martin Pitt (pitti) wrote :

> it is a desireable goal to snapshot on EC2 and move to Azure or similar hypervisor change that would be broken by your suggestion

I think this is by far not the common case -- the vast majority of cloud instances in the world will only ever run on one guest ("cattle, not pets"). So for this corner case we might introduce a "portable:" boolean userdata option which does not minimize the initrd?

Revision history for this message
Martin Pitt (pitti) wrote :

On current 22.04 and kinetic cloud images my default initramfs is 102 (!) MB, and takes almost 20s to unpack from grub -- i.e. the time between booting the grub menu entry and seeing the first kernel message. Admittedly this is with the full linux-image-generic kernel (as I need scsi_debug).

MODULES=dep reduces it to some 50 MB, and removing /lib/firmware/amdgpu to 33 MB, which then unpacks in less than 3 seconds.

tags: added: rls-kk-incoming
Revision history for this message
Martin Pitt (pitti) wrote :

> Admittedly this is with the full linux-image-generic kernel

With just the -virtual image it's a lot better, so most of that slow boot time is indeed attributed to the extra modules. I still wonder, unpacking 100 MB even in a single-core QEMU VM shouldn't take 15 seconds surely? Grub seems to have some especially slow unpacker here.

Either way, this was mostly a side issue, sorry -- I didn't find another existing bug for "boot is slow due to initramfs unpack".

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.