update-binfmts is slow on boot

'update-binfmts --enable' is used to tell the kernel about registered binary formats. Without this, the registered binary formats won't actually work.

The right fix is probably a rewrite in C ...

Thanks for the explanation! I just noticed that update-binfmts and sysklogd (see bug #290127 or bug #285530) are the only users of perl during boot (confirmed by reprofiling and checking readahead list, and by using a perl wrapper), so there is potential to shave of another second or two there.

Just a sketch of how it can be done in sh, supporting the "magic" format of the DB. Maybe it can fall back to the full perl version if it encounters an unknown format.

I made another alternative: Let update-binfmts maintain a cache of the strings it would write to $PROCDIR/register. At boot time just write those strings into $PROCDIR/register, and only run the full update-binfmts --enable if the cache is incomplete.

This way the whole format and DB parsing is kept in one place as before.

https://code.launchpad.net/~tormodvolden/binfmt-support/bootcache

The gain in boot performance can be seen on the bootchart diagram. One seconds-long red chunk less! (If you test this with the perl machinery already loaded, you won't see much difference.)

Thanks, this looks great. I made some adjustments:

  http://bzr.debian.org/loggerhead/binfmt-support/trunk/revision/65.1.10?compare_revid=65.1.2

... but have largely merged this as-is. I'll push it into Karmic after Alpha 6.

Are you sure about "useless use of echo"? I tried it, and I think there is a newline issue.

Really? The kernel should ignore a trailing newline - it won't be
recognised as a valid flag character, which will cause
fs/binfmt_misc.c:check_special_flags to break out, and then
fs/binfmt_misc.c:create_entry will silently skip the trailing newline.
The instructions in Documentation/binfmt_misc.txt will include a
trailing newline in the string written to register.

For that matter, update-binfmts itself has always written a trailing
newline to register. cat won't add a second one or anything.

What did you observe to go wrong?

Let me check it again later tonight. I remember I got an error which went away when I added the echo kludge. Now that I see that the register string already has a newline, I can't say why it would help...

For some reason there is a difference if the format has already been registered. But this will not happen in our init script anyway.

$ sudo /usr/sbin/update-binfmts --disable jar
$ sudo sh -c "echo $(cat /var/cache/binfmts/jar) > /proc/sys/fs/binfmt_misc/register"
$ sudo sh -c "echo $(cat /var/cache/binfmts/jar) > /proc/sys/fs/binfmt_misc/register"
$
$ sudo /usr/sbin/update-binfmts --disable jar
$ sudo sh -c "cat /var/cache/binfmts/jar > /proc/sys/fs/binfmt_misc/register"
$ sudo sh -c "cat /var/cache/binfmts/jar > /proc/sys/fs/binfmt_misc/register"
cat: write error: Invalid argument

This case should be covered by the check in the init script for whether "$PROCDIR/$fmt" exists, so I think it's fine to disregard that.

This bug was fixed in the package binfmt-support - 1.2.14

---------------
binfmt-support (1.2.14) unstable; urgency=low

  [ Tormod Volden ]
  * Maintain a cache of binary format registration instructions, and use
    them from shell at boot time if possible to avoid the Perl slow path
    (LP: #320822).

binfmt-support (1.2.13) unstable; urgency=low

  * Upgrade to debhelper 7. Uses override targets, so requires 7.0.50.
  * Check that /etc/default/rcS exists before sourcing it.
  * Stop hardcoding path to update-binfmts.
  * Remove Default-Stop from init script; cf. changelog entry for 1.2.8.

 -- Colin Watson <email address hidden> Sat, 19 Sep 2009 21:48:13 +0100