gssd does not start reliably at boot if /usr is on a separate partition

A possible solution is to move rpc.gssd from /usr/sbin to /sbin. As it is even possible that /usr is imported by NFS4, also rpc.idmapd should be moved from /usr/sbin to /sbin.

On Fri, Jul 30, 2010 at 09:31:46AM -0000, Christoph wrote:
> A possible solution is to move rpc.gssd from /usr/sbin to /sbin. As it
> is even possible that /usr is imported by NFS4, also rpc.idmapd should
> be moved from /usr/sbin to /sbin.

That's only a solution if we move all the dependency libraries from /usr/lib
to /lib first, FWIW.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

This bug probably needs a similar solution as bug #525154.

But a solution such as 525154 does not solve the problem pointed out in comment #2 and that's that /usr should be mountable as an NFS4 filesystem. Yes, that means moving all binaries and libraries necessary to /sbin and /lib. Is that really a problem?

I personally don't see an issue with putting all of those libs in /lib and binaries in /sbin, since they ultimately support mounting filesystems other than /.

I have to wonder if those with more experience can shed light on any hidden gotchyas there. I do see that rpc.gssd has a lot of libs in /usr:

$ ldd /usr/sbin/rpc.gssd
 linux-vdso.so.1 => (0x00007fff54eb9000)
 librpcsecgss.so.3 => /usr/lib/librpcsecgss.so.3 (0x00007f70f7992000)
 libgssglue.so.1 => /usr/lib/libgssglue.so.1 (0x00007f70f7788000)
 libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x00007f70f74c3000)
 libcom_err.so.2 => /lib/libcom_err.so.2 (0x00007f70f72bf000)
 libc.so.6 => /lib/libc.so.6 (0x00007f70f6f3c000)
 libdl.so.2 => /lib/libdl.so.2 (0x00007f70f6d37000)
 libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x00007f70f6b10000)
 libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 (0x00007f70f6908000)
 libkeyutils.so.1 => /lib/libkeyutils.so.1 (0x00007f70f6704000)
 libresolv.so.2 => /lib/libresolv.so.2 (0x00007f70f64e9000)
 libpthread.so.0 => /lib/libpthread.so.0 (0x00007f70f62cb000)
 /lib64/ld-linux-x86-64.so.2 (0x00007f70f7bc5000)

There are no particular major gotchas, although there's no clean way today to handle -dev packages for libraries in /lib. It's more that moving all of these files to /lib requires touching lots of packages, will take some time to finish, *and* is not a complete solution because gssd still needs to have access to /var/lib/nfs/rpc_pipefs, which requires ensuring that /var is mounted before startup.

Fixing the upstart job is also not a complete solution if you really want to have /usr on NFSv4; but we've never supported /usr on NFSv4 before now, so that's not really a regression. Whereas gssd failing to start up when /usr is *not* on NFSv4 is a regression.

You may also refer to LP #643289 posts #2, #3, #4, which include (among others) a fix for gssd start-up problems. Please note that all of #2, #3, #4 should be applied to make it work.

While I have not tested my gssd fix as intensely as similar ones for idmapd/statd, all share major functionality, so I guess/hope it will be of use to work around the initial problem of gssd start-up reported here.

This bug was fixed in the package nfs-utils - 1:1.2.6-3ubuntu2

---------------
nfs-utils (1:1.2.6-3ubuntu2) quantal; urgency=low

  [ Steve Langasek ]
  * Adjust upstart jobs to treat TYPE=nfs and TYPE=nfs4 mounts identically,
    since TYPE=nfs4 is considered deprecated.
  * Fix various boot-time race conditions between mountall and nfs-utils by
    moving handling of the 'mounting' events to separate gssd-mounting and
    idmapd-mounting jobs. Requires mountall 2.41 or better to avoid deadlock
    on boot. LP: #643289, LP: #611397.
  * Fix the stop conditions: never stop on 'runlevel [06]' since that gives
    the system no time to cleanly unmount nfs mounts; instead, stop only on
    the unmounted-remote-filesystems event.
  * Newer versions of gssd don't talk to portmap, so don't make the upstart
    job depend on it.
  * Add an instance to statd-mounting, and change it to just wait for statd
    instead of trying to trigger it potentially out of order. This also means
    we don't need to try to force portmap to start from statd.

  [ Matthew L. Dailey ]
  * Add "-e" (ticket expiry is error) option to rpc.gssd to prevent hangs due
    to EKEYEXPIRED error from kernel on ticket expiry. LP: #794112
 -- Stephane Graber <email address hidden> Fri, 28 Sep 2012 13:58:43 -0400