Xvfb fails with empty /var/lib/xkb, causing build failures

yes, xfvb, but still ftbfs

Looks like the test is failing while trying to do RANDR operations within xvfb:

make[3]: Entering directory `/home/bryce/pygtk/pygtk-2.22.0/dbg-build-2.6/tests'
Xlib: extension "RANDR" missing on display ":99".
Traceback (most recent call last):
  File "/home/bryce/pygtk/pygtk-2.22.0/tests/runtests.py", line 41, in <module>
    suite.addTest(loader.loadTestsFromName(name))
  File "/usr/lib/python2.6/unittest.py", line 576, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
  File "/home/bryce/pygtk/pygtk-2.22.0/tests/test_enum.py", line 9, in <module>
    class PObject(gobject.GObject):
  File "/home/bryce/pygtk/pygtk-2.22.0/tests/test_enum.py", line 10, in PObject
    enum = gobject.property(type=gtk.WindowType, default=gtk.WINDOW_TOPLEVEL)
  File "/usr/lib/python2.6/dist-packages/gobject/propertyhelper.py", line 112, in __init__
    self.type = self._type_from_python(type)
  File "/usr/lib/python2.6/dist-packages/gobject/propertyhelper.py", line 201, in _type_from_python
    raise TypeError("Unsupported type: %r" % (type_,))
TypeError: Unsupported type: <class 'gtk._gtk.WindowType'>
[83021 refs]
make[3]: *** [check-local] Error 1

Actually, some light googling shows the 'Xlib: extension "RANDR" missing on display ":99".' is innocuous and can be ignored.

I'm able to reproduce the same test failure from the pygtk build directory without xvfb (i.e. DISPLAY=:0), so this seems not to be a bug in xvfb but just a failing test that isn't reported well.

@Doko, before I close out the xorg-server task, can you review and see if you concur, or if I missed something please elaborate. I spot checked a few of your other recently reported bugs to see if there are other instances of xvfb-related failures but didn't see any so am guessing this is just an outlier.

I didn't see other xvfb related failures for the test rebuild, I just added xvfb on a notice from the desktop team. please sort this out internally.

Alright, well both RAOF and I looked at this and agree it seems not to be an xvfb issue, but rather breakage in the test itself.

The RANDR error is just because xvfb by default does not load that module; there's options to make it load it if the test does need it, but I haven't played around with that myself, so YMMV.

The RANDR errors here are not related to the failure in the build log Matthias linked to. The failure there is:

xvfb-run: error: Xvfb failed to start

I can reproduce this when running xvfb-run under fakeroot (as will be the case when xvfb-run is called from an 'install' or 'binary' target on the autobuilder). Something has regressed to make xvfb-run incompatible with fakeroot.

Running by hand, I see:

$ fakeroot Xvfb :99 -noreset -nolisten tcp(EE) GLX: could not load software renderer
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
[dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list!
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Error: Cannot open "/var/lib/xkb/file8s3uEn" to write keyboard description
> Exiting
(EE) Error compiling keymap (-)
(EE) XKB: Couldn't compile keymap
(EE) XKB: Failed to load keymap. Loading default keymap instead.
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Error: Cannot open "/var/lib/xkb/file64o1de" to write keyboard description
> Exiting
(EE) Error compiling keymap (-)
(EE) XKB: Couldn't compile keymap
XKB: Failed to compile keymap
Keyboard initialization failed. This could be a missing or incorrect setup of xkeyboard-config.

Fatal server error:
Failed to activate core devices.
$

So if xvfb thinks it's root (which is the purpose of fakeroot), it fails to start if it can't write to /var/lib/xkb.

On Tue, Aug 23, 2011 at 05:42:50PM -0000, Steve Langasek wrote:
> Running by hand, I see:
>
> $ fakeroot Xvfb :99 -noreset -nolisten tcp(EE) GLX: could not load software renderer
>
> So if xvfb thinks it's root (which is the purpose of fakeroot), it fails
> to start if it can't write to /var/lib/xkb.

Interesting; I don't reproduce that error because I have the directory:

bryce@clanfield:~$ fakeroot Xvfb :99 -noreset -nolisten tcp
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
^C/usr/bin/fakeroot: line 1: kill: (3456) - No such process
bryce@clanfield:~$ ls /var/lib/xkb/
server-02D8252E59564A234380F1E5417646A9DB3B7452.xkm server-B20D7FC79C7F597315E3E501AEF10E0D866E8E92.xkm
bryce@clanfield:~$

bryce@clanfield:~$ dpkg -S /var/lib/xkb
x11-xkb-utils: /var/lib/xkb

Package: xvfb
Architecture: any
Depends:
 xserver-common (>= ${source:Version}),
 ${shlibs:Depends},
 ${misc:Depends},
 xauth,
 x11-xkb-utils

The directory itself will exist in a chroot, but it will be empty, which seems to trigger the failure.

Confirming that an empty /var/lib/xkb/ breaks Xvfb.

No, there seems to be more to it than that.

root@clanfield:~# ls /var/lib/xkb/
server-02D8252E59564A234380F1E5417646A9DB3B7452.xkm server-B20D7FC79C7F597315E3E501AEF10E0D866E8E92.xkm
root@clanfield:~# rm /var/lib/xkb/*
root@clanfield:~# fakeroot Xvfb :99 -noreset -nolisten tcp
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
^C/usr/bin/fakeroot: line 1: kill: (4379) - No such process
root@clanfield:~# ls /var/lib/xkb/
server-02D8252E59564A234380F1E5417646A9DB3B7452.xkm

And in a freshly upgraded chroot:

(chroot)root@humber:~# rm /var/lib/xkb/server-*.xkm
(chroot)root@humber:~# fakeroot Xvfb :99 -noreset -nolisten tcp
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
[dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list!
^C(chroot)root@humber:~# ls /var/lib/xkb/
server-02D8252E59564A234380F1E5417646A9DB3B7452.xkm

I think the empty /var/lib/xkb/ may be a symptom rather than a root cause. Guessing that in the buildd's there's no keyboard so no compiled xkb keymap is going to get generated.

Martin, can you elaborate on how you're reproducing it? Are you doing it in a local chroot or on a buildd?

Another detail that would help me figure this out is whether there have been /var/lib/xkb/*.xkm files in the past (i.e. the regression is that they're no longer being generated), or if they've always been missing (in which case the regression is xvfb is now expecting them when it didn't).

Also, can someone able to reproduce this test if merely doing
  touch /var/lib/xkb/server-1234123412341234123412341234123412341234.xkm
makes it proceed.

Nevermind; I found a way to reproduce it locally now.

Test case I'm using is adding the following line to the hello package's binary-arch target:

  Xvfb :99

In particular, the fault is occurring within the X server code for the xkbcomp call. The call it's trying to make is something like:

  xkbcomp -w 1 -R/usr/share/X11/xkb/keymap -xkm xfree86 /var/lib/xkb/server-foo.xkm

Error: Cannot open "/var/lib/xkb/server-foo.xkm" to write keyboard description
                  Exiting

In natty, Xvfb runs properly because it uses /tmp/server-foo.xkm, which it can write to. Indeed, the above xkbcomp call fails in called in binary-arch rules, but simply changing to /tmp succeeds:

  xkbcomp -w 1 -R/usr/share/X11/xkb/keymap -xkm xfree86 /var/lib/xkb/server-foo.xkm

The code in question that decides what directory to use is xserver/xkb/ddxLoad.c:

tatic void
OutputDirectory(
    char* outdir,
    size_t size)
{
#ifndef WIN32
    /* Can we write an xkm and then open it too? */
    if (access(XKM_OUTPUT_DIR, W_OK | X_OK) == 0 && (strlen(XKM_OUTPUT_DIR) < size))
    {
        (void) strcpy (outdir, XKM_OUTPUT_DIR);
    } else

In fakeroot 1.16-1, Debian added a patch to add access() support, to fix debian bug #629956.

However, this broke apt (debian bug #630591) and other stuff (debian bug #630129), and so the patch was reverted in fakeroot 1.17-1:

  http://packages.qa.debian.org/f/fakeroot/news/20110818T113211Z.html

As a workaround, you can make pygtk succeed via the LD_PRELOAD trick:

  G_HOME=/ PYTHON=/usr/bin/python$* LD_PRELOAD="" xvfb-run -s "-screen 0 1280x1024x24 -noreset" $(MAKE) -C build-$* check

The real solution is to pull fakeroot 1.17-1 and get rid of fakeroot's bugged access().

Not an X bug after all.

I believe the solution to this is to sync fakeroot 1.17-1 from debian.

Getting binaries for oneiric...
[Updating] fakeroot (1.16-1 [Ubuntu] < 1.17-1 [Debian])
 * Trying to add fakeroot...
2011-08-25 01:38:48 INFO - <fakeroot_1.17-1.dsc: downloading from http://ftp.debian.org/debian/>
2011-08-25 01:38:48 INFO - <fakeroot_1.17-1.debian.tar.gz: downloading from http://ftp.debian.org/debian/>
2011-08-25 01:38:48 INFO - <fakeroot_1.17.orig.tar.bz2: downloading from http://ftp.debian.org/debian/>
I: fakeroot [main] -> fakeroot_1.16-1 [main].