Comment 9 for bug 635523

For the past year, distro bugtrackers have been receiving reports that Xephyr segfaults when it tries to map its client-facing root window:

  https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/635523
  https://bugzilla.redhat.com/show_bug.cgi?id=518960
  https://qa.mandriva.com/show_bug.cgi?id=47928

The cause of the problem is that, on some 24bpp hosts, this computation:

  http://cgit.freedesktop.org/xorg/xserver/tree/hw/kdrive/ephyr/ephyr.c#n255

yields a value for priv->bytes_per_line which is too small. priv->bytes_per_line is then used by Xephyr to create its host-side image data buffer (resulting in a buffer that is too small). Then, when Xephyr maps its root window, it segfaults by writing beyond the end of the too-small image data buffer while filling its root window in response to expose-event/damage processing.

As for fixes: ajax proposed one fix for this problem six months ago that seems to have gotten lost after an unanswered request for an amendment by keithp:

  http://patchwork.freedesktop.org/patch/1327/

I tested this patch against Ubuntu's xserver-xorg_2:1.9.0-0ubuntu7 package (from Maverick) and can confirm that it fixed the segfault for me in that environment.