update-grub ignores pvops kernels on Xen domU

Perhaps I will produce a patch...

This patch is intended to make update-grub:

(a) be better at detecting kernels which are suitable for user in a Xen domU, and

(b) be able to treat some kernels as suitable for both Xen-domU operation AND non-Xen-domU operation (previously it was one or the other but not both)

Update:

Previously I was running over the 2.6.32 kernel out of Debian Squeeze, and saw the behaviour mentioned previously.

I have now switched to the 2.6.32 kernel implied by the Ubuntu package "linux-image-2.6.32-22-virtual" (version 2.6.32-22.33), which is 2.6.32-22-server.

If I now run the original unpatched "update-grub", it finds all the relevant kernels I have installed, and does not ignore any of them, i.e. the behaviour as reported previously no longer happens.

However I *am* running inside a Xen domU. In theory, if the bug I reported were true, "update-grub" should be ignoring all of my kernels.

So what is going on?

The answer is that "update-grub" does NOT believe it is running inside a Xen domU, when in fact it is running inside a Xen domU, which appears to be another bug.

The two bugs together mean that update-grub does not ignore relevant kernels inside a domU. However it is not ignoring them for the wrong reasons. (I suppose you could call it a white box bug.)

There is a bit starting around line 1406:

   1406 if [ "$indomU" = "detect" ]; then
   1407 if [ -e /proc/xen/capabilities ] && ! grep -q "control_d" /proc/xen/capabilities; then
   1408 indomU="true"
   1409 else
   1410 indomU="false"
   1411 fi
   1412 fi

On my system (which is running inside a Xen domU), when running the Ubuntu kernel, the directory /proc/xen is empty. As such, the second "if" test finds nothing and sets "indomU" to "false".

(Under the Debian kernel, this must have been resolving to "true". Though switching back to the Debian kernel again, I am seeing false. Queer.)

So the "indomU" indicator is being computed incorrectly as "false" when actually running inside a domU.

I would speculate that the means of detecting whether inside a domU was based on earlier kernel behaviour and no longer works correctly.

So I would say we may need to use a second way of checking if we are inside a domU.

Sniffing around, we may be able do this by reading /sys/hypervisor/type. On my system that results in the string "xen".

So perhaps we could say something like:

if ( [ -e /proc/xen/capabilities ] && ! grep -q "control_d" /proc/xen/capabilities ) \
  || [ "`cat /sys/hypervisor/type 2>/dev/null`" = xen ] ; then
    indomU="true"
else
    indomU="false"
fi

This should then give us a better idea of whether we are inside a Xen domU.

I'm using the 'ec2' kernels that come with lucid, which show the same problem. They have CONFIG_XEN_UNPRIVILEGED_GUEST set, and the update-grub script only checks for CONFIG_XEN_PRIVILEGED_GUEST.

There is a second (small) bug on line 1416 by the way:

 if `echo "$xenKernels" | grep -q "$kern "` || `echo "$kern" | grep -q "xen"`; then

That space after $kern in the first grep -q command causes the last kernel to always be dropped off the list, which is not desirable. It even means that if there's only one Xen kernel found, it will never be added to menu.lst.

One user using grub and natty is bitten by this problem.
All kernels since the 2.6.37-9 have the XEN_PRIVILEGED_GUEST set.
update-grub uses this criteria to check if the kernel is a Xen kernel or a normal kernel.

That means that normal kernels don't show up in the grub menu since the 2.6.37-9 kernel.

This is especially nasty for the natty and for the maverick version.

Hi !

have excatly the same problem with the -server and generic-pae kernel. In my configs XEN_PRIVILEGED_GUEST is NOT set (acutal system, yesterday updated).

I have done a small fix for update-grub:

--- update-grub.old 2011-04-06 17:37:34.198906717 +0200
+++ update-grub 2011-04-06 17:37:20.437907250 +0200
@@ -1382,7 +1382,7 @@

 xenKernels=""
-for ver in `grep -l CONFIG_XEN_PRIVILEGED_GUEST=y /boot/config* | sed -e s%/boot/config-%%`; do
+for ver in `grep -l CONFIG_XEN=y /boot/config* | sed -e s%/boot/config-%%`; do
   # ver is a kernel version
   kern="/boot/vmlinuz-$ver"
   if [ -r $kern ] ; then
@@ -1413,7 +1413,7 @@

 sortedKernels=""
 for kern in $(/bin/ls -1vr /boot | grep -v "dpkg-*" | grep "^vmlinuz-") ; do
- if `echo "$xenKernels" | grep -q "$kern "` || `echo "$kern" | grep -q "xen"`; then
+ if `echo "$xenKernels" | grep -q "$kern"` || `echo "$kern" | grep -q "xen"`; then
                is_xen=1
        else
                is_xen=

Regards,
Sebastian

Without wishing to sound like a voice from the peanut gallery:

* Bug was reported 10 months ago

* Bug was reported clearly, and at length, using proper (English) language, properly spelled and punctuated

* Problem was analysed and a potential solution devised and provided

* Subsequently, further problems have been identified, and potential solutions devised and provided for those also

It seems to me that "the community" has very much "done its bit" on this bug, resulting in good quality information in the comments.

It seems to me that it should not be difficult for the mainter to process this bug, and improve the package.

However I cannot see any comments on here to say that any significant actions (such as assessing the suggested changes and applying them) have been taken.

I would be grateful if this bug could get some air time with the maintainer.

Errata for previous comment:

"... should not be difficult for the mainter to process ..."

should have been

"... should not be difficult for the maintainer to process ..."

Sorry for missing this earlier - I pay rather more attention to grub2 than grub these days. I think this is already fixed in Ubuntu 11.04, though; see bug 692691. Perhaps you could check that and confirm that it matches your bug? The patch I applied is here:

  http://bazaar.launchpad.net/~ubuntu-core-dev/grub/ubuntu/revision/904

I hadn't realised that this also affected older releases. It should be possible to backport it, if so.

Hi !

Patch looks good to me - on my system it works fine.

It would be good if the patch could be applied quickly - my system is lucid (LTS).

Thanks,
Sebastian

Yes, *please* backport this fix.

Hello,

i got an update for update-grub - the Xen Kernels are now recognized correct - except the first one.
The typo mistake has not been fixed:

--- update-grub.original 2011-06-07 14:55:56.849136053 +0200
+++ update-grub 2011-06-07 14:56:09.773353733 +0200
@@ -1413,7 +1413,7 @@

 sortedKernels=""
 for kern in $(/bin/ls -1vr /boot | grep -v "dpkg-*" | grep "^vmlinuz-") ; do
- if `echo "$xenKernels" | grep -q "$kern "` || `echo "$kern" | grep -q "xen"`; then
+ if `echo "$xenKernels" | grep -q "$kern"` || `echo "$kern" | grep -q "xen"`; then
                is_xen=1
        else
                is_xen=

Please fix this also,
Thanks,
Sebastian

Status changed to 'Confirmed' because the bug affects multiple users.

This is no more a supported version; and grub legacy upstream is also stopped, only receiving possible random fixes locally

update-grub.old 2011-04-06 17:37:34.198906717 +0200
+++ update-grub 2011-04-06 17:37:20.437907250 +0200
@@ -1382,7 +1382,7 @@

 xenKernels=""
-for ver in `grep -l CONFIG_XEN_PRIVILEGED_GUEST=y /boot/config* | sed -e s%/boot/config-%%`; do
+for ver in `grep -l CONFIG_XEN=y /boot/config* | sed -e s%/boot/config-%%`; do
   # ver is a kernel version
   kern="/boot/vmlinuz-$ver"
   if [ -r $kern ] ; then
@@ -1413,7 +1413,7 @@

 sortedKernels=""
 for kern in $(/bin/ls -1vr /boot | grep -v "dpkg-*" | grep "^vmlinuz-") ; do
- if `echo "$xenKernels" | grep -q "$kern "` || `echo "$kern" | grep -q "xen"`; then
+ if `echo "$xenKernels" | grep -q "$kern"` || `echo "$kern" | grep -q "xen"`; then
                is_xen=1
        else
                is_xen=

update-grub.old

 xenKernels=""
-for ver in `grep -l CONFIG_XEN_PRIVILEGED_GUEST=y /boot/config* | sed -e s%/boot/config-%%`; do
+for ver in `grep -l CONFIG_XEN=y /boot/config* | sed -e s%/boot/config-%%`; do
   # ver is a kernel version
   kern="/boot/vmlinuz-$ver"
   if [ -r $kern ] ; then
@@ -1413,7 +1413,7 @@

 sortedKernels=""
 for kern in $(/bin/ls -1vr /boot | grep -v "dpkg-*" | grep "^vmlinuz-") ; do
- if `echo "$xenKernels" | grep -q "$kern "` || `echo "$kern" | grep -q "xen"`; then
+ if `echo "$xenKernels" | grep -q "$kern"` || `echo "$kern" | grep -q "xen"`; then
                is_xen=1
        else
                is_xen=