VMBuilder

VMBuilder doesn't work with grub2

Bug #410886 reported by Andreas Heck on 2009-08-08

This bug affects 10 people

	Status	Importance	Assigned to	Milestone
VMBuilder	Fix Released	Undecided	Unassigned
vm-builder (Ubuntu)	Fix Released	High	Soren Hansen	Ubuntu ubuntu-9.10
Karmic	Fix Released	High	Soren Hansen	Ubuntu ubuntu-9.10

Bug Description

If you run vmbuilder on a fresh karmic installation you get the following error message at the end of the vmbuilder run:

2009-08-09 01:06:56,792 INFO : Cleaning up
Traceback (most recent call last):
  File "/home/ahe/Desktop/Projekte/VMbuilder/vmbuilder-gui/vmbuilder", line 29, in <module>
    VMBuilder.run()
  File "/home/ahe/Desktop/Projekte/VMbuilder/vmbuilder-gui/VMBuilder/__init__.py", line 65, in run
    frontend.run()
  File "/home/ahe/Desktop/Projekte/VMbuilder/vmbuilder-gui/VMBuilder/plugins/cli/__init__.py", line 70, in run
    vm.create()
  File "/home/ahe/Desktop/Projekte/VMbuilder/vmbuilder-gui/VMBuilder/vm.py", line 483, in create
    self.install()
  File "/home/ahe/Desktop/Projekte/VMbuilder/vmbuilder-gui/VMBuilder/vm.py", line 416, in install
    self.distro.install_bootloader()
  File "/home/ahe/Desktop/Projekte/VMbuilder/vmbuilder-gui/VMBuilder/plugins/ubuntu/distro.py", line 173, in install_bootloader
    EOT''')
  File "/home/ahe/Desktop/Projekte/VMbuilder/vmbuilder-gui/VMBuilder/util.py", line 105, in run_cmd
    proc = subprocess.Popen(args, stdin=stdin_arg, stderr=subprocess.PIPE, stdout=subprocess.PIPE, env=proc_env)
  File "/usr/lib/python2.6/subprocess.py", line 595, in __init__
    errread, errwrite)
  File "/usr/lib/python2.6/subprocess.py", line 1092, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

A quick look at VMBuilder/vm.py shows that it tries to call the program "grub" but this program doesn't exist with grub2.

The main problem here is that you can't have both of the packages grub and grub-pc installed at the same time since they replace each other. It looks like the maintainers of the grub packages didn't think about the not so common use case where you install a bootmanager package but don't want it to boot your machine. But if you have a freshly installed karmic machine you need both bootloader packages to be present or otherwise you couldn't create any VMs with an Ubuntu version older than karmic, even if vmbuilder supported grub2. A similar problem would occur, if you used an PC with EFI and the EFI version of grub2 so that you needed the PC version of grub2 to install the bootloader of the virtual machine.

See original description

Related branches

lp:~kirkland/vmbuilder/410886

lp:~cjwatson/vmbuilder/chroot-grub

Merged into lp:vmbuilder/trunk

Soren Hansen: Approve on 2009-10-08

lp:ubuntu/lucid/vm-builder

Andreas Heck (aheck) on 2009-08-09

description:

updated

Dustin Kirkland  (kirkland) on 2009-09-11

Changed in vmbuilder:
status:	New → Confirmed

Dustin Kirkland  (kirkland) on 2009-09-12

Changed in vm-builder (Ubuntu):
status:	New → Confirmed
importance:	Undecided → High
milestone:	none → karmic-alpha-6
milestone:	karmic-alpha-6 → ubuntu-9.10-beta

Thierry Carrez (ttx) on 2009-09-18

Changed in vm-builder (Ubuntu Karmic):
assignee:	nobody → Soren Hansen (soren)
milestone:	ubuntu-9.10-beta → none

Revision history for this message

Soren Hansen (soren) wrote on 2009-09-21:

Adding grub2 support to VMBuilder seems very non-trivial at the moment. However, it seems that grub and grub2 can peacefully coexist now? I'm certainly using grub2 on my laptop, but have just installed grub and VMBuilder builds virtual machines just fine now.

Revision history for this message

Colin Watson (cjwatson) wrote on 2009-10-02:

I think that's actually a bug. grub and grub2 are meant to conflict.

Revision history for this message

Colin Watson (cjwatson) wrote on 2009-10-05:

You already have grub installed in the chroot at this point, as far as I can see. Why not just call grub in the chroot, rather than relying on whatever's installed in the host system?

Revision history for this message

Colin Watson (cjwatson) wrote on 2009-10-05:

lp:~cjwatson/vmbuilder/chroot-grub changes vmbuilder to call grub in the chroot. There's a certain amount of faff involved to let the chroot see the disk images so that it can install grub to them, but it's not all that bad.

Soren Hansen (soren) on 2009-10-08

Changed in vm-builder (Ubuntu Karmic):
assignee:	Soren Hansen (soren) → Colin Watson (cjwatson)

Revision history for this message

Maximilian Mill (maximilian-mill) wrote on 2009-10-08:

vm-builder-debug-grub.log Edit (3.8 KiB, text/plain)

Hello Collin,

with your change the first ubuntu-vm-builder run worked for me. But the second try it didnt work.

i did this :
bzr diff -r 350 > patch (in your branch)
and applied this to /usr/share/pyshared/VMBUILDER/plugins/ubuntu/distro.py

This is cut from the --debug logfile(fulllog attached):
2009-10-08 16:17:17,530 DEBUG : Checking if "/boot/grub/stage1" exists... no
2009-10-08 16:17:17,531 DEBUG : Checking if "/grub/stage1" exists... no
2009-10-08 16:17:17,531 DEBUG :
2009-10-08 16:17:17,531 DEBUG : Error 15: File not found

This is the command with i used (im using KVM with libvirt):
sudo ubuntu-vm-builder --debug kvm karmic --dest=ubuntu-kvm/glusterfs2/ --hostname=glusterfs2 --domain=glusterfs2 --addpkg vim --mem 256 --libvirt qemu:///system

Revision history for this message

Maximilian Mill (maximilian-mill) wrote on 2009-10-08:

Eh, sorry for the typos.

Revision history for this message

Steve Beattie (sbeattie) wrote on 2009-10-08:

Colin, people who upgraded from jaunty to karmic and did the test out grub2 will probably have both grub and grub2 installed (that is at least the situation with one of my hosts). Unfortunately, update-manager andapt-getdist-upgrade is resolving the added conflict in https://launchpad.net/ubuntu/+source/grub2/1.97~beta3-1ubuntu8 such that it's uninstalling grub-pc and not grub, leaving a probable unlikely surprise for those of us in this situation:

  $ sudo apt-get dist-upgrade
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  Calculating upgrade... Done
  The following packages will be REMOVED:
    eclipse-source grub-pc grub2 libuniconf4.4
  [SNIP]

Also, is the intended conflicting version really meant to be << 0.97-54, when the latest versions of grub (v1) that we ship is 0.97-29ubuntu58? Is it possible you meant <<0.97-29ubuntu54 ? Otherwise, why was 0.97-54 chosen?

Revision history for this message

Maximilian Mill (maximilian-mill) wrote on 2009-10-09:

In post #5 i did say that it worked the first try. That wasn't true for karmic. I forgot that i used 'ubuntu-vm-builder kvm jaunty' not
'ubuntu-vm-builder kvm karmic'.

So this just needs to be fixed for karmic guests.

Revision history for this message

Iain Allan (iain-allan) wrote on 2009-10-14:

I am seeing the same problem on Karmic with ubuntu-vm-builder and grub not finding the stage1 grub file.

I converted the qcow2 image created by vmbuilder to raw and then mounted the image to inspect the grub directory. It seems that the grub stage files are actually in /boot/grub/i386-pc.

If you copy the stage files from boot/grub you can then manually install grub and get the KVM image to boot.

# cd ubuntu-kvm ; mkdir mnt
# qemu-img convert -O raw disk0.qcow2 disk0.img
# mount -oloop,offset=16384 -t ext3 disk0.img mnt
# cp mnt/boot/grub/i386-pc/stage* mnt/boot/grub
# grub --device-map=/dev/null
grub> device (hd0) ../disk0.img
device (hd0) ../disk0.img
grub> root (hd0,0)
root (hd0,0)
grub> setup (hd0)
setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... no
Running "install /boot/grub/stage1 (hd0) /boot/grub/stage2 p /boot/grub/menu.lst "... succeeded
Done.
# qemu-img convert -O qcow2 disk0.img disk0.qcow2

Revision history for this message

Buddha (lord-buddha) wrote on 2009-10-15:

#10

I also ran into a bug in this area. I don't see the stack trace the reporter of this bug go.

This is what I see. Is this the same issue ?

2009-10-15 11:04:44,627 INFO : Copying to disk images
2009-10-15 11:04:44,628 DEBUG : ['rsync', '-aHA', '/tmp/vmbuildernkGANu/root/', '/tmp/vmbuildernkGANu/target']
2009-10-15 11:04:48,215 INFO : Installing bootloader
2009-10-15 11:04:48,216 DEBUG : ['grub', '--device-map=/tmp/vmbuildernkGANu/device.map', '--batch']
2009-10-15 11:04:48,216 DEBUG : stdin was set and it was a string: root (hd0,0)
setup (hd0)
EOT
2009-10-15 11:04:48,221 DEBUG : Oh, dear, an exception occurred
2009-10-15 11:04:48,222 INFO : Cleaning up
2009-10-15 11:04:48,222 DEBUG : ['umount', '/tmp/vmbuildernkGANu/root/proc']
2009-10-15 11:04:48,227 DEBUG : umount: /tmp/vmbuildernkGANu/root/proc: not mounted
2009-10-15 11:04:48,227 DEBUG : ['umount', '/tmp/vmbuildernkGANu/root/dev/pts']
2009-10-15 11:04:48,241 DEBUG : umount: /tmp/vmbuildernkGANu/root/dev/pts: not mounted
2009-10-15 11:04:48,241 DEBUG : ['umount', '/tmp/vmbuildernkGANu/root/dev']
2009-10-15 11:04:48,249 DEBUG : umount: /tmp/vmbuildernkGANu/root/dev: not mounted
2009-10-15 11:04:48,249 DEBUG : Unmounting /tmp/vmbuildernkGANu/target/
2009-10-15 11:04:48,250 DEBUG : ['umount', '/tmp/vmbuildernkGANu/target/']
2009-10-15 11:04:51,977 DEBUG : ['kpartx', '-d', '/tmp/vmbuildernkGANu/disk0.img']
2009-10-15 11:04:51,983 DEBUG : loop deleted : /dev/loop0
2009-10-15 11:04:51,983 DEBUG : ['kpartx', '-d', '/tmp/vmbuildernkGANu/disk0.img']
2009-10-15 11:04:51,989 DEBUG : ['rmdir', 'ubuntu-kvm']
2009-10-15 11:04:51,992 DEBUG : ['rm', '-rf', '/tmp/vmbuildernkGANu']
2009-10-15 11:04:52,337 DEBUG : ['rm', '/tmp/tmpZbZv9q']
Couldn't find the program 'grub' on your system

Revision history for this message

Iain Allan (iain-allan) wrote on 2009-10-15:

#11

Buddha,

You need to install grub version 1 if your Karmic install is using grub2. VMbuilder will then be able to create an image but the next problem is the image will not boot unless you manually install grub. Grub seems to be unable to find the grub stage files.

Revision history for this message

Arnaud (arnaud-oss) wrote on 2009-10-17:

#12

Using a fresh install of karmic (grub2 only on host)

I had to manually apply the patch made by Colin Watson (comment #4) and it almost worked...

First question, why is this patch not yet included in an updated package of vmbuilder ?

Second, I encountered the same problem as Iain Allan (comment #9), I think I fixed it, patch below...

I can finally build karmic vm on karmic host, ... and start them!

patch against python-vm-builder 0.11.1-0ubuntu1

--- /usr/share/pyshared/VMBuilder/plugins/ubuntu/dapper.py
+++ /usr/share/pyshared/VMBuilder/plugins/ubuntu/dapper.py
@@ -306,7 +306,7 @@

def install_grub(self):
self.run_in_target('apt-get', '--force-yes', '-y', 'install', 'grub')
- run_cmd('cp', '-a', '%s%s/%s/' % (self.destdir, self.grubroot, self.vm.arch == 'amd64' and 'x86_64-pc' or 'i386-pc'), '%s/boot/grub' % self.destdir)
+ run_cmd('cp', '-a', '%s%s/%s/.' % (self.destdir, self.grubroot, self.vm.arch == 'amd64' and 'x86_64-pc' or 'i386-pc'), '%s/boot/grub' % self.destdir)

def create_devices(self):
import VMBuilder.plugins.xen

Revision history for this message

Colin Watson (cjwatson) wrote on 2009-10-20:

#13

Steve, I chose -54 because that's the version in Debian which transformed grub into a dummy transitional package. Theoretically at some point we might get round to merging that.

I'm afraid there's not a whole lot we can do about the fact that the added Conflicts is confusing for people with both grub and grub-pc installed. That was an error state, plain and simple - the packages shipped the same files! This didn't cause an error due to Replaces, but it's definitely wrong.

Matt Zimmerman (mdz) on 2009-10-20

Changed in vm-builder (Ubuntu Karmic):
status:	Confirmed → Triaged
assignee:	Colin Watson (cjwatson) → Soren Hansen (soren)
milestone:	none → ubuntu-9.10