KVM crashes when -vga is set to vmware.

I'm confirming this on current Karmic qemu-kvm-0.11. It's not segfaulting, however, so I'm changing the title accordingly, and lowering the priority to 'low', since it's not crashing. Still, I'm not seeing any output in the window.

Also, I'll copy this to upstream QEMU, as they're aware of this, and working the issue (perhaps).

:-Dustin

Doh!

Eating my words now... Just needed to give the VM a little more time. Segfault. Boom.

kirkland@x200:/local/virt/img$ kvm -hda karmic-desktop.img -vga vmware
vmsvga_value_write: guest runs Linux.
Segmentation fault (core dumped)

Upping to 'medium' again.

:-Dustin

Same error
vmsvga_value_write: guest runs Linux.
/home/ebb/scripts/ubuntukvm: line 1: 8273 Segmentation fault kvm -vga vmware -m 1024 -soundhw es1370 -hda ubuntudisk.img -net nic,macaddr=00:1d:92:ab:3f:80 -net tap,ifname=tap2,script=no,downscript=no

Guest is ubuntu-9.10-desktop-i386

I see this too (Lucid guest). As a workaround, using "-vnc :1" instead of the default sdl output lets me use -vga vmware.

I figured out the cause of the crash, at least in my system. With SDL, qemu-kvm advertises the cursor setting capability to the guest, and therefore gets a DEFINE_CURSOR command. In the crashing case, it gets a request for a 64x64 cursor at 32bpp, which requires 64*64 = 4096 32-bit words to store. However struct vmsvga_cursor_definition_s.image[] is declared to have size only 1024, so handling the cursor request overruns the array.

I'm attaching a patch that enlarges the image array, which fixes the issue (for me at least -- I am able to run a Lucid guest with the vmware X.org driver on a Karmic host with "-vga vmware" with this applied, which used to crash). I've not checked if upstream has fixed this yet.

Excellent, thanks, Roland.

We can carry that patch in Ubuntu.

To get it upstream, you're going to need to submit it to the qemu-devel mailing list. Anthony isn't able to take patches submitted through Launchpad, unfortunately...

OK, I've sent this on upstream and in fact there is a whole series of vmware_vga stuff at <http://<email address hidden>/> that might be worth picking up.

I think it would be nice to get a fixed qemu-kvm into Karmic during the Lucid cycle, since vmware is the fastest video driver for kvm, and being able to test Lucid images in Karmic would be nice.

Are you planning on picking up qemu 0.12 for Lucid? Seems like it would be a good idea, since the new stable series would probably be better supportable long-term.

Er, <http://<email address hidden>/msg21420.html> is the series of vmware patches.

Off-topic, but yes, I'm trying to package qemu-kvm-0.12 right now.
There are a number of blocking issues with the release, though.

Hmm, still broken on qemu-0.12.1.2. Screenshot attached.

Anthony, can you reproduce on HEAD?

Dustin: not sure what that screenshot you uploaded is. Is that the qemu-kvm window itself?

If so I don't think you're seeing the same bug -- the issue being tracked here (and which I'm pretty sure is fixed in qemu 0.12) is a problem where the vmware-vga has a buffer overrun and crashes the host kvm process. Your screenshot looks more like the guest crashing to me -- what guest is it?

I just tested the latest lucid-desktop live CD with -vga vmware on a lucid host, and it works fine. Host has:

$ apt-cache policy qemu-kvm
qemu-kvm:
  Installed: 0.12.2-0ubuntu1
  Candidate: 0.12.2-0ubuntu1
  Version table:
 *** 0.12.2-0ubuntu1 0
        500 http://us.archive.ubuntu.com lucid/main Packages
        100 /var/lib/dpkg/status

Cool! Thanks.