Ubuntu
qemu-kvm package

CPU hot plug kills the guest

Bug #878422 reported by Simon Déziel on 2011-10-19

This bug affects 2 people

	Status	Importance	Assigned to
libvirt (Ubuntu)	Invalid	Low	Unassigned
Oneiric	Won't Fix	Undecided	Unassigned
qemu-kvm (Ubuntu)	Fix Released	High	Unassigned
Oneiric	Fix Released	Undecided	Unassigned

Bug Description

===============================================
SRU Justification:
1. Impact: libvirt-qemu guest crashes
2. Development fix: refuse attempt to hotplug a new cpu
3. Stable fix: same as development fix
4. Test case:
a. create a vdisk: qemu-img create disk.img 1G
b. start a qemu instance: kvm -vnc :1 -monitor stdio -hda disk.img
c. in the monitor on stdio, add a cpu:
     cpu_set 2 online
   Without this patch, kvm will print an error and die. With the patch, it
   will continue to run.
5. Regression potential: none
===============================================

I'm using a Natty host and an Oneiric guest (both fully up to date) to do some KVM testing. Hot plugging a CPU like this :

virsh setvcpus oneiric --count 2 --live

generates the following messages in /var/log/libvirt/qemu/oneiric.log :

kvm: /build/buildd/qemu-kvm-0.14.0+noroms/hw/qdev.c:95: qdev_create_from_info: Assertion `bus->allow_hotplug' failed.
2011-10-19 14:32:53.124: shutting down

From there, obviously the guest dies.

Host details :
$ apt-cache policy qemu-kvm
qemu-kvm:
  Installed: 0.14.0+noroms-0ubuntu4.4
  Candidate: 0.14.0+noroms-0ubuntu4.4
  Version table:
*** 0.14.0+noroms-0ubuntu4.4 0
        500 http://ca.archive.ubuntu.com/ubuntu/ natty-updates/main amd64 Packages
        500 http://ca.archive.ubuntu.com/ubuntu/ natty-security/main amd64 Packages
        100 /var/lib/dpkg/status
     0.14.0+noroms-0ubuntu4 0
        500 http://ca.archive.ubuntu.com/ubuntu/ natty/main amd64 Packages
$ apt-cache policy libvirt-bin
libvirt-bin:
  Installed: 0.8.8-1ubuntu6.6
  Candidate: 0.8.8-1ubuntu6.6
  Version table:
*** 0.8.8-1ubuntu6.6 0
        500 http://ca.archive.ubuntu.com/ubuntu/ natty-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     0.8.8-1ubuntu6.5 0
        500 http://ca.archive.ubuntu.com/ubuntu/ natty-updates/main amd64 Packages
        500 http://ca.archive.ubuntu.com/ubuntu/ natty-security/main amd64 Packages
     0.8.8-1ubuntu6 0
        500 http://ca.archive.ubuntu.com/ubuntu/ natty/main amd64 Packages

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: qemu-kvm 0.14.0+noroms-0ubuntu4.4
ProcVersionSignature: Ubuntu 2.6.38-12.51-generic 2.6.38.8
Uname: Linux 2.6.38-12-generic x86_64
Architecture: amd64
Date: Wed Oct 19 14:30:44 2011
KvmCmdLine: Error: command ['ps', '-C', 'kvm', '-F'] failed with exit code 1: UID PID PPID C SZ RSS PSR STIME TTY TIME CMD
MachineType: LENOVO 2516CTO
ProcEnviron:
LANGUAGE=en_US:en
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-2.6.38-12-generic root=/dev/mapper/vg0-root ro possible_cpus=4 swapaccount=0 nodelayacct pcie_aspm=force splash vt.handoff=7
SourcePackage: qemu-kvm
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 05/20/2011
dmi.bios.vendor: LENOVO
dmi.bios.version: 6IET78WW (1.38 )
dmi.board.name: 2516CTO
dmi.board.vendor: LENOVO
dmi.board.version: Not Available
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvr6IET78WW(1.38):bd05/20/2011:svnLENOVO:pn2516CTO:pvrThinkPadT410:rvnLENOVO:rn2516CTO:rvrNotAvailable:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 2516CTO
dmi.product.version: ThinkPad T410
dmi.sys.vendor: LENOVO

See original description

Tags:

Related branches

lp:ubuntu/precise/qemu-kvm

Revision history for this message

Simon Déziel (sdeziel) wrote on 2011-10-19:

BootDmesg.txt Edit (62.6 KiB, text/plain; charset="utf-8")
CurrentDmesg.txt Edit (43.8 KiB, text/plain; charset="utf-8")
Dependencies.txt Edit (2.5 KiB, text/plain; charset="utf-8")
Lspci.txt Edit (15.0 KiB, text/plain; charset="utf-8")
Lsusb.txt Edit (478 bytes, text/plain; charset="utf-8")
ProcCpuinfo.txt Edit (3.3 KiB, text/plain; charset="utf-8")
ProcInterrupts.txt Edit (2.1 KiB, text/plain; charset="utf-8")
ProcModules.txt Edit (3.3 KiB, text/plain; charset="utf-8")
RelatedPackageVersions.txt Edit (876 bytes, text/plain; charset="utf-8")
UdevDb.txt Edit (143.8 KiB, text/plain; charset="utf-8")
UdevLog.txt Edit (306.1 KiB, text/plain; charset="utf-8")

Revision history for this message

Simon Déziel (sdeziel) wrote on 2011-10-19:

KVM guest XML definition Edit (2.2 KiB, text/xml)

Here is the guest definition extracted from "virsh dumpxml oneiric"

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2011-10-19:

Thanks for reporting this bug. I can reproduce this even with the latest oneiric version.

Marking priority low since the host affects the guest (rather than guest affecting host), and since it can be worked around.

Changed in qemu-kvm (Ubuntu):
importance:	Undecided → Low
status:	New → Confirmed

Revision history for this message

Sergey Svishchev (svs) wrote on 2011-11-30:

Simon, does the guest start normally after that?

I've seen this and had to reboot the host, because 'virsh start' did not work again ("error: Timed out during operation: cannot acquire state change lock")

Revision history for this message

Simon Déziel (sdeziel) wrote on 2011-11-30:

Sergey, here is the steps I've followed :

$ virsh start oneiric
Domain oneiric started

$ virsh setvcpus oneiric --count 2 --live

$ virsh list
Id Name State
----------------------------------

At this point the guest is dead and I need to start it again :

$ virsh start oneiric
Domain oneiric started

$ virsh vcpucount oneiric
maximum config 2
maximum live 2
current config 1
current live 1

You see that the CPU that was hot plugged is not there anymore. The offline method works though :

$ virsh setvcpus oneiric --count 2 --config

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2011-12-01:

Ok, sorry, Simon, for taking so long. As it turns out, cpu hotplug is simply not yet supported. It had been implemented in qemu-kvm, but near as I can tell it was dropped when it failed to get into qemu.

However, qemu should at least fail more gracefully, or, failing that, libvirt should be refusing to ask qemu to hotplug cpus. If we can't get this fixed upstream soon, then we should carry a patch for LTS.

Changed in qemu-kvm (Ubuntu):
importance:	Low → High

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2011-12-01:

This is not a bug in qemu.git upstream because cpu_set does not exist there. It is however a bug in qemu-kvm.git.

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2011-12-01:

proposed debdiff Edit (3.6 KiB, text/plain)

We are in soft freeze so I can't upload this now, but I've rolled a fix into the following debdiff.

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2011-12-01:

Note that with the debdiff above applied to qemu, the virsh setvcpu command doesn't crash the VM any more, but it doesn't give any indication of the failure.

Changed in libvirt (Ubuntu):
status:	New → Confirmed
importance:	Undecided → Low

Revision history for this message

Launchpad Janitor (janitor) wrote on 2011-12-01:

#10

This bug was fixed in the package qemu-kvm - 0.15.0+noroms-0ubuntu7

---------------
qemu-kvm (0.15.0+noroms-0ubuntu7) precise; urgency=low

  * debian/qemu-ifdown: don't use full paths for sbin/ifconfig, especially
    as those paths are wrong. (LP: #898234)
  * debian/qemu-kvm.default and debian/qemu-kvm.upstart: optionally load
    the vhost_net module.
  * debian/patches/dont-try-to-hotplug-cpu.patch: trying to hotplug a cpu
    crashes qemu. So just don't do it! (LP: #878422)
-- Serge Hallyn <email address hidden> Wed, 30 Nov 2011 11:37:28 -0600

Changed in qemu-kvm (Ubuntu):
status:	Confirmed → Fix Released

Ubuntu Foundations Team Bug Bot (crichton) on 2011-12-02

tags:

added: patch

Serge Hallyn (serge-hallyn) on 2011-12-05

Changed in libvirt (Ubuntu):
status:	Confirmed → Invalid

Serge Hallyn (serge-hallyn) on 2011-12-06

description:

updated

Serge Hallyn (serge-hallyn) on 2011-12-06

description:

updated

Revision history for this message

Martin Pitt (pitti) wrote on 2011-12-09: Please test proposed package

#11

Hello Simon, or anyone else affected,

Accepted qemu-kvm into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in qemu-kvm (Ubuntu Oneiric):
status:	New → Fix Committed
tags:	added: verification-needed

Revision history for this message

Simon Déziel (sdeziel) wrote on 2011-12-09:

#12

Martin, I unfortunately cannot test right now as my laptop runs Natty and I cannot upgrade to Oneiric yet. I'll give that a try as soon as I switch to Oneiric. If this patch lands in natty-proposed, I'll be happy to test this immediately.

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2011-12-09:

#13

Tests fine for me on oneiric, fwiw.

Jean-Baptiste Lallement (jibel) on 2011-12-09

tags:

added: verification-done
removed: verification-needed

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2012-01-06:

#14

Verified on oneiric.

(qemu) cpu_set 2 online
cpu hotplug not supported
(qemu) quit

Revision history for this message

Launchpad Janitor (janitor) wrote on 2012-01-09:

#15

This bug was fixed in the package qemu-kvm - 0.14.1+noroms-0ubuntu6.1

---------------
qemu-kvm (0.14.1+noroms-0ubuntu6.1) oneiric-proposed; urgency=low

  * debian/qemu-ifdown: don't use full paths for sbin/ifconfig, especially
    as those paths are wrong. (LP: #898234)
  * debian/patches/dont-try-to-hotplug-cpu.patch: trying to hotplug a cpu
    crashes qemu. So just don't do it! (LP: #878422)
-- Serge Hallyn <email address hidden> Tue, 06 Dec 2011 14:57:13 -0600

Changed in qemu-kvm (Ubuntu Oneiric):
status:	Fix Committed → Fix Released

Revision history for this message

Rolf Leggewie (r0lf) wrote on 2014-12-03:

#16

oneiric has seen the end of its life and is no longer receiving any updates. Marking the oneiric task for this ticket as "Won't Fix".

Changed in libvirt (Ubuntu Oneiric):
status:	New → Won't Fix

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Patches

proposed debdiff Edit

Add patch

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Ubuntuqemu-kvm package

CPU hot plug kills the guest

Bug Description

Related branches

Other bug subscribers

Patches

Bug attachments

Remote bug watches

Ubuntu
qemu-kvm package