internal error: Cannot find suitable emulator for ppc64le

Bug #1418221 reported by Scott Moser
26
This bug affects 3 people
Affects Status Importance Assigned to Milestone
libvirt (Ubuntu)
Fix Released
High
Unassigned
Utopic
Fix Released
High
Unassigned

Bug Description

Impact; cannot start kvm vms through libvirt on ppc64le
Test case: start kvm vm through nova
Regression potential: this patch is cherrypicked from upstream and only affects how the emulator is chosen for ppc systems, so there should be no regressions.

on ppc64el (with 'uname -m' of ppc64le), libvirt will complain:
 libvirt: QEMU Driver error : internal error: Cannot find suitable emulator for ppc64le

This is easily reproducible with the python-libvirt bindings:

$ sudo apt-get install python-libvirt libvirtd-bin qemu-system-ppc
$ python -c 'import libvirt; libvirt.open().getVersion(); print("GOOD")'
libvirt: QEMU Driver error : internal error: Cannot find suitable emulator for ppc64le
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 3679, in getVersion
    if ret == -1: raise libvirtError ('virConnectGetVersion() failed', conn=self)
libvirt.libvirtError: internal error: Cannot find suitable emulator for ppc64le

The issue here is that it raises exception, which is not expected in nova-compute at least.

A workaround can be acheived via:
$ sudo ln -s /usr/bin/qemu-system-ppc64 /usr/bin/qemu-system-ppc64le
$ sudo stop libvirt-bin; sudo start libvirt-bin

Revision history for this message
Scott Moser (smoser) wrote :

this hack patch does make the provided test case pass.

Ryan Beisner (1chb1n)
tags: added: openstack uosci
Revision history for this message
Serge Hallyn (serge-hallyn) wrote : Re: [Bug 1418221] [NEW] internal error: Cannot find suitable emulator for ppc64le

So it sounds like commit da636d83dc6b1d070a705786b4daef8644eaca13 would
fix it?

tags: added: patch
Revision history for this message
Scott Moser (smoser) wrote :

Serge,
  1.2.12 has the change you pointed at.
  I believe that what that patch really does is give big endian ppc64 the ability to spawn little endian vms (via qemu-system-ppc64).
  What we need is the recognition that libvirt running on ppc64le should by default use the ppc64 machine capability. That should spawn ppc64le and ppc64[be] machines using qemu-system-ppc64. As it is right now, it defaults to expecting to run vms via 'qemu-system-ppc64le' by default, a program that isn't going to exist.

the hack patch i have there basically just patches the "qemu default emulator" to be qemu-system-ppc64.

Revision history for this message
Scott Moser (smoser) wrote :

Attaching my current hacky patch that gets us fairly close to functional openstack nova-compute on ppc64el.

Changed in libvirt (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Serge Hallyn (serge-hallyn) wrote : Re: [Bug 1418221] Re: internal error: Cannot find suitable emulator for ppc64le

Thanks, Scott, I'll push this to the package. Have you discussed this
patch on the mailing list?

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 1.2.12-0ubuntu3

---------------
libvirt (1.2.12-0ubuntu3) vivid; urgency=medium

  * Apply patch from smoser to make libvirt on ppc64le functional.
    (LP: #1418221)
 -- Serge Hallyn <email address hidden> Mon, 09 Feb 2015 12:09:49 -0600

Changed in libvirt (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Prerna (prerna) wrote :

The patch mentioned in comment #1 is hackish, and does not address the problem completely.
Libvirt seems to fail on ubuntu because it does not find an appropriate emulator for ppc64le in course of capabilities probe:

# virsh capabilities

...[snip]:
<guest>
    <os_type>hvm</os_type>
    <arch name='ppc64'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-ppc64</emulator>
      <machine maxCpus='256'>pseries</machine>
      <machine maxCpus='1'>ref405ep</machine>
      ....
</guest>.
<guest>
    <os_type>hvm</os_type>
    <arch name='ppcemb'>
 ...
</guest>

However, when I run upstream libvirt on another little endian distro such as Fedora 21, I explicitly see a listing for 'ppc64le' guest :
<guest>
    <os_type>hvm</os_type>
    <arch name='ppc64'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-ppc64</emulator>
      <machine maxCpus='256'>pseries</machine>
      <machine maxCpus='1'>ref405ep</machine>
      <machine maxCpus='1'>virtex-ml507</machine>
      <machine maxCpus='32'>ppce500</machine>
     ....
</guest>
 <guest>
    <os_type>hvm</os_type>
    <arch name='ppc64le'> ==> See that an explicit emulator is listed under '<emulator>' tag
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-kvm</emulator>
      <machine maxCpus='256'>pseries</machine>
      <machine maxCpus='1'>ref405ep</machine>
      <machine maxCpus='1'>virtex-ml507</machine>
      <machine maxCpus='32'>ppce500</machine>
     ...
</guest>
<guest>
    <os_type>hvm</os_type>
    <arch name='ppcemb'>
 ...
</guest>

I'm investigating why virsh capabilities omits a guest listing for ppc64el arch on Ubuntu.

Revision history for this message
Scott Moser (smoser) wrote :

Prerna,
  The interesting thing that I see in your virsh capabilities output is that you're using 'qemu-kvm' for the emulator for ppc64el. Ubuntu does not provide 'qemu-kvm' as a binary or symlink to the appropriate executable.
Also, as I suggested in the bug opening, a symlink from qemu-system-ppc64el to qemu-system-ppc64 does get us past the initial issue, but I believe I still see some bugs doing that...

Revision history for this message
Prerna (prerna) wrote :

Hi Scott,
Libvirt code tends to look for kvm binaries when the guest architecture matches the host.
It tries to find one of the following in path:
- qemu-kvm
-kvm

1) A symlink called 'qemu-kvm' from qemu-system-ppc64 fixes it. Would that not be an easy fix to have ?

2) If not, maybe we can explicitly associate 'qemu-system-ppc64' as the binary of choice for PowerPC LE hosts too.

I can cook up a patch for #2 and post it upstream if that is what you'd prefer.

Revision history for this message
Scott Moser (smoser) wrote :

So here is what we've found.
 a.) qemu-kvm package provides a 'kvm' executable that ends up fixing this issue. However, that requires someone to install qemu-kvm (may or may not be ok).
 b.) symlinking /usr/bin/qemu-system-ppc64le -> qemu-system-ppc64 also seems to solve this issue.

I had done 'b' before, but had some fallout, I believe related to a different bug that we've not yet filed.

So I think that we're going to put 'b' in place as it doesn't seem terribly unreasonable.

Prena, do you disagree with that work around?

Revision history for this message
Prerna (prerna) wrote :

Hi Scott,
This workaround may not be required anymore. I have posted a patch to fix this issue for PPC64 guests : https://www.redhat.com/archives/libvir-list/2015-February/msg00484.html.

Could you pull this into your tree ?
Also, I'd recommend pulling all 4 patches in this series-- if not, at least 1,2,3 are all must-fixes.

Regards,
Prerna

Revision history for this message
Prerna (prerna) wrote :

This is now a part of upstream libvirt :
------------------------------------------

commit bdbe723fcd8b55678b0ad984134f7b79d6fbeb89
Author: Prerna Saxena <email address hidden>
Date: Sun Feb 15 09:48:00 2015 +0530

    PowerPC : Make 'qemu-system-ppc64' the default emulator on ppc64[le].

    PowerPC : Explicitly associate 'qemu-system-ppc64' as the
     default emulator for all 64-bit PowerPC guests ( both Big & Little Endian )

    Signed-off-by: Prerna Saxena <email address hidden>

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Prerna,

I believe this bug is already fixed in vivid, as of 1:2.2+dfsg-5expubuntu4
qemu-system-ppc64le is a symbolic link to qemu-system-ppc. Can you confirm
whether the bug is fixed? If not, I'll apply your patch in vivid.

Did you need this applied in utopic?

Revision history for this message
Mauro S M Rodrigues (maurorodrigues) wrote :

Guys can you please backport this to utopic?

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

Will do

Changed in libvirt (Ubuntu Utopic):
importance: Undecided → High
description: updated
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Scott, or anyone else affected,

Accepted libvirt into utopic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/1.2.8-0ubuntu11.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in libvirt (Ubuntu Utopic):
status: New → Fix Committed
tags: added: verification-needed
Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

(Looks like this failed to build because we also need commit da636d83)

Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

I've pushed a new version 1.2.8-0ubuntu11.6 which fixes the ftbfs.

Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Scott, or anyone else affected,

Accepted libvirt into utopic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/1.2.8-0ubuntu11.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 1.2.8-0ubuntu11.8

---------------
libvirt (1.2.8-0ubuntu11.8) utopic; urgency=medium

  [ Seyeong Kim ]
  * d/p/fix_libvirtd_killed_by_sigsegv.patch: fix incorrect backport
    (LP: #1464175)

  [ Edward Hope-Morley ]
  * Add post-start to upstart (/etc/init/libvirt-bin.conf) and
    sysv (/etc/init.d/libvirt-bin) to ensure libvirt-sock
    created before up (LP: #1455608)

 -- Chris J Arges <email address hidden> Wed, 08 Jul 2015 11:55:28 -0500

Changed in libvirt (Ubuntu Utopic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.