OpenStack Compute (nova)

openstack-nova-compute service fails with - libvirtError: internal error: CPU feature `avx' specified more than once

Bug #1267191 reported by Kashyap Chamarthy on 2014-01-08

This bug affects 12 people

	Status	Importance	Assigned to
OpenStack Compute (nova)	Invalid	Undecided	Unassigned
nova (Ubuntu)	Fix Released	Undecided	Unassigned
Trusty	Fix Released	Undecided	Unassigned

Bug Description

[Impact]

* cpu features list which is being sent to libvirt,
when creating a domain or calling compareCPU, must contain only
unique entries. Multiple issues arise when we are updating the
features attribute in LibvirtConfigCPU class (for example during
migration).

[Test Case]

* A two node openstack havana on ubuntu 14.04. Migrating a instance to other node.

- Migrate the instance
- Observe /var/log/nova/compute.log and /var/log/libvirt.log.
- /var/log/libvirtd.log: 2014-03-27 18:23:17.141+0000: 2659: error : virCPUDefParseXML:413 : XML error: CPU feature `wdt' specified more than once.

[Regression Potential]

* Same fix for 1303536 applied to Icehouse.

* Fix is running in a big production without any problem.

* A regression would continue to cause nova not to start (as is happening
today with this bug) under described conditions.

[Other Info]

--------------------------------------------
Original Description (Clone of a Fedora Bug)
--------------------------------------------

Restarting openstack-nova-compute serivce fails with:

libvirtError: internal error: CPU feature `avx' specified more than once

Version
---------

    $ rpm -q openstack-nova libvirt qemu-system-x86
    openstack-nova-2014.1-0.4.b1.fc21.noarch
    libvirt-1.1.3.2-1.fc20.x86_64
    qemu-system-x86-1.6.1-3.fc20.x86_64

Test env
----------

A two node OpenStack RDO set-up configured manually on two Fedora 20
VMs:

- Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
vSwitch plugin and GRE tunneling).

- Compute node: Nova (nova-compute), Neutron (openvswitch-agent)

How reproducible: Consistently.

Steps to Reproduce
------------------

$ systemctl restart openstack-nova-compute

Observe /var/log/nova/compute.log

Actual results
--------------

$ tail -f /var/log/nova/compute.log
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup

Expected results
----------------

Compute service should start successfully.

Additional info
---------------

Status of openstack-nova-compute service

$ systemctl status openstack-nova-compute
openstack-nova-compute.service - OpenStack Nova Compute Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
   Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
  Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
Main PID: 1529 (code=exited, status=0/SUCCESS)

Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...in doit
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup result = proxy_ca...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...xy_call
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = execute(f,*a...kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib/pyth...tworker
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup File "/usr/lib64/py...lineCPU
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup if ret is None: r...n=self)
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup

[reply] [−]
Private
Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST

More contextual trace from compute.log:

[. . .]
2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver
2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup x.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.thread.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self._exit_event.wait()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.greenlet.switch()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup service.start()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes())
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.update_status()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities()
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup
2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
[. . .]

See original description

Tags:

Related branches

lp:~corey.bryant/nova/2014.1.2

Merged into lp:~ubuntu-server-dev/nova/icehouse at revision 694

Ubuntu Server Developers: Pending requested 2014-08-12

lp:~ubuntu-server-dev/nova/icehouse

Revision history for this message

Kashyap Chamarthy (kashyapc) wrote on 2014-01-08:

This is a clone of -- Fedora/RDO related bug -- https://bugzilla.redhat.com/show_bug.cgi?id=1049391

Matt Riedemann (mriedem) on 2014-01-10

tags:

added: libvirt

Revision history for this message

Kyle Mestery (mestery) wrote on 2014-01-15:

With a stock Fedora 20 environment and using devstack with the latest upstream OpenStack components, I see something very similar, basically the same error with a different CPU flag. See the paste here:

http://paste.openstack.org/show/61278/

Revision history for this message

Kashyap Chamarthy (kashyapc) wrote on 2014-01-16:

As pastebins expire, posting the actual error & CPU flag Kyle is seeing:

[. . .]
2014-01-15 09:33:56.530 20957 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
2014-01-15 09:33:56.530 20957 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
2014-01-15 09:33:56.530 20957 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `tsc-deadline' specified more than once
2014-01-15 09:33:56.530 20957 TRACE nova.openstack.common.threadgroup
[. . .]

Also, in the referenced downstream bug, another reporter (Attila Fazekas) mentioned a different CPU flag:

[. . .]
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
libvirt.libvirtError: internal error: CPU feature `rdtscp' specified more than once
[. . .]

Revision history for this message

Vladik Romanovsky (vladik-romanovsky) wrote on 2014-01-16:

Could you please also post: /proc/cpuinfo and virsh capabilities

Thanks.

Revision history for this message

Davanum Srinivas (DIMS) (dims-v) wrote on 2014-01-17:

https://bugzilla.redhat.com/show_bug.cgi?id=1049391#c11 points at a possible libvirtd bug

Revision history for this message

Kashyap Chamarthy (kashyapc) wrote on 2014-01-23:

Download full text (8.0 KiB)

@Vladic, I should note that I was running in an Nested Virtualization (KVM on KVM, Intel) environment.

Here we go, from Compute host (which is a guest-hypervisor, running Fedora 20):

1. /proc/cpuinfo:

$ cat /proc/cpuinfo | head -25
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 42
model name : Intel Xeon E312xx (Sandy Bridge)
stepping : 1
microcode : 0x1
cpu MHz : 1864.644
cache size : 4096 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm
bogomips : 3729.28
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
-------------

2. And, capabilities of driver (libvirt/KVM):

$ virsh capabilities
<capabilities>

  <host>
    <uuid>26eededc-a845-48e8-998d-86bb14a89bbf</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Westmere</model>
      <vendor>Intel</vendor>
      <topology sockets='20' cores='1' threads='1'/>
      <feature name='rdtscp'/>
      <feature name='pdpe1gb'/>
      <feature name='hypervisor'/>
      <feature name='x2apic'/>
      <feature name='pcid'/>
      <feature name='vmx'/>
      <feature name='pclmuldq'/>
      <feature name='ss'/>
      <feature name='vme'/>
    </cpu>
    <power_management>
      <suspend_disk/>
      <suspend_hybrid/>
    </power_management>
    <migration_features>
      <live/>
      <uri_transports>
        <uri_transport>tcp</uri_transport>
      </uri_transports>
    </migration_features>
    <topology>
      <cells num='1'>
        <cell id='0'>
          <memory unit='KiB'>19092304</memory>
          <cpus num='20'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
            <cpu id='1' socket_id='1' core_id='0' siblings='1'/>
            <cpu id='2' socket_id='2' core_id='0' siblings='2'/>
            <cpu id='3' socket_id='3' core_id='0' siblings='3'/>
            <cpu id='4' socket_id='4' core_id='0' siblings='4'/>
            <cpu id='5' socket_id='5' core_id='0' siblings='5'/>
            <cpu id='6' socket_id='6' core_id='0' siblings='6'/>
            <cpu id='7' socket_id='7' core_id='0' siblings='7'/>
            <cpu id='8' socket_id='8' core_id='0' siblings='8'/>
            <cpu id='9' socket_id='9' core_id='0' siblings='9'/>
            <cpu id='10' socket_id='10' core_id='0' siblings='10'/>
            <cpu id='11' socket_id='11' core_id='0' siblings='11'/>
            <cpu id='12' socket_id='12' core_id='0' siblings='12'/>
            <cpu id='13' socket_id='13' core_id='0' siblings='13'/>
            <cpu id='14' socket_id='14' core_id='0' siblings='14'/>
            <cpu id='15' socket_id='15' core_id='0' siblings='15'/>
            <cpu id='16' socket_id='16' core_id='0' siblings='16'/>
            <cpu id='17' socke...

@Vladic, I should note that I was running in an Nested Virtualization (KVM on KVM, Intel) environment.

Here we go, from Compute host (which is a guest-hypervisor, running Fedora 20):

1. /proc/cpuinfo:

$ cat /proc/cpuinfo  | head -25
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel Xeon E312xx (Sandy Bridge)
stepping        : 1
microcode       : 0x1
cpu MHz         : 1864.644
cache size      : 4096 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm
bogomips        : 3729.28
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
-------------

2. And, capabilities of driver (libvirt/KVM):

$ virsh capabilities
<capabilities>

<guest>
    <os_type>hvm</os_type>
    <arch name='i686'>
      <wordsize>32</wordsize>
      <emulator>/usr/bin/qemu-system-i386</emulator>
      <machine canonical='pc-i440fx-1.6' maxCpus='255'>pc</machine>
      <machine maxCpus='255'>pc-q35-1.4</machine>
      <machine maxCpus='255'>pc-q35-1.5</machine>
      <machine canonical='pc-q35-1.6' maxCpus='255'>q35</machine>
      <machine maxCpus='1'>isapc</machine>
      <machine maxCpus='255'>pc-0.10</machine>
      <machine maxCpus='255'>pc-0.11</machine>
      <machine maxCpus='255'>pc-0.12</machine>
      <machine maxCpus='255'>pc-0.13</machine>
      <machine maxCpus='255'>pc-0.14</machine>
      <machine maxCpus='255'>pc-0.15</machine>
      <machine maxCpus='255'>pc-1.0</machine>
      <machine maxCpus='255'>pc-1.1</machine>
      <machine maxCpus='255'>pc-1.2</machine>
      <machine maxCpus='255'>pc-1.3</machine>
      <machine maxCpus='255'>pc-i440fx-1.4</machine>
      <machine maxCpus='255'>pc-i440fx-1.5</machine>
      <machine maxCpus='1'>none</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/bin/qemu-kvm</emulator>
        <machine canonical='pc-i440fx-1.6' maxCpus='255'>pc</machine>
        <machine maxCpus='255'>pc-q35-1.4</machine>
        <machine maxCpus='255'>pc-q35-1.5</machine>
        <machine canonical='pc-q35-1.6' maxCpus='255'>q35</machine>
        <machine maxCpus='1'>isapc</machine>
        <machine maxCpus='255'>pc-0.10</machine>
        <machine maxCpus='255'>pc-0.11</machine>
        <machine maxCpus='255'>pc-0.12</machine>
        <machine maxCpus='255'>pc-0.13</machine>
        <machine maxCpus='255'>pc-0.14</machine>
        <machine maxCpus='255'>pc-0.15</machine>
        <machine maxCpus='255'>pc-1.0</machine>
        <machine maxCpus='255'>pc-1.1</machine>
        <machine maxCpus='255'>pc-1.2</machine>
        <machine maxCpus='255'>pc-1.3</machine>
        <machine maxCpus='255'>pc-i440fx-1.4</machine>
        <machine maxCpus='255'>pc-i440fx-1.5</machine>
        <machine maxCpus='1'>none</machine>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
      <pae/>
      <nonpae/>
    </features>
  </guest>

<guest>
    <os_type>hvm</os_type>
    <arch name='x86_64'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-x86_64</emulator>
      <machine canonical='pc-i440fx-1.6' maxCpus='255'>pc</machine>
      <machine maxCpus='255'>pc-q35-1.4</machine>
      <machine maxCpus='255'>pc-q35-1.5</machine>
      <machine canonical='pc-q35-1.6' maxCpus='255'>q35</machine>
      <machine maxCpus='1'>isapc</machine>
      <machine maxCpus='255'>pc-0.10</machine>
      <machine maxCpus='255'>pc-0.11</machine>
      <machine maxCpus='255'>pc-0.12</machine>
      <machine maxCpus='255'>pc-0.13</machine>
      <machine maxCpus='255'>pc-0.14</machine>
      <machine maxCpus='255'>pc-0.15</machine>
      <machine maxCpus='255'>pc-1.0</machine>
      <machine maxCpus='255'>pc-1.1</machine>
      <machine maxCpus='255'>pc-1.2</machine>
      <machine maxCpus='255'>pc-1.3</machine>
      <machine maxCpus='255'>pc-i440fx-1.4</machine>
      <machine maxCpus='255'>pc-i440fx-1.5</machine>
      <machine maxCpus='1'>none</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/bin/qemu-kvm</emulator>
        <machine canonical='pc-i440fx-1.6' maxCpus='255'>pc</machine>
        <machine maxCpus='255'>pc-q35-1.4</machine>
        <machine maxCpus='255'>pc-q35-1.5</machine>
        <machine canonical='pc-q35-1.6' maxCpus='255'>q35</machine>
        <machine maxCpus='1'>isapc</machine>
        <machine maxCpus='255'>pc-0.10</machine>
        <machine maxCpus='255'>pc-0.11</machine>
        <machine maxCpus='255'>pc-0.12</machine>
        <machine maxCpus='255'>pc-0.13</machine>
        <machine maxCpus='255'>pc-0.14</machine>
        <machine maxCpus='255'>pc-0.15</machine>
        <machine maxCpus='255'>pc-1.0</machine>
        <machine maxCpus='255'>pc-1.1</machine>
        <machine maxCpus='255'>pc-1.2</machine>
        <machine maxCpus='255'>pc-1.3</machine>
        <machine maxCpus='255'>pc-i440fx-1.4</machine>
        <machine maxCpus='255'>pc-i440fx-1.5</machine>
        <machine maxCpus='1'>none</machine>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
    </features>
  </guest>

</capabilities>

Revision history for this message

Kashyap Chamarthy (kashyapc) wrote on 2014-01-23:

Also, a Libvirt developer on the downstream bug (mentioned above & here):

https://bugzilla.redhat.com/show_bug.cgi?id=1049391#c11

says:

"this is apparently a bug in libvirtd code that handles VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES."

Davanum Srinivas (DIMS) (dims-v) on 2014-01-27

tags:

added: libvirt1x

Revision history for this message

Kashyap Chamarthy (kashyapc) wrote on 2014-02-03:

Update:

tl;dr -- Fix for this (libvirt bug) is committed in upstream; available from this libvirt version of Fedora -- libvirt-1.1.3.3-4.fc20

Upstream libvirt bug fix -- https://www.redhat.com/archives/libvir-list/2014-January/msg01314.html

Test
-----

1. Stop all OpenStack services (on both Controller and Compute nodes)

$ openstack-service stop

2. Update to libvirt RPMs, version -- libvirt-1.1.3.3-4.fc20

3. Restart Libvirt on Compute node:

$ systemctl restart libvirtd

4. Restart OpenStack Nova compute (it should come up successfully)

    $ systemctl status openstack-nova-compute -l
openstack-nova-compute.service - OpenStack Nova Compute Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
   Active: active (running) since Tue 2014-01-28 06:50:10 EST; 2s ago
Main PID: 2482 (nova-compute)
   CGroup: /system.slice/openstack-nova-compute.service
           └─2482 /usr/bin/python /usr/bin/nova-compute --logfile /var/log/nova/compute.log

Jan 28 06:50:10 node2-compute systemd[1]: Started OpenStack Nova Compute Server.$ openstack-service stop

Jan 28 06:50:10 node2-compute systemd[1]: Started OpenStack Nova Compute Server.

Kashyap Chamarthy (kashyapc) on 2014-02-03

Changed in nova:
status:	New → Incomplete
status:	Incomplete → Fix Committed

Revision history for this message

Kashyap Chamarthy (kashyapc) wrote on 2014-02-03:

Hmm, just wondering. . .

Ideally, these kind of bugs under the hood technologies that affect OpenStack could be closed as reason -- "CLOSED, UPSTREAM"

But, closing it as "Fix Committed" is a misleading -- as there's no actual fix committed in openstack-nova.

So, closing it as "Invalid, Not a bug" (technically, this is right -- as it's not a Nova bug; but a bug in libvirt that _affected_ Nova).

Changed in nova:
status:	Fix Committed → Invalid

Revision history for this message

Rafael David Tinoco (rafaeldtinoco) wrote on 2014-06-26:

#10

Here goes my SRU proposal:

nova (1:2014.1.1-0ubuntu2) trusty; urgency=high

  * Cherry picked two fixes from upstream.
    - [2cebfd2] libvirt: convert cpu features attribute from list to a set (LP: #1267191)
    - [b86a0e5] Fixes rdb backend image size (LP: #1219658)

I've created a hotfix from 1:2014.1.1-0ubuntu1 (already in -proposed) for some other uses.

Since I'm creating 1 fix for 2 bugs, please take a look at url bellow for debdiff:

https://bugs.launchpad.net/nova/+bug/1219658

Rafael David Tinoco (rafaeldtinoco) on 2014-07-29

description:	updated
description:	updated

Rafael David Tinoco (rafaeldtinoco) on 2014-07-29

description:

updated

Chris J Arges (arges) on 2014-07-29

Changed in nova (Ubuntu):
status:	New → Fix Released

Revision history for this message

Chris J Arges (arges) wrote on 2014-07-29: Please test proposed package

#11

Hello Kashyap, or anyone else affected,

Accepted nova into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/1:2014.1.1-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Trusty):
status:	New → Fix Committed
tags:	added: verification-needed

Revision history for this message

Rafael David Tinoco (rafaeldtinoco) wrote on 2014-08-06:

#12

Changed this to verification-done because since this SRU proposal this has been running in a big server farm without any issues or regressions.

tags:

added: verification-done
removed: verification-needed

Revision history for this message

Scott Kitterman (kitterman) wrote on 2014-08-06: Update Released

#13

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message

Launchpad Janitor (janitor) wrote on 2014-08-06:

#14

This bug was fixed in the package nova - 1:2014.1.1-0ubuntu2

---------------
nova (1:2014.1.1-0ubuntu2) trusty; urgency=high

  * Cherry picked two fixes from upstream.
    - [2cebfd2] libvirt: convert cpu features attribute from list to a set (LP: #1267191)
    - [b86a0e5] Fixes rdb backend image size (LP: #1219658)
-- Rafael David Tinoco <email address hidden> Wed, 25 Jun 2014 12:56:15 -0300

Changed in nova (Ubuntu Trusty):
status:	Fix Committed → Fix Released

刘志远 (351843010-k) on 2016-01-26

summary:

- openstack-nova-compute service fails with - libvirtError: internal
+ openstack-nova-compute service fails with - libvirtError: internal
error: CPU feature `avx' specified more than once