euca-run-instances fails to run VM

Bug #655217 reported by Donghoon Kim
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Hisaki Ohara

Bug Description

I tried to run VM with the docs,
http://wiki.openstack.org/NovaInstall
and
http://github.com/vishvananda/novascript
the docs say using screen tools, but I tried with nohup and saved all the logs to files for error tracking.

anyway, failed when runnig euca-run-instances.

The nova-compute log messages is as follows:
===============================================
libvir: QEMU error : internal error process exited while connecting to monitor: chardev: opening backend "file" failed
ERROR:root:instance i-wt4mcqm1: Failed to spawn
Traceback (most recent call last):
  File "/root/openstack/nova/nova/compute/manager.py", line 86, in run_instance
    yield self.driver.spawn(instance_ref)
  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 823, in _inlineCallbacks
    result = g.send(result)
  File "/root/openstack/nova/nova/virt/libvirt_conn.py", line 218, in spawn
    yield self._conn.createXML(xml, 0)
  File "/usr/lib/python2.6/dist-packages/libvirt.py", line 1289, in createXML
    if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirtError: internal error process exited while connecting to monitor: chardev: opening backend "file" failed

2010-10-06 00:13:04+0900 [-] (root): ERROR instance i-wt4mcqm1: Failed to spawn
2010-10-06 00:13:04+0900 [-] Traceback (most recent call last):
2010-10-06 00:13:04+0900 [-] File "/root/openstack/nova/nova/compute/manager.py", line 86, in run_instance
2010-10-06 00:13:04+0900 [-] yield self.driver.spawn(instance_ref)
2010-10-06 00:13:04+0900 [-] File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 823, in _inlineCallbacks
2010-10-06 00:13:04+0900 [-] result = g.send(result)
2010-10-06 00:13:04+0900 [-] File "/root/openstack/nova/nova/virt/libvirt_conn.py", line 218, in spawn
2010-10-06 00:13:04+0900 [-] yield self._conn.createXML(xml, 0)
2010-10-06 00:13:04+0900 [-] File "/usr/lib/python2.6/dist-packages/libvirt.py", line 1289, in createXML
2010-10-06 00:13:04+0900 [-] if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
2010-10-06 00:13:04+0900 [-] libvirtError: internal error process exited while connecting to monitor: chardev: opening backend "file" failed
2010-10-06 00:13:04+0900 [-]
libvir: QEMU error : Domain not found: no domain with matching name 'i-wt4mcqm1'
2010-10-06 00:13:04+0900 [-] Unhandled error in Deferred:
2010-10-06 00:13:04+0900 [-] Unhandled Error
        Traceback (most recent call last):
          File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 307, in errback
            self._startRunCallbacks(fail)
          File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 354, in _startRunCallbacks
            self._runCallbacks()
          File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 371, in _runCallbacks
            self.result = callback(self.result, *args, **kw)
          File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 879, in gotResult
            _inlineCallbacks(r, g, deferred)
        --- <exception caught here> ---
          File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 821, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python2.6/dist-packages/twisted/python/failure.py", line 338, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/root/openstack/nova/nova/compute/manager.py", line 98, in run_instance
            self._update_state(context, instance_id)
          File "/root/openstack/nova/nova/compute/manager.py", line 62, in _update_state
            state = self.driver.get_info(instance_ref.name)['state']
          File "/root/openstack/nova/nova/virt/libvirt_conn.py", line 334, in get_info
            virt_dom = self._conn.lookupByName(instance_name)
          File "/usr/lib/python2.6/dist-packages/libvirt.py", line 1442, in lookupByName
            if ret is None:raise libvirtError('virDomainLookupByName() failed', conn=self)
        libvirt.libvirtError: Domain not found: no domain with matching name 'i-wt4mcqm1'
===============================================

and.. libvirt log is like this.
===============================================
root@gagamels2:/var/log/libvirt/qemu# cat i-wt4mcqm1.log
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 512 -smp 1,sockets=1,cores=1,threads=1 -name i-wt4mcqm1 -uuid 61dd3ee4-c5e2-86de-d257-56696eb2463b -nographic -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/i-wt4mcqm1.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -boot c -kernel /root/openstack/nova/nova/../instances/i-wt4mcqm1/kernel -initrd /root/openstack/nova/nova/../instances/i-wt4mcqm1/ramdisk -append 'root=/dev/vda1 console=ttyS0' -drive file=/root/openstack/nova/nova/../instances/i-wt4mcqm1/disk,if=none,id=drive-virtio-disk0,boot=on,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0 -device rtl8139,vlan=0,id=net0,mac=02:16:3e:02:4c:64,bus=pci.0,addr=0x2 -net tap,fd=63,vlan=0,name=hostnet0 -chardev file,id=serial0,path=/root/openstack/nova/nova/../instances/i-wt4mcqm1/console.log -device isa-serial,chardev=serial0 -usb -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
chardev: opening backend "file" failed
===============================================

so I tried to run the kvm command directly in command line..
but errors like this.. and kvm didn't run.
===============================================
TUNGETIFF ioctl() failed: Bad file descriptor
TUNSETOFFLOAD ioctl() failed: Bad file descriptor
qemu: could not open disk image console=ttyS0: No such file or directory
===============================================

so I changed -append option
from -append root=/dev/vda1 console=ttyS0
to -append 'root=/dev/vda1 console=ttyS0'
and tried again...
so the messages "qemu: could not open disk image console=ttyS0: No such file or directory" disappeared...
(but TUNGETIFF and TUNSETOFFLOAD related messages was still there..
I guess it is relaged with -net tap blabla option. am I wrong?)
and It seemed that kvm was running..

So, I updated append option from $NOVA_DIR/nova/nova/virt/libvirt.qemu.xml.template
and restart all the nova modules..

but same errors again...

I did search the bugs list about euca-run-instances problems, but my situation wasn't there..

What's wrong with my test and machine ?
QEMU or KVM env?

Related branches

Revision history for this message
Jay Pipes (jaypipes) wrote :

Hi Hisaki! Assigning this bug to you since you graciously supplied a hopeful fix! :) We'll review the merge proposal shortly. Thanks!

Changed in nova:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Hisaki Ohara (hisakister)
Hisaki Ohara (hisaki)
Changed in nova:
status: Triaged → Fix Released
Revision history for this message
Mika Kohonen (mika-kohonen-1986) wrote :

To fix this problem you only need to to add write permissions to the directory structure created by novascript.

If you look at the /var/log/libvirt/qemu/instance-INSTANCE_ID.log you will see the error written in the last line. It's trying to create the console.log file but there is a problem with the permissions. Probably you are running the novascript as root in a folder with root only permissions.

Revision history for this message
Ahmed El-Gamil (ahmedelgamil) wrote :

Executing the Nova script from within the /opt directory saved the permissions problem that Mika mentioned, I originally downloaded Nova in /root which may have caused the problem

Revision history for this message
Alex (alexis-letessier) wrote :

libvirt-qemu must have access to the nova/instances repository

Revision history for this message
Zachary Wasserman (zachwass2000) wrote :

I am having the same issue here. How can I give libvirt-qemu access to the nova/instances repository?

Revision history for this message
Bayanzul Lodoysamba (bayanzul) wrote :

is it write access for libvirt-qemu?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.