OpenStack Compute (nova)

InstanceNotFound error in c-cpu log after good tempest run

Bug #1084699 reported by David Kranz on 2012-11-29

This bug report is a duplicate of: Bug #1084698: "Failed storing info cache" in n-cpu log after good tempest run. Edit Remove

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	In Progress	Undecided	Davanum Srinivas (DIMS)

Bug Description

From http://logs.openstack.org/periodic/periodic-tempest-devstack-vm-check-hourly/404/logs/screen-n-cpu.txt
There are 11 of these.

2012-11-29 19:35:08 DEBUG nova.compute.manager [req-38d0d1e6-a4de-49bd-a312-7680dafdc3e5 ServerPersonalityTestJSON-user ServerPersonalityTestJSON-tenant] [instance: 84ba75b1-f721-44f5-96ef-71c27a002a00] Instance has been destroyed from under us while trying to set it to ERROR _set_instance_error_state /opt/stack/nova/nova/compute/manager.py:353
2012-11-29 19:35:08 29538 DEBUG nova.openstack.common.rpc.amqp [-] Making asynchronous call on conductor ... multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:352
2012-11-29 19:35:08 29538 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is d43e8f494afb4928bf76ddb73bbfba28 multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:355
2012-11-29 19:35:08 DEBUG nova.utils [req-f86b71df-9f36-4fa5-aaf6-cc1c09c6c770 ServerMetadataTest-user ServerMetadataTest-tenant] Result was 0 execute /opt/stack/nova/nova/utils.py:209
2012-11-29 19:35:08 DEBUG nova.network.linux_net [req-f86b71df-9f36-4fa5-aaf6-cc1c09c6c770 ServerMetadataTest-user ServerMetadataTest-tenant] IPTablesManager.apply completed with success _apply /opt/stack/nova/nova/network/linux_net.py:381
2012-11-29 19:35:08 29538 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 276, in _process_data
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/exception.py", line 115, in wrapped
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp temp_level, payload)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/exception.py", line 90, in wrapped
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return f(*args, **kw)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 209, in decorated_function
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp pass
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 195, in decorated_function
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 224, in decorated_function
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 954, in run_instance
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp do_run_instance()
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 228, in inner
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp retval = f(*args, **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 953, in do_run_instance
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp admin_password, is_first_time, node, instance)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 630, in _run_instance
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp self._set_instance_error_state(context, instance['uuid'])
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 586, in _run_instance
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp self._start_building(context, instance)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 813, in _start_building
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp None))
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 340, in _instance_update
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/conductor/api.py", line 69, in instance_update
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp updates)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/conductor/rpcapi.py", line 46, in instance_update
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp updates=updates_p))
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/proxy.py", line 80, in call
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return rpc.call(context, self._get_topic(topic), msg, timeout)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 108, in call
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return _get_impl().call(cfg.CONF, context, topic, msg, timeout)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/impl_kombu.py", line 754, in call
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp rpc_amqp.get_connection_pool(conf, Connection))
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 369, in call
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp rv = list(rv)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 337, in __iter__
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp raise result
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp InstanceNotFound_Remote: Instance 84ba75b1-f721-44f5-96ef-71c27a002a00 could not be found.
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 276, in _process_data
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/conductor/manager.py", line 60, in instance_update
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp context, instance_uuid, updates)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/api.py", line 685, in instance_update_and_get_original
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp values)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 127, in wrapper
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return f(*args, **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1831, in instance_update_and_get_original
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp copy_old_instance=True)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1842, in _instance_update
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp session=session)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 127, in wrapper
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp return f(*args, **kwargs)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 1493, in instance_get_by_uuid
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp raise exception.InstanceNotFound(instance_id=uuid)
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp InstanceNotFound: Instance 84ba75b1-f721-44f5-96ef-71c27a002a00 could not be found.
2012-11-29 19:35:08 29538 TRACE nova.openstack.common.rpc.amqp

Davanum Srinivas (DIMS) (dims-v) on 2012-12-06

Changed in nova:
assignee:	nobody → Davanum Srinivas (dims-v)

OpenStack Infra (hudson-openstack) on 2012-12-06

Changed in nova:
status:	New → In Progress

Revision history for this message

Davanum Srinivas (DIMS) (dims-v) wrote on 2012-12-06:

same problem as bug 1084698

Report a bug

This report contains Public information

Everyone can see this information.

Duplicate of bug #1084698 Remove

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.