Ubuntu
neutron package

Neutron has an empty database after deploying juno on utopic

Bug #1372893 reported by Ryan Beisner
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron (Ubuntu)
Invalid
Undecided
Unassigned
neutron-api (Juju Charms Collection)
Fix Released
High
Liam Young
nova-cloud-controler (Juju Charms Collection)
Fix Released
High
Liam Young

Bug Description

Symptom:
$ nova secgroup-list
ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500)

Cause:
Neutron has an empty database when deploying juno on utopic.

Recreate:
 * juju deployed juno on utopic via next.yaml in openstack-charm-testing
 * attempted to list or add security groups, found: ERROR (ClientException)
 * Inspected nova api log, found: ConnectionFailed: Connection to neutron failed: HTTPConnectionPool
 * Confirmed neutron-server service status was ok.
 * Inspected neutron server log, found: "Table 'neutron.ml2_gre_allocations' doesn't exist")
 * Connected to mysql as neutron user, found 0 tables in the neutron database.

This appears to be by design upstream, and in resolution of https://bugs.launchpad.net/neutron/+bug/1207402.
See patch https://review.openstack.org/#/c/40296.

Add'l debug details follow.

##### version info
$ neutron --version
2.3.8

$ dpkg -l | grep neutron
neutron-common:
  Installed: 1:2014.2~b3-0ubuntu3
  Candidate: 1:2014.2~b3-0ubuntu3
  Version table:
 *** 1:2014.2~b3-0ubuntu3 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages
        100 /var/lib/dpkg/status

$ apt-cache policy neutron-common
ii neutron-common 1:2014.2~b3-0ubuntu3 all Neutron is a virtual network service for Openstack - common
ii neutron-plugin-ml2 1:2014.2~b3-0ubuntu3 all Neutron is a virtual network service for Openstack - ML2 plugin
ii neutron-plugin-openvswitch-agent 1:2014.2~b3-0ubuntu3 all Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
ii python-neutron 1:2014.2~b3-0ubuntu3 all Neutron is a virutal network service for Openstack - Python library
ii python-neutronclient 1:2.3.8-0ubuntu1 all client - Neutron is a virtual network service for Openstack

##### nova api log, following command: nova secgroup-list
2014-09-22 17:09:29.296 27708 INFO nova.osapi_compute.wsgi.server [req-dc732ec4-e711-4ecc-adc4-a691edf51b5c None] 10.5.0.47 "GET /v2/4caaba56adf746cbb58a1516f2de913d/servers/detail HTTP/1.1" status: 200 len: 123 time: 0.1823571
2014-09-22 17:09:40.415 27708 INFO nova.osapi_compute.wsgi.server [req-088afc89-5366-4acf-896d-a54943976e5c None] 10.5.0.47 "GET /v2/4caaba56adf746cbb58a1516f2de913d/os-keypairs HTTP/1.1" status: 200 len: 124 time: 0.0216541
2014-09-22 17:09:51.342 27708 ERROR nova.network.security_group.neutron_driver [req-615fea13-ef7f-4897-93c6-e57ef69c1ecd None] Neutron Error getting security groups
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver Traceback (most recent call last):
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 158, in list
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver security_groups = neutron.list_security_groups(**search_opts).get(
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 99, in with_params
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver ret = self.function(instance, *args, **kwargs)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 481, in list_security_groups
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver retrieve_all, **_params)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1335, in list
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver for r in self._pagination(collection, path, **params):
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1348, in _pagination
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver res = self.get(path, params=params)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1321, in get
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver headers=headers, params=params)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1298, in retry_request
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver headers=headers, params=params)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1241, in do_request
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver content_type=self.content_type())
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 180, in do_request
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver **kwargs)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 133, in _cs_request
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver raise exceptions.ConnectionFailed(reason=e)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver ConnectionFailed: Connection to neutron failed: HTTPConnectionPool(host='10.207.1.89', port=9696): Max retries exceeded with url: /v2.0/security-groups.json?tenant_id=4caaba56adf746cbb58a1516f2de913d (Caused by <class 'socket.error'>: [Errno 111] ECONNREFUSED)
2014-09-22 17:09:51.342 27708 TRACE nova.network.security_group.neutron_driver
2014-09-22 17:09:51.350 27708 ERROR nova.api.openstack [req-615fea13-ef7f-4897-93c6-e57ef69c1ecd None] Caught error: Connection to neutron failed: HTTPConnectionPool(host='10.207.1.89', port=9696): Max retries exceeded with url: /v2.0/security-groups.json?tenant_id=4caaba56adf746cbb58a1516f2de913d (Caused by <class 'socket.error'>: [Errno 111] ECONNREFUSED)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack Traceback (most recent call last):
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 124, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return req.get_response(self.application)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack application, catch_exc_info=False)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack app_iter = application(self.environ, start_response)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return resp(environ, start_response)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 663, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return self.app(env, start_response)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return resp(environ, start_response)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return resp(environ, start_response)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack response = self.app(environ, start_response)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return resp(environ, start_response)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack resp = self.call_func(req, *args, **self.kwargs)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return self.func(req, *args, **kwargs)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 908, in __call__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack content_type, body, accept)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 974, in _process_stack
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack action_result = self.dispatch(meth, request, action_args)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 1058, in dispatch
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack return method(req=request, **action_args)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/security_groups.py", line 312, in index
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack search_opts=search_opts)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 162, in list
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack LOG.exception(_("Neutron Error getting security groups"))
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack six.reraise(self.type_, self.value, self.tb)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/nova/network/security_group/neutron_driver.py", line 158, in list
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack security_groups = neutron.list_security_groups(**search_opts).get(
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 99, in with_params
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack ret = self.function(instance, *args, **kwargs)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 481, in list_security_groups
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack retrieve_all, **_params)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1335, in list
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack for r in self._pagination(collection, path, **params):
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1348, in _pagination
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack res = self.get(path, params=params)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1321, in get
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack headers=headers, params=params)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1298, in retry_request
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack headers=headers, params=params)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1241, in do_request
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack content_type=self.content_type())
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 180, in do_request
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack **kwargs)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 133, in _cs_request
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack raise exceptions.ConnectionFailed(reason=e)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack ConnectionFailed: Connection to neutron failed: HTTPConnectionPool(host='10.207.1.89', port=9696): Max retries exceeded with url: /v2.0/security-groups.json?tenant_id=4caaba56adf746cbb58a1516f2de913d (Caused by <class 'socket.error'>: [Errno 111] ECONNREFUSED)
2014-09-22 17:09:51.350 27708 TRACE nova.api.openstack
2014-09-22 17:09:51.359 27708 INFO nova.api.openstack [req-615fea13-ef7f-4897-93c6-e57ef69c1ecd None] http://10.207.1.80:8774/v2/4caaba56adf746cbb58a1516f2de913d/os-security-groups returned with HTTP 500
2014-09-22 17:09:51.361 27708 INFO nova.osapi_compute.wsgi.server [req-615fea13-ef7f-4897-93c6-e57ef69c1ecd None] 10.5.0.47 "GET /v2/4caaba56adf746cbb58a1516f2de913d/os-security-groups HTTP/1.1" status: 500 len: 271 time: 0.0436802

##### neutron-server service is running and listening
$ ip addr | grep gl && status neutron-server && ls /var/log/neutron -alh
    inet 10.207.1.89/16 brd 10.207.255.255 scope global eth0
neutron-server start/running, process 26731
total 225M
drwxr-x--- 2 neutron adm 4.0K Sep 22 17:17 .
drwxrwxr-x 11 root syslog 4.0K Sep 22 15:55 ..
-rw-r--r-- 1 root root 114K Sep 22 17:17 1
-rw-r--r-- 1 neutron neutron 225M Sep 22 18:04 server.log

##### neutron server log, following command: nova secgroup-list
##### this log file grew to 150mb in a matter of minutes following deployment, with the same error looped:
2014-09-22 17:17:52.073 19404 TRACE neutron.service ProgrammingError: (ProgrammingError) (1146, "Table 'neutron.ml2_gre_allocations' doesn't exist") 'SELECT ml2_gre_allocations.gre_id AS ml2_gre_allocations_gre_id, ml2_gre_allocations.allocated AS ml2_gre_allocations_allocated \nFROM ml2_gre_allocations' ()
2014-09-22 17:17:52.073 19404 TRACE neutron.service
2014-09-22 17:17:52.084 19404 CRITICAL neutron [-] ProgrammingError: (ProgrammingError) (1146, "Table 'neutron.ml2_gre_allocations' doesn't exist") 'SELECT ml2_gre_allocations.gre_id AS ml2_gre_allocations_gre_id, ml2_gre_allocations.allocated AS ml2_gre_allocations_allocated \nFROM ml2_gre_allocations' ()
2014-09-22 17:17:52.084 19404 TRACE neutron Traceback (most recent call last):
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/bin/neutron-server", line 10, in <module>
2014-09-22 17:17:52.084 19404 TRACE neutron sys.exit(main())
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/server/__init__.py", line 48, in main
2014-09-22 17:17:52.084 19404 TRACE neutron neutron_api = service.serve_wsgi(service.NeutronApiService)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/service.py", line 105, in serve_wsgi
2014-09-22 17:17:52.084 19404 TRACE neutron LOG.exception(_('Unrecoverable error: please check log '
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/excutils.py", line 82, in __exit__
2014-09-22 17:17:52.084 19404 TRACE neutron six.reraise(self.type_, self.value, self.tb)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/service.py", line 102, in serve_wsgi
2014-09-22 17:17:52.084 19404 TRACE neutron service.start()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/service.py", line 73, in start
2014-09-22 17:17:52.084 19404 TRACE neutron self.wsgi_app = _run_wsgi(self.app_name)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/service.py", line 168, in _run_wsgi
2014-09-22 17:17:52.084 19404 TRACE neutron app = config.load_paste_app(app_name)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/common/config.py", line 186, in load_paste_app
2014-09-22 17:17:52.084 19404 TRACE neutron app = deploy.loadapp("config:%s" % config_path, name=app_name)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2014-09-22 17:17:52.084 19404 TRACE neutron return loadobj(APP, uri, name=name, **kw)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2014-09-22 17:17:52.084 19404 TRACE neutron return context.create()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2014-09-22 17:17:52.084 19404 TRACE neutron return self.object_type.invoke(self)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2014-09-22 17:17:52.084 19404 TRACE neutron **context.local_conf)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2014-09-22 17:17:52.084 19404 TRACE neutron val = callable(*args, **kw)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 28, in urlmap_factory
2014-09-22 17:17:52.084 19404 TRACE neutron app = loader.get_app(app_name, global_conf=global_conf)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2014-09-22 17:17:52.084 19404 TRACE neutron name=name, global_conf=global_conf).create()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2014-09-22 17:17:52.084 19404 TRACE neutron return self.object_type.invoke(self)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2014-09-22 17:17:52.084 19404 TRACE neutron **context.local_conf)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2014-09-22 17:17:52.084 19404 TRACE neutron val = callable(*args, **kw)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/auth.py", line 71, in pipeline_factory
2014-09-22 17:17:52.084 19404 TRACE neutron app = loader.get_app(pipeline[-1])
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2014-09-22 17:17:52.084 19404 TRACE neutron name=name, global_conf=global_conf).create()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2014-09-22 17:17:52.084 19404 TRACE neutron return self.object_type.invoke(self)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2014-09-22 17:17:52.084 19404 TRACE neutron return fix_call(context.object, context.global_conf, **context.local_conf)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
2014-09-22 17:17:52.084 19404 TRACE neutron val = callable(*args, **kw)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/api/v2/router.py", line 71, in factory
2014-09-22 17:17:52.084 19404 TRACE neutron return cls(**local_config)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/api/v2/router.py", line 75, in __init__
2014-09-22 17:17:52.084 19404 TRACE neutron plugin = manager.NeutronManager.get_plugin()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/manager.py", line 219, in get_plugin
2014-09-22 17:17:52.084 19404 TRACE neutron return weakref.proxy(cls.get_instance().plugin)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/manager.py", line 213, in get_instance
2014-09-22 17:17:52.084 19404 TRACE neutron cls._create_instance()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/lockutils.py", line 249, in inner
2014-09-22 17:17:52.084 19404 TRACE neutron return f(*args, **kwargs)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/manager.py", line 199, in _create_instance
2014-09-22 17:17:52.084 19404 TRACE neutron cls._instance = cls()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/manager.py", line 114, in __init__
2014-09-22 17:17:52.084 19404 TRACE neutron plugin_provider)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/manager.py", line 140, in _get_plugin_instance
2014-09-22 17:17:52.084 19404 TRACE neutron return plugin_class()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 125, in __init__
2014-09-22 17:17:52.084 19404 TRACE neutron self.type_manager.initialize()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/managers.py", line 146, in initialize
2014-09-22 17:17:52.084 19404 TRACE neutron driver.obj.initialize()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/type_gre.py", line 71, in initialize
2014-09-22 17:17:52.084 19404 TRACE neutron self._initialize(cfg.CONF.ml2_type_gre.tunnel_id_ranges)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/type_tunnel.py", line 65, in _initialize
2014-09-22 17:17:52.084 19404 TRACE neutron self.sync_allocations()
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/type_gre.py", line 89, in sync_allocations
2014-09-22 17:17:52.084 19404 TRACE neutron allocs = (session.query(GreAllocation).all())
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2300, in all
2014-09-22 17:17:52.084 19404 TRACE neutron return list(self)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2412, in __iter__
2014-09-22 17:17:52.084 19404 TRACE neutron return self._execute_and_instances(context)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2427, in _execute_and_instances
2014-09-22 17:17:52.084 19404 TRACE neutron result = conn.execute(querycontext.statement, self._params)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute
2014-09-22 17:17:52.084 19404 TRACE neutron return meth(self, multiparams, params)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection
2014-09-22 17:17:52.084 19404 TRACE neutron return connection._execute_clauseelement(self, multiparams, params)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
2014-09-22 17:17:52.084 19404 TRACE neutron compiled_sql, distilled_params
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
2014-09-22 17:17:52.084 19404 TRACE neutron context)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1156, in _handle_dbapi_exception
2014-09-22 17:17:52.084 19404 TRACE neutron util.raise_from_cause(newraise, exc_info)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
2014-09-22 17:17:52.084 19404 TRACE neutron reraise(type(exception), exception, tb=exc_tb)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
2014-09-22 17:17:52.084 19404 TRACE neutron context)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute
2014-09-22 17:17:52.084 19404 TRACE neutron cursor.execute(statement, parameters)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
2014-09-22 17:17:52.084 19404 TRACE neutron self.errorhandler(self, exc, value)
2014-09-22 17:17:52.084 19404 TRACE neutron File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
2014-09-22 17:17:52.084 19404 TRACE neutron raise errorclass, errorvalue
2014-09-22 17:17:52.084 19404 TRACE neutron ProgrammingError: (ProgrammingError) (1146, "Table 'neutron.ml2_gre_allocations' doesn't exist") 'SELECT ml2_gre_allocations.gre_id AS ml2_gre_allocations_gre_id, ml2_gre_allocations.allocated AS ml2_gre_allocations_allocated \nFROM ml2_gre_allocations' ()
2014-09-22 17:17:52.084 19404 TRACE neutron

.

##### Checked databases and tables as neutron user:
mysql -u neutron -p -h 10.207.1.82

mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| neutron |
+--------------------+
2 rows in set (0.00 sec)

mysql> use neutron;
Database changed
mysql> show tables;
Empty set (0.00 sec)

mysql>

.

##### juju status
environment: osci-sv7
machines:
  "0":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.71
    instance-id: d48ad51b-68c0-41d0-90f1-85fbbcce004f
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
    state-server-member-status: has-vote
  "1":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.72
    instance-id: bb46b1f2-67a9-4489-91e0-9b1e42a1b3d7
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "2":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.78
    instance-id: 21e1f250-b30c-47f0-a485-c8e0b43703a4
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "3":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.75
    instance-id: 7d78f0c7-160e-4124-a1e8-a5aa549144a4
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "4":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.73
    instance-id: 3bcb032c-c19a-4c65-ae8b-1b7fe08b8dde
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "5":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.83
    instance-id: f7bbfd68-79b2-4b2b-9437-250eeb8a8740
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "6":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.74
    instance-id: 29c43279-f36d-4634-88af-7cdb74bbd694
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "7":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.79
    instance-id: 598ba197-839f-4249-94f2-7e2b27631883
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "8":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.76
    instance-id: 524c8fe8-5823-4670-bb05-8a7b7c1f590a
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "9":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.82
    instance-id: 76c210a4-4ae9-461a-ac9d-11bbbe73471f
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "10":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.89
    instance-id: cd6806a0-0fa7-4896-a77f-f978afc69b97
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "11":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.77
    instance-id: 73c14869-38c4-441d-8b44-9eb36d0545c5
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "12":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.80
    instance-id: ecc2fddb-23d0-414f-9d61-3ede5f98a27f
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "13":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.81
    instance-id: 1a112354-5619-4979-b139-8f74525b8326
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=2 mem=4096M root-disk=10240M
  "14":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.88
    instance-id: 7ee8dfd4-f76b-4edb-a6fe-53a2125a73d0
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=2 mem=4096M root-disk=10240M
  "15":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.84
    instance-id: ec488c3c-a6d0-44f1-803d-337a65e31f31
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=2 mem=4096M root-disk=10240M
  "16":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.85
    instance-id: 9255a21c-e37d-4d46-abef-03777ac516b6
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "17":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.86
    instance-id: 40241606-0926-440e-98b8-ef4f04ce417e
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "18":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.87
    instance-id: b6beccfe-7199-4bee-acf7-f36c5656db9e
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "19":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.90
    instance-id: 26cd1bda-d738-4f4c-b5fb-1b4e9d06c393
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "20":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.91
    instance-id: 5e14240f-679c-4af3-aea0-a84c98c2275d
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "21":
    agent-state: started
    agent-version: 1.20.7
    dns-name: 10.207.1.92
    instance-id: 92648880-b63a-41eb-986a-9d70f07a33a2
    instance-state: ACTIVE
    series: utopic
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
services:
  ceilometer:
    charm: local:utopic/ceilometer-44
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      ceilometer-service:
      - ceilometer-agent
      identity-service:
      - keystone
      shared-db:
      - mongodb
    units:
      ceilometer/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "1"
        open-ports:
        - 8777/tcp
        public-address: 10.207.1.72
  ceilometer-agent:
    charm: local:utopic/ceilometer-agent-20
    exposed: false
    relations:
      ceilometer-service:
      - ceilometer
      nova-ceilometer:
      - nova-compute
    subordinate-to:
    - nova-compute
  ceph:
    charm: local:utopic/ceph-105
    exposed: false
    relations:
      client:
      - cinder
      - glance
      - nova-compute
      mon:
      - ceph
    units:
      ceph/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "2"
        public-address: 10.207.1.78
      ceph/1:
        agent-state: started
        agent-version: 1.20.7
        machine: "3"
        public-address: 10.207.1.75
      ceph/2:
        agent-state: started
        agent-version: 1.20.7
        machine: "4"
        public-address: 10.207.1.73
  cinder:
    charm: local:utopic/cinder-136
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      ceph:
      - ceph
      cinder-volume-service:
      - nova-cloud-controller
      cluster:
      - cinder
      identity-service:
      - keystone
      image-service:
      - glance
      shared-db:
      - mysql
    units:
      cinder/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "5"
        public-address: 10.207.1.83
  glance:
    charm: local:utopic/glance-150
    exposed: false
    relations:
      ceph:
      - ceph
      cluster:
      - glance
      identity-service:
      - keystone
      image-service:
      - cinder
      - nova-cloud-controller
      - nova-compute
      shared-db:
      - mysql
    units:
      glance/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "6"
        open-ports:
        - 9292/tcp
        public-address: 10.207.1.74
  keystone:
    charm: local:utopic/keystone-0
    exposed: false
    relations:
      cluster:
      - keystone
      identity-service:
      - ceilometer
      - cinder
      - glance
      - neutron-api
      - nova-cloud-controller
      - openstack-dashboard
      - swift-proxy
      shared-db:
      - mysql
    units:
      keystone/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "7"
        public-address: 10.207.1.79
  mongodb:
    charm: local:utopic/mongodb-30
    exposed: false
    relations:
      database:
      - ceilometer
      replica-set:
      - mongodb
    units:
      mongodb/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "8"
        open-ports:
        - 27017/tcp
        - 27019/tcp
        - 27021/tcp
        - 28017/tcp
        public-address: 10.207.1.76
  mysql:
    charm: local:utopic/mysql-326
    exposed: false
    relations:
      cluster:
      - mysql
      shared-db:
      - cinder
      - glance
      - keystone
      - neutron-api
      - neutron-gateway
      - nova-cloud-controller
      - nova-compute
    units:
      mysql/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "9"
        public-address: 10.207.1.82
  neutron-api:
    charm: local:utopic/neutron-api-0
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      cluster:
      - neutron-api
      identity-service:
      - keystone
      neutron-api:
      - nova-cloud-controller
      neutron-plugin-api:
      - neutron-openvswitch
      shared-db:
      - mysql
    units:
      neutron-api/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "10"
        open-ports:
        - 9696/tcp
        public-address: 10.207.1.89
  neutron-gateway:
    charm: local:utopic/quantum-gateway-64
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      cluster:
      - neutron-gateway
      quantum-network-service:
      - nova-cloud-controller
      shared-db:
      - mysql
    units:
      neutron-gateway/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "11"
        public-address: 10.207.1.77
  neutron-openvswitch:
    charm: local:utopic/neutron-openvswitch-0
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      neutron-plugin:
      - nova-compute
      neutron-plugin-api:
      - neutron-api
    subordinate-to:
    - nova-compute
  nova-cloud-controller:
    charm: local:utopic/nova-cloud-controller-500
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      cinder-volume-service:
      - cinder
      cloud-compute:
      - nova-compute
      cluster:
      - nova-cloud-controller
      identity-service:
      - keystone
      image-service:
      - glance
      neutron-api:
      - neutron-api
      quantum-network-service:
      - neutron-gateway
      shared-db:
      - mysql
    units:
      nova-cloud-controller/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "12"
        open-ports:
        - 3333/tcp
        - 8773/tcp
        - 8774/tcp
        - 9696/tcp
        public-address: 10.207.1.80
  nova-compute:
    charm: local:utopic/nova-compute-132
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      ceph:
      - ceph
      cloud-compute:
      - nova-cloud-controller
      compute-peer:
      - nova-compute
      image-service:
      - glance
      neutron-plugin:
      - neutron-openvswitch
      nova-ceilometer:
      - ceilometer-agent
      shared-db:
      - mysql
    units:
      nova-compute/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "13"
        public-address: 10.207.1.81
        subordinates:
          ceilometer-agent/2:
            upgrading-from: local:utopic/ceilometer-agent-20
            agent-state: started
            agent-version: 1.20.7
            public-address: 10.207.1.81
          neutron-openvswitch/2:
            upgrading-from: local:utopic/neutron-openvswitch-0
            agent-state: started
            agent-version: 1.20.7
            public-address: 10.207.1.81
      nova-compute/1:
        agent-state: started
        agent-version: 1.20.7
        machine: "14"
        public-address: 10.207.1.88
        subordinates:
          ceilometer-agent/0:
            upgrading-from: local:utopic/ceilometer-agent-20
            agent-state: started
            agent-version: 1.20.7
            public-address: 10.207.1.88
          neutron-openvswitch/0:
            upgrading-from: local:utopic/neutron-openvswitch-0
            agent-state: started
            agent-version: 1.20.7
            public-address: 10.207.1.88
      nova-compute/2:
        agent-state: started
        agent-version: 1.20.7
        machine: "15"
        public-address: 10.207.1.84
        subordinates:
          ceilometer-agent/1:
            upgrading-from: local:utopic/ceilometer-agent-20
            agent-state: started
            agent-version: 1.20.7
            public-address: 10.207.1.84
          neutron-openvswitch/1:
            upgrading-from: local:utopic/neutron-openvswitch-0
            agent-state: started
            agent-version: 1.20.7
            public-address: 10.207.1.84
  openstack-dashboard:
    charm: local:utopic/openstack-dashboard-32
    exposed: false
    relations:
      cluster:
      - openstack-dashboard
      identity-service:
      - keystone
    units:
      openstack-dashboard/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "16"
        open-ports:
        - 80/tcp
        - 443/tcp
        public-address: 10.207.1.85
  rabbitmq-server:
    charm: local:utopic/rabbitmq-server-128
    exposed: false
    relations:
      amqp:
      - ceilometer
      - cinder
      - neutron-api
      - neutron-gateway
      - neutron-openvswitch
      - nova-cloud-controller
      - nova-compute
      cluster:
      - rabbitmq-server
    units:
      rabbitmq-server/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "17"
        open-ports:
        - 5672/tcp
        public-address: 10.207.1.86
  swift-proxy:
    charm: local:utopic/swift-proxy-147
    exposed: false
    relations:
      cluster:
      - swift-proxy
      identity-service:
      - keystone
      swift-storage:
      - swift-storage-z1
      - swift-storage-z2
      - swift-storage-z3
    units:
      swift-proxy/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "18"
        open-ports:
        - 8080/tcp
        public-address: 10.207.1.87
  swift-storage-z1:
    charm: local:utopic/swift-storage-90
    exposed: false
    relations:
      swift-storage:
      - swift-proxy
    units:
      swift-storage-z1/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "19"
        public-address: 10.207.1.90
  swift-storage-z2:
    charm: local:utopic/swift-storage-91
    exposed: false
    relations:
      swift-storage:
      - swift-proxy
    units:
      swift-storage-z2/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "20"
        public-address: 10.207.1.91
  swift-storage-z3:
    charm: local:utopic/swift-storage-92
    exposed: false
    relations:
      swift-storage:
      - swift-proxy
    units:
      swift-storage-z3/0:
        agent-state: started
        agent-version: 1.20.7
        machine: "21"
        public-address: 10.207.1.92

Related branches

lp:~gnuoy/charms/trusty/nova-cloud-controller/next-fix-1372893
James Page: Approve
Diff: 384 lines (+116/-29)
4 files modified
hooks/nova_cc_hooks.py (+31/-6)
hooks/nova_cc_utils.py (+14/-4)
unit_tests/test_nova_cc_hooks.py (+38/-9)
unit_tests/test_nova_cc_utils.py (+33/-10)
lp:~gnuoy/charms/trusty/neutron-api/next-fix-1372893
James Page: Approve
Diff: 356 lines (+135/-25)
7 files modified
hooks/neutron_api_context.py (+3/-3)
hooks/neutron_api_hooks.py (+33/-5)
hooks/neutron_api_utils.py (+16/-0)
templates/icehouse/neutron.conf (+1/-0)
unit_tests/test_neutron_api_context.py (+20/-14)
unit_tests/test_neutron_api_hooks.py (+49/-2)
unit_tests/test_neutron_api_utils.py (+13/-1)
Ryan Beisner (1chb1n)
tags: added: osci
Ryan Beisner (1chb1n)
no longer affects: neutron-api (Ubuntu)
Liam Young (gnuoy)
Changed in nova-cloud-controler (Juju Charms Collection):
importance: Undecided → High
assignee: nobody → Liam Young (gnuoy)
Liam Young (gnuoy)
Changed in neutron-api (Juju Charms Collection):
assignee: nobody → Liam Young (gnuoy)
importance: Undecided → High
Revision history for this message
Ryan Beisner (1chb1n) wrote :

Nova sec operations are now OK after deploying with ncc and neutron-api.
http://paste.ubuntu.com/8459292/

FYI, the deployer bundle used:
http://paste.ubuntu.com/8459289/

lint checks pass for:
https://code.launchpad.net/~gnuoy/charms/trusty/nova-cloud-controller/next-fix-1372893
https://code.launchpad.net/~gnuoy/charms/trusty/neutron-api/next-fix-1372893
revnos 103 and 40, respectively.

unit test pass for:
https://code.launchpad.net/~gnuoy/charms/trusty/nova-cloud-controller/next-fix-1372893
revno 104

unit test fails for:
https://code.launchpad.net/~gnuoy/charms/trusty/neutron-api/next-fix-1372893
revno 41
unit test results: http://paste.ubuntu.com/8459538/

Retesting deployment with 104 + 41.

Ryan Beisner (1chb1n)
tags: added: uosci
removed: osci
Revision history for this message
Ryan Beisner (1chb1n) wrote :

fyi, same bundle deploys ok with 104+41, and nova-secgroup cmds succeed.

## revnos deployed:
ceilometer: 52
ceilometer-agent: 41
ceph: 82
cinder: 45
glance: 62
keystone: 79
mongodb: 52
mysql: 126
neutron-api: 41
neutron-gateway: 65
neutron-openvswitch: 31
nova-cloud-controller: 104
nova-compute: 80
openstack-dashboard: 37
rabbitmq-server: 59
swift-proxy: 60
swift-storage-z1: 43
swift-storage-z2: 43
swift-storage-z3: 43

## nova secgroup cmd output
+ nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
 +-------------+-----------+---------+-----------+--------------+
| tcp | 22 | 22 | 0.0.0.0/0 | |
 +-------------+-----------+---------+-----------+--------------+
+ nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
 +-------------+-----------+---------+-----------+--------------+
| icmp | -1 | -1 | 0.0.0.0/0 | |
 +-------------+-----------+---------+-----------+--------------+
+ nova secgroup-add-rule default udp 53 53 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
 +-------------+-----------+---------+-----------+--------------+
| udp | 53 | 53 | 0.0.0.0/0 | |
 +-------------+-----------+---------+-----------+--------------+
+ nova secgroup-add-rule default tcp 80 80 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
 +-------------+-----------+---------+-----------+--------------+
| tcp | 80 | 80 | 0.0.0.0/0 | |
 +-------------+-----------+---------+-----------+--------------+
+ nova secgroup-add-rule default tcp 443 443 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
 +-------------+-----------+---------+-----------+--------------+
| tcp | 443 | 443 | 0.0.0.0/0 | |
 +-------------+-----------+---------+-----------+--------------+
+ nova secgroup-add-rule default tcp 3128 3128 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
 +-------------+-----------+---------+-----------+--------------+
| tcp | 3128 | 3128 | 0.0.0.0/0 | |
 +-------------+-----------+---------+-----------+--------------+

JuanJo Ciarlante (jjo)
tags: added: canonical-bootstack
Revision history for this message
JuanJo Ciarlante (jjo) wrote :

FYI using /next branch, r103 on trusty/icehouse, single nova--c-c and mysql units, I'm getting:

unit: nova-cloud-controller/0: machine: 0/lxc/8 agent-state: error details: hook failed: "shared-db-relation-changed"

2014-09-30 19:01:36 INFO shared-db-relation-changed sqlalchemy.exc.OperationalError: (OperationalError) (1050, "Table 'networks' already exists") '\nCREATE TABLE networks (\n\ttenant_id VARCHAR(255), \n\tid VARCHAR(36) NOT NULL, \n\tname VARCHAR(255), \n\tstatus VARCHAR(16), \n\tadmin_state_up BOOL, \n\tshared BOOL, \n\tPRIMARY KEY (id), \n\tCHECK (admin_state_up IN (0, 1)), \n\tCHECK (shared IN (0, 1))\n)\n\n' ()

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Using these revs from -next, Trusty and Utopic deploy cleanly:
    neutron-api: 44
    nova-cloud-controller: 103

Revision history for this message
Ryan Beisner (1chb1n) wrote :

And Precise deploy fails with same as jjo reports:
    hook failed: "shared-db-relation-changed"

Revision history for this message
Ryan Beisner (1chb1n) wrote :

FYI/FWIW, there was err in my deploy test logic from gnuoy's MP. I deploy tested it against Juno, which passed.
Once it merged, our CI deploy tests which run -next against P, T & U, started to fail. Naturally, it'd be *much* better to detect that at the MP level. Adjusting logic.

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Today's commits on n-api and ncc triggered deploy tests. P, T + U all deploy cleanly with -next branches:

ceilometer: 52
ceilometer-agent: 41
ceph: 82
cinder: 45
glance: 62
keystone: 79
mongodb: 52
mysql: 126
neutron-api: 45 ****
neutron-gateway: 67
neutron-openvswitch: 32
nova-cloud-controller: 104 ****
nova-compute: 80
openstack-dashboard: 37
rabbitmq-server: 59
swift-proxy: 60
swift-storage-z1: 43
swift-storage-z2: 43
swift-storage-z3: 43

Revision history for this message
Ryan Beisner (1chb1n) wrote :

Also just so you know, unit test on n-api is failing:

Starting tests...
...............................................F..F
======================================================================
FAIL: test_register_configs (unit_tests.test_neutron_api_utils.TestNeutronAPIUtils)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/checkout/neutron-api/unit_tests/test_neutron_api_utils.py", line 134, in test_register_configs
    self.assertItemsEqual(_regconfs.configs, confs)
AssertionError: Element counts were not equal:
First has 1, Second has 0: '/etc/apache2/sites-available/openstack_https_frontend.conf'
First has 0, Second has 1: '/etc/apache2/sites-available/openstack_https_frontend'

======================================================================
FAIL: test_restart_map (unit_tests.test_neutron_api_utils.TestNeutronAPIUtils)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/checkout/neutron-api/unit_tests/test_neutron_api_utils.py", line 115, in test_restart_map
    self.assertItemsEqual(_restart_map, expect)
AssertionError: Element counts were not equal:
First has 1, Second has 0: '/etc/apache2/sites-available/openstack_https_frontend.conf'
First has 0, Second has 1: '/etc/apache2/sites-available/openstack_https_frontend'

Name Stmts Miss Cover Missing
---------------------------------------------------------
hooks/neutron_api_context 61 4 93% 27-29, 67
hooks/neutron_api_hooks 167 5 97% 159-160, 233, 349-350
hooks/neutron_api_utils 88 0 100%
---------------------------------------------------------
TOTAL 316 9 97%
----------------------------------------------------------------------
Ran 51 tests in 2.185s

FAILED (failures=2)
make: *** [test] Error 1

James Page (james-page)
Changed in neutron (Ubuntu):
status: New → Invalid
Changed in neutron-api (Juju Charms Collection):
status: New → Fix Committed
Changed in nova-cloud-controler (Juju Charms Collection):
status: New → Fix Committed
Edward Hope-Morley (hopem)
tags: added: openstack
Edward Hope-Morley (hopem)
Changed in neutron-api (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in nova-cloud-controler (Juju Charms Collection):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.