quantum server shouldn't return tracebacks to clients

Bug #980033 reported by dan wendlandt
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Guilherme Salgado

Bug Description

Currently, the Quantum server returns exceptions tracebacks to the client. This exposes information about a software issue, as well as what plugin is in use and potentially how it is configured to tenant. This is really none of the tenant's business.

We should log the traceback in the server logs, but then return a generic error to the tenant.

Here's an example of what I got back as a client:

(http://paste.ubuntu.com/960965/ has a pretty-printed version of the traceback below)
danwent@ubuntu:~/quantum$ quantum create_net default net1
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/python_quantumclient-2012.2-py2.7.egg/quantum/client/cli_lib.py", line 275, in create_net
    res = client.create_network(data)
  File "/usr/local/lib/python2.7/dist-packages/python_quantumclient-2012.2-py2.7.egg/quantum/client/__init__.py", line 145, in with_params
    ret = self.function(instance, *args)
  File "/usr/local/lib/python2.7/dist-packages/python_quantumclient-2012.2-py2.7.egg/quantum/client/__init__.py", line 423, in create_network
    return self.post(self.networks_path, body=body)
  File "/usr/local/lib/python2.7/dist-packages/python_quantumclient-2012.2-py2.7.egg/quantum/client/__init__.py", line 384, in post
    headers=headers, params=params)
  File "/usr/local/lib/python2.7/dist-packages/python_quantumclient-2012.2-py2.7.egg/quantum/client/__init__.py", line 305, in do_request
    self._handle_fault_response(status_code, data)
  File "/usr/local/lib/python2.7/dist-packages/python_quantumclient-2012.2-py2.7.egg/quantum/client/__init__.py", line 228, in _handle_fault_response
    EXCEPTION_HANDLERS[self.version](status_code, des_error_body)
  File "/usr/local/lib/python2.7/dist-packages/python_quantumclient-2012.2-py2.7.egg/quantum/client/__init__.py", line 119, in exception_handler_v11
    raise exceptions.QuantumClientException(message=msg)
uantumClientException: 500-{'message': 'Traceback (most recent call last):\n File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 336, in handle_one_response\n result = self.application(self.environ, start_response)\n File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in __call__\n return app(environ, start_response)\n File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n return resp(environ, start_response)\n File "/usr/lib/pymodules/python2.7/routes/middleware.py", line 131, in __call__\n response = self.app(environ, start_response)\n File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n return resp(environ, start_response)\n File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n return resp(environ, start_response)\n File "/usr/lib/pymodules/python2.7/routes/middleware.py", line 131, in __call__\n response = self.app(environ, start_response)\n File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__\n return resp(environ, start_response)\n File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__\n resp = self.call_func(req, *args, **self.kwargs)\n File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func\n return self.func(req, *args, **kwargs)\n File "/home/danwent/quantum/quantum/wsgi.py", line 748, in __call__\n action_result = self.dispatch(request, action, args)\n File "/home/danwent/quantum/quantum/wsgi.py", line 780, in dispatch\n return controller_method(request=request, **action_args)\n File "/home/danwent/quantum/quantum/api/api_common.py", line 108, in the_func\n return func(*args, **kwargs)\n File "/home/danwent/quantum/quantum/api/networks.py", line 134, in create\n **body)\n File "/home/danwent/quantum/quantum/plugins/sample/SamplePlugin.py", line 211, in create_network\n new_net = db.network_create(tenant_id, net_name)\n File "/home/danwent/quantum/quantum/db/api.py", line 119, in network_create\n session.flush()\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1559, in flush\n self._flush(objects)\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1630, in _flush\n flush_context.execute()\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 331, in execute\n rec.execute(self)\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 475, in execute\n uow\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/mapper.py", line 2264, in _save_obj\n execute(statement, multiparams)\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1405, in execute\n params)\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1538, in _execute_clauseelement\n compiled_sql, distilled_params\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1646, in _execute_context\n context)\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1639, in _execute_context\n context)\n File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 330, in do_execute\n cursor.execute(statement, parameters)\nOperationalError: (OperationalError) no such table: networks u\'INSERT INTO networks (uuid, tenant_id, name, op_status) VALUES (?, ?, ?, ?)\' (\'ee799487-7335-4712-b52b-10f7dd748d7a\', u\'default\', u\'net1\', \'UNKNOWN\')\n'}

dan wendlandt (danwent)
tags: added: low-hanging-fruit
description: updated
Revision history for this message
Guilherme Salgado (salgado) wrote :

I wanted to give this one a go (as it's tagged as low-hanging-fruit and so I thought it'd be a good choice for me to get started) and after playing with quantum a bit I think a fix for this would have to go in Resource.__call__(). I'm thinking that the most appropriate thing to do would be to have a separate except block there catching everything that's not a webob.exc.HTTPException, and treat those as internal errors, logging them but not including the original exception in the Fault() object returned. Something like:

        try:
            action_result = self.dispatch(request, action, args)
        except webob.exc.HTTPException as ex:
            LOG.info(_("HTTP exception thrown: %s"), unicode(ex))
            action_result = Fault(ex,
                                  self._xmlns,
                                  self._fault_body_function)
        except Exception as exc:
            LOG.exception("Internal error: %s", unicode(exc))
            action_result = Fault(exception.Error("Internal error"),
                                  self._xmlns,
                                  self._fault_body_function)

That way the error messages of "expected" exceptions (i.e. those wrapped in an HTTPException by @common.APIFaultWrapper) will be sent to the client but everything else will be sent to clients with a generic "internal error" message.

Does this sound reasonable?

Revision history for this message
dan wendlandt (danwent) wrote :

Hi Guilherme,

Thanks for taking a look at this. At a glance you, your description sounds correct. I'd write it up, test it, and then submit it for code review. Thanks!

Changed in quantum:
assignee: nobody → Guilherme Salgado (salgado)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to quantum (master)

Fix proposed to branch: master
Review: https://review.openstack.org/7019

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to quantum (master)

Reviewed: https://review.openstack.org/7019
Committed: http://github.com/openstack/quantum/commit/d1c563c57df14b4d968bd576bc366d7db86b3e79
Submitter: Jenkins
Branch: master

commit d1c563c57df14b4d968bd576bc366d7db86b3e79
Author: Guilherme Salgado <email address hidden>
Date: Wed May 2 08:36:56 2012 -0300

    Change Resource.__call__() to not leak internal errors.

    That method now catches any non-expected errors raised by the controller's
    action method and turn them into an HTTPServerError. Fixes bug 980033.

    Change-Id: I7f71c029fae0e27a799f11de0802bde1003683e2

Changed in quantum:
status: In Progress → Fix Committed
dan wendlandt (danwent)
Changed in quantum:
milestone: none → folsom-1
importance: Undecided → Medium
Thierry Carrez (ttx)
Changed in quantum:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in quantum:
milestone: folsom-1 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.