openvswitch-agent down when ovs-vsctl list-port fails

Bug #1037902 reported by Nachi Ueno
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Gary Kotton

Bug Description

Exception handling for command execution failure looks missing especially for rpc support.
Agent looks easily down.

2012-08-17 04:59:43 DEBUG [quantum.agent.linux.utils]
Command: ['sudo', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
Exit code: 142
Stdout: ''
Stderr: ''
Traceback (most recent call last):
  File "/opt/stack/quantum/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 864, in <module>
    main()
  File "/opt/stack/quantum/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 836, in main
    enable_tunneling = cfg.CONF.OVS.enable_tunneling
  File "/opt/stack/quantum/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 387, in daemon_loop
    self.rpc_loop()
  File "/opt/stack/quantum/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 368, in rpc_loop
    port_info = self.update_ports(ports)
  File "/opt/stack/quantum/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py", line 296, in update_ports
    ports = self.int_br.get_vif_port_set()
  File "/opt/stack/quantum/quantum/agent/linux/ovs_lib.py", line 211, in get_vif_port_set
    port_names = self.get_port_name_list()
  File "/opt/stack/quantum/quantum/agent/linux/ovs_lib.py", line 175, in get_port_name_list
    res = self.run_vsctl(["list-ports", self.br_name])
  File "/opt/stack/quantum/quantum/agent/linux/ovs_lib.py", line 53, in run_vsctl
    return utils.execute(full_args, root_helper=self.root_helper)
  File "/opt/stack/quantum/quantum/agent/linux/utils.py", line 55, in execute
    raise RuntimeError(m)
RuntimeError:
Command: ['sudo', 'ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']
Exit code: 142
Stdout: ''
Stderr: ''
DEBUG:amqplib:Closed channel #1
2012-08-17 04:59:43 DEBUG [amqplib] Closed channel #1

Yaguang Tang (heut2008)
Changed in quantum:
status: New → Confirmed
Revision history for this message
Gary Kotton (garyk) wrote :

Can you please provide details on how to reproduce the problem.
Thanks
Gary

Revision history for this message
Gary Kotton (garyk) wrote :

Hi,
I have seen that http://openvswitch.org/pipermail/discuss/2010-November/004516.html indicates that the ovs server is down. I was able to reproduce.
Thanks
Gary

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to quantum (master)

Fix proposed to branch: master
Review: https://review.openstack.org/11622

Changed in quantum:
assignee: nobody → Gary Kotton (garyk)
status: Confirmed → In Progress
Revision history for this message
Gary Kotton (garyk) wrote :
Changed in quantum:
milestone: none → folsom-rc1
importance: Undecided → Medium
Revision history for this message
Nachi Ueno (nati-ueno) wrote :

Hi Gary

Thank you for your fixing this.
This exception also happens if the bridge is not created by another issue.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to quantum (master)

Reviewed: https://review.openstack.org/11622
Committed: http://github.com/openstack/quantum/commit/1798c82dfa7de36de2502bb44e8b646d93574290
Submitter: Jenkins
Branch: master

commit 1798c82dfa7de36de2502bb44e8b646d93574290
Author: Gary Kotton <email address hidden>
Date: Sun Aug 19 04:11:06 2012 -0400

    Treat exceptions when invoking ovs-vsctl

    Fixes bug 1037902

    Change-Id: I3f5bbac2f0c390fd7eaae4eb6338e5ac600c9aba

Changed in quantum:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in quantum:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in quantum:
milestone: folsom-rc1 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.