linux_net fails to start dnsmasq as root

Bug #1073253 reported by clayg
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Andrew Laski

Bug Description

The command to start dnsmasq has some bash environment prefixes (FLAGFILE=, NETWORK_ID=) but the call to execute isn't marked with shell=True. So when "Make nova-rootwrap optional" got merged (Change-Id: I2cf5c743c6f4d6f235a54ca70c11d8191deeed6a) instead of always prefixing the commandline with "sudo" - if you're running as root (as I do on some automated dev/test/build vms) - the commandline now started with "FLAGFILE" and subprocess throws a OSError/ENOENT because there's no executable with that name ;)

2012-10-30 12:17:57 DEBUG nova.utils [req-7dc40cb3-8410-4868-927c-1b5d50ede910 None None] Running cmd (subprocess): FLAGFILE=/etc/nova/nova.conf NETWORK_ID=1
 dnsmasq --strict-order --bind-interfaces --conf-file= --domain=novalocal --pid-file=/etc/nova/state/networks/nova-virbr0.pid --listen-address=10.0.0.1 --exc
ept-interface=lo --dhcp-range=set:'private',10.0.0.2,static,120s --dhcp-lease-max=256 --dhcp-hostsfile=/etc/nova/state/networks/nova-virbr0.conf --dhcp-scrip
t=/opt/nova/bin/nova-dhcpbridge --leasefile-ro execute /opt/nova/nova/utils.py:171
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 336, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
  File "/opt/nova/nova/service.py", line 123, in run_server
    server.start()
  File "/opt/nova/nova/service.py", line 389, in start
    self.manager.init_host()
  File "/opt/nova/nova/network/manager.py", line 1943, in init_host
    super(FlatDHCPManager, self).init_host()
  File "/opt/nova/nova/network/manager.py", line 889, in init_host
    self._setup_network_on_host(ctxt, network)
  File "/opt/nova/nova/network/manager.py", line 1954, in _setup_network_on_host
    self.driver.update_dhcp(context, dev, network)
  File "/opt/nova/nova/network/linux_net.py", line 772, in update_dhcp
    restart_dhcp(context, dev, network_ref)
  File "/opt/nova/nova/openstack/common/lockutils.py", line 229, in inner
    retval = f(*args, **kwargs)
  File "/opt/nova/nova/network/linux_net.py", line 858, in restart_dhcp
    _execute(*cmd, run_as_root=True)
  File "/opt/nova/nova/network/linux_net.py", line 950, in _execute
    return utils.execute(*cmd, **kwargs)
  File "/opt/nova/nova/utils.py", line 179, in execute
    shell=shell)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/green/subprocess.py", line 25, in __init__
    subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

It was a little hard to track down, because obviously running on the commandline works. It only happens on the boxes where I'm running services as root. And subprocess wasn't being really specific about what exactly it couldn't find :P

I looked for precedent adding shell=True to the call, but it seems the nova codebase prefers to explicitly set the environ with the env command.

Tags: dnsmasq linux net
Revision history for this message
clayg (clay-gerrard) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/15048

Changed in nova:
assignee: nobody → clayg (clay-gerrard)
status: New → In Progress
Changed in nova:
assignee: clayg (clay-gerrard) → Andrew Laski (alaski)
Revision history for this message
Thierry Carrez (ttx) wrote :

FWIW running without rootwrap (direct sudo or as root) is deprecated in Folsom. It should become unsupported in Grizzly the moment we start allowing rootwrap to directly run python code snippets (instead of only allowing it to shell out).

Changed in nova:
importance: Undecided → Low
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/15048
Committed: http://github.com/openstack/nova/commit/1d7d733d017c0a8fe2f171d9822d752ff72d9fc6
Submitter: Jenkins
Branch: master

commit 1d7d733d017c0a8fe2f171d9822d752ff72d9fc6
Author: Andrew Laski <email address hidden>
Date: Tue Oct 30 16:54:41 2012 -0400

    Use env to set environ when starting dnsmasq

    Explictly use env to setup needed environment variables for dnsmasq when
    starting to avoid problems with subprocess mis-interpreting the first
    command line argument as the executable name.

    Also update DnsmasqFilter to accept any command that starts with env
    followed by a set of args that include an equals sign, as long as the
    next arg is the dnsmasq command.

    fixes bug #1073253

    Change-Id: I8ac08ba2d2309934a67ed2cb28049ed5d3277d63

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.