[intrepid] ifupdown network manager does not blacklist/unmanage mapped devices in managed=false mode

stephan-lap:~# dpkg -l | grep network-manager
ii network-manager 0.7~~svn20081018t105859-0ubuntu1 network management framework daemon
ii network-manager-gnome 0.7~~svn20081020t000444-0ubuntu1 network management framework (GNOME frontend)
ii network-manager-openvpn 0.7~~svn20081015t024626-0ubuntu1 network management framework (OpenVPN plugin)
ii network-manager-pptp 0.7~~svn20081015t024626-0ubuntu1 network management framework (PPTP plugin)
ii network-manager-vpnc 0.7~~svn20081015t024626-0ubuntu1 network management framework (VPNC plugin)

why do you want to use the applet? If you want to use them you should enable managed=true in /etc/NetworkMananger/nm-system-settings.conf

otherwise you opt into using the old ifupdown - thats the feature.

anyway, the other problem that your mapped interfaces do not autoconnect can probably quite easily be solved by propagating the autoconnect state of the "main"-connection to the mapped connections in the ifupdown legacy plugin.

meaning: if you have multiple wifi configurations mapped (which probably should be a common use case of mapping), NM will just autoconnect to whatever of the configured SSIDs it has in range ;).

> why do you want to use the applet? If you want to use them you should enable
> managed=true in /etc/NetworkMananger/nm-system-settings.conf
> otherwise you opt into using the old ifupdown - thats the feature.

I would not have complained if that was really the case. The problem is that NetworkManager
kills my eth0 and then nm-applet does not offer any way to get network connectivity back.
If I restart NetworkManager later, it suddenly works, which is not consistent in its own right.

I believe that NetworkManager should have two absolute requirements:

1. if any device is touched at all by NetworkManager, then nm-applet must be visible
2. a restart of NetworkManager may not change nm-applet's view of the NM state.

BTW: my "wanted" operation mode for NM would be to leave my wired connection's initial state alone,
or at least leave the /etc/network/interfaces decision on the default interface active. I'd like to use
NM for wireless, and I'd be ok to use it for wired (if it'd work).

> anyway, the other problem that your mapped interfaces do not autoconnect can probably quite easily
> be solved by propagating the autoconnect state of the "main"-connection to the mapped connections
> in the ifupdown legacy plugin.

Not sure that would be a solution, right now all I want is NetworkManager to expose its state to nm-applet.

> meaning: if you have multiple wifi configurations mapped (which probably should be a common use
> case of mapping), NM will just autoconnect to whatever of the configured SSIDs it has in range ;).

In my case I am ok with the wireless connectivity (or better I was until one of the later intrepid alphas). I use mappings for the wired connections, as the script determines whether I am physically related in our Portland office, my home office, out Amsterdam office, or whatever.

I agree that its a bug that the applet is visible when you restarted NM but not before.

Could you also attach a screenshot how the applet looks like (when the dropdown is open) after restarting NM? I mean from what I see in the logs you shouldnt be able to do much there even after the restart. Screenshot would clarify that.

it does what I consider to be the right thing:

offer me all mapping ifaces and if I select one, it works fine.

where is my screenshot?

I noticed by scanning the boot logs, that 1. the interfaces eth0 and eth1 were up and configured correctly by the time NetworkManager started. The NetworkManager startup was marked as fail, but the processes were running, which is why I did not see this.

what didnt you see?

After boot there was no nm-applet icon in the GNOME notification area. I expected to see one there. The applet was running, I suppose, but was not showing in the notification area. At this time, all networking interfaces were killed by NetworkManager.

After restart of NetworManager the nm-applet icon did show up (with the red cross indicating no connection), and offered me the mapped interfaces. I'd have expected this to happen after boot, or my interfaces be left alone.

Ideal operation for me would have been:

after boot nm-applet shows up, the mapping-selected wired connection is active, the others are available for selection, wireless is enabled.

If that is not possible, I'd have liked to have NetworkManager handle just wireless, and leave my wired connections completely alone.

If that is also not possible, I guess I'll just deinstall NetworkManager ;-)

does managed=true help you in /etc/NetworkManager/nm-system-settings.conf ?

yes, it seems to. I tried it this morning. Then nm-applet shows, wireless appears and the mapped interfaces are showing in the nm-applet menu.

In managed=false, this seems to be a race of some kind. I used sysv-rc to take NetworkManager out of the boot, and start it for the first time in an xterm, but everything just appeared. Maybe I should create a new runlevel, that is all boot stuff until NetworkManager and then try things in there.

Nov 2 11:26:42 stephan-lap nm-system-settings: SCPlugin-Ifupdown: autoconnect
Nov 2 11:26:42 stephan-lap nm-system-settings: SCPluginIfupdown: management mode: unmanaged
Nov 2 11:26:42 stephan-lap nm-system-settings: SCPlugin-Ifupdown: devices added (udi: /org/freedesktop/Hal/devices/net_00_11_43_75_eb_d0, iface: eth0)
Nov 2 11:26:42 stephan-lap nm-system-settings: SCPlugin-Ifupdown: devices added (udi: /org/freedesktop/Hal/devices/net_00_13_ce_e7_d9_f8, iface: eth1)
Nov 2 11:26:42 stephan-lap nm-system-settings: Ifupdown: get unmanaged devices count: 1
Nov 2 11:26:42 stephan-lap nm-system-settings: SCPlugin-Ifupdown: (152797424) ... get_connections.
Nov 2 11:26:42 stephan-lap nm-system-settings: SCPlugin-Ifupdown: (152797424) connections count: 6
Nov 2 11:26:42 stephan-lap nm-system-settings: Ifupdown: get unmanaged devices count: 1
Nov 2 11:26:42 stephan-lap nm-system-settings: SCPlugin-Ifupdown: end _init.

look at the management mode: unmanaged und then the unmanaged devices count. Doesn't this look odd? Which device is managed, even though I set managed=false?

in hal_device_added_cb the code only checks for exact matches in iface names. I suspect it'd make more sense to check by prefix, i.e. for a device of name "eth0" check for any iface_connection of name "eth0" or starting with "eth0-". As it is, there is no way to unmanage a device that does not have an iface line in /etc/network/interfaces

an alternative would be to allow for a mapping to be treated as an iface for the hal device added callback. Then the system should also know that eth0 is a managed device.

i updated the bug descriuption/title ... please check that that this nails your symptoms.

suggesting as SRU ... in case we have a safe fix for this.

re your latest comments: i think the problem is just that mappings are not considered as "configured" connections so the ifupdown plugin we ship doesnt blacklist eth0.

The current subject line addresses the more important part of the issues, i.e. the inconsistent behavior.
With this fix, managed=false should consistently let things alone, if eth0 and eth1 are listed in iface or mapping.

We did not yet understand or debug the reason, why NetworkManager does not even start properly
during my boot process if I have managed=false. It seems to be half-functional, triggered by a race,
and that's a problem in itself. I guess, I'll (as an experiment) try gdb first and then use tracing syslog
to nail this one. It should be a separate bug, though.

P.S. the more I think about it, the more I believe, that a mapping in /etc/network/interfaces indicates,
that this is a well-known device, so unmanage_well_known should apply. It is a pity network manager
can not be configured to be completely inactive even for not-well-known devices, maybe that should be
an enhancement request...

I think the managed= is very misleading. A better name would be: manage_only_well_known_devices (if this really has to be a boolean) If a SRU is made, perhaps a comment in the defalt nm-system-settings.xml, that explains what this does would be in order.

For a future Ubuntu release I'd propose to change the value to be an enumeration:
manage=none (not possible, at this moment)
manage=known (aka false)
manage=all (aka true)

the managed= thing is ok nobody will understand the well_known_devices better anyway. could you open a new bug for the startup race ... so we can track that down independently from this triaged thing now.

I would be open to allow more values (not just true/false). but i fail to see the user case. if you have one we should open a third bug for that feature request too. Thanks!

to make NM also recognize network devices that are not properly in hal, we should need a new abstraction layer in NM that allows to poll for interface information from libnl i guess. Thats definitly enhanced and would be a forth bug in our series :).

Couldn't be possible to blacklist also the interfaces for which a mapping directive exists (in addition to ones for which a iface directive exists)?

I agree, this is a patch I intend to submit.

What is the right place to report bugs in the ifupdown plugin in general? I assume, this component is debian specific, not ubuntu specific. The more I look at the code, the more I have to suggest some cleanup exercise, like memory management. E.g. I do not think interface_parser.c should allocate memory using g_strudup and free it using free. I have to look in more detail, but there are a few locations in the code, that just smell like memory leaks.

http://bugzilla.gnome.org/enter_bug.cgi?product=NetworkManager

This is the official bugtracker as reported in project homepage. Network manager is a GNOME project, and I think the best way to contact upstream is via that BTS or the mailing lists (from here: http://www.gnome.org/projects/NetworkManager/)

the issue was, that the ifupdown plugin may or may not be part of the GNOME project, I was not sure. upstream of ubuntu is debian then GNOME. I checked the GNOME code now, and you are right, this is the place to report these kinds of "cleanup issues".

Please note, that this is the minimal patch I'd include in an update, not the changes I'd like to include in this file. memory management in this file is still messed up and syslog messages are pretty inconsistent, and ...

I tested, that with this change applied all combinations work (but of course the NetworkManager startup at boot still fails in the managed=false case).

thanks for the patch. here is the right place to submit ifupdown stuff as I mostly maintain this upstream anyway. I currently accumulate the patches we need to fix for intrepid and once we are sure they are of high quality forward them.

Your patch looks good from a first glance. I will take a closer look and run some tests.

If you want to help adding a few more much needed features just speak to me on #nm on freenode.

upload SRU: network-manager_0.7~~svn20081018t105859-0ubuntu1.8.10.1_source.changes to ubuntu/intrepid-proposed

This bug was fixed in the package network-manager - 0.7~~svn20081018t105859-0ubuntu2

---------------
network-manager (0.7~~svn20081018t105859-0ubuntu2) jaunty; urgency=low

  * fix LP: #292054 - Some drivers take too long to associate (Was:
    network-manager 0.7 always asks for WPA passphrase); we workaround
    this driver/wpasupplicant bug by giving association more time
    (e.g. 60sec instead of 25sec)
    - add debian/patches/lp292054_tune_supplicant_timeout_60s.patch
    - update debian/patches/series
  * fix LP: #256905 - dbus policy file (nm-avahi-autoipd.conf) not properly
    deployed in package; install nm-avahi-autoipd.conf
    - update debian/network-manager.install
  * fix LP: #282207 - [Sierra] NM 0.7 does not set APN for AT&T 3G connection;
    apply fix from Jerone Young
    - add debian/patches/lp282207_set_apn_at_syntax.patch
    - update debian/patches/series
  * fix LP: #268667 - not all required ppp options get set on command line
    which makes ppp use bad values from /etc/ppp/options; we backport upstream
    fix
    - add debian/patches/lp268667_more_ppp_default_options.patch
    - update debian/patches/series
  * fix LP: #291564 - ifupdown network manager does not blacklist/unmanage
    mapped devices in managed=false mode; thanks to Stephan Trebels for the
    patch
    - add debian/patches/lp291564_ifupdown_unmanage_mapping_and_iface.patch
    - update debian/patches/series
  * fix LP: #291902 - ifupdown plugin should not export any parsed connection
    configuration when running in managed=false mode; we fix this by exporting
    empty connection list in unmanaged mode
    - add debian/patches/lp291902_IFUPDOWN_dont_export_connection_in_unmanaged_mode.patch
    - update debian/patches/series
  * belt-and-braces fix LP: #290468 VPN fails, "/usr/bin/nm-ppp-starter
    missing"; we remove obsolete conffiles in -pptp .preinst; in case user
    modified them they will be renamed to .dpkg-bak; this patch takes care that
    NM doesn't consider files in /etc/NetworkManager/VPN that don't have a
    .name filename suffix.
    - add debian/patches/lp290468_only_consider_name_suffix_VPN_service_files.patch
    - update debian/patches/series
  * fix LP: #303142 - 3G [Option] some modems take a while time to register on
    network (CREG); we use g_timeout_add instead of _idle_add to give the
    modem some rest during registration phase.
    - add debian/patches/lp303142_more_time_for_automatic_registration.patch
    - update debian/patches/series

 -- Alexander Sack <email address hidden> Fri, 28 Nov 2008 13:47:07 +0100

bug 303159 now deals with further mapping support. subscribed you to it Stephan. Maybe post an example mapping interfaces snippet to that bug description thanks.

Accepted network-manager into intrepid-proposed, please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

On Fri, Nov 28, 2008 at 01:45:51PM -0000, Martin Pitt wrote:
> Accepted network-manager into intrepid-proposed, please test and give
> feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed
> for documentation how to enable and use -proposed. Thank you in advance!
>
> ** Changed in: network-manager (Ubuntu Intrepid)
> Status: In Progress => Fix Committed
> Target: intrepid-updates => None
>
> ** Tags added: verification-needed
>

Stephan, could you verify the packages in proposed?

 - Alexander

For whatever reason I did not find it in intrepid-proposed, so I downloaded the deb and installed it.
It seems to work the same as my hand-patched version, so I'd consider this working. The startup race persists, which is expected. With this, the bug can be closed.

checked was (from /var/lib/dpkg/status):

Package: network-manager
Status: install ok installed
Priority: optional
Section: net
Installed-Size: 1940
Maintainer: Ubuntu Core Dev Team <email address hidden>
Architecture: i386
Version: 0.7~~svn20081018t105859-0ubuntu1.8.10.1
Replaces: network-manager-pptp (<< 0.7~~)
Depends: libc6 (>= 2.4), libdbus-1-3 (>= 1.0.2), libdbus-glib-1-2 (>= 0.74), libglib2.0-0 (>= 2.16.0), libhal1 (>= 0.5.8.1), libnl1, libnm-glib0 (>= 0.7~~svn20080908), libnm-util0 (>= 0.7~~svn20080908), libnspr4-0d (>= 1.8.0.10), libnss3-1d (>= 3.12.0~1.9b1), libpolkit-dbus2 (>= 0.7), libpolkit2 (>= 0.7), libuuid1 (>= 1.05), iproute, iputils-arping, lsb-base (>= 2.0-6), wpasupplicant (>= 0.6.1~), dbus (>= 0.60), hal (>= 0.5.7.1), update-notifier-common
Recommends: network-manager-gnome | network-manager-kde
Conflicts: network-manager-pptp (<< 0.7~~)
Conffiles:
 /etc/dbus-1/system.d/NetworkManager.conf 2f9a653fcad0a7e091dac46b4288065d
 /etc/dbus-1/system.d/nm-dhcp-client.conf 5fe2b938444987d8644d782b7cdee710
 /etc/dbus-1/system.d/nm-dispatcher.conf e1d16d69e5e8065ca37ac8245b562a42
 /etc/dbus-1/system.d/nm-system-settings.conf 21b3892f02ec1acceb48b25baa988ff0
 /etc/dbus-1/system.d/nm-avahi-autoipd.conf 1789b8276205e55c633afe1f3eb9b433
 /etc/NetworkManager/dispatcher.d/01ifupdown 2bc1ae04a18972cd4b9acfaf6aa2104f
 /etc/NetworkManager/nm-system-settings.conf bc4a4aa7e1b84e080946bf081dd87eb0
 /etc/init.d/NetworkManager 3b22837a479a87d3cbb4a75e5cd9e10e
Description: network management framework daemon
 NetworkManager attempts to keep an active network connection available at all
 times. It is intended only for the desktop use-case, and is not intended for
 usage on servers. The point of NetworkManager is to make networking
 configuration and setup as painless and automatic as possible. If using DHCP,
 NetworkManager is _intended_ to replace default routes, obtain IP addresses
 from a DHCP server, and change nameservers whenever it sees fit.
 .
 This package provides the userspace daemons.
 .
  Homepage: http://www.gnome.org/projects/NetworkManager/
Original-Maintainer: Riccardo Setti <email address hidden>

On Sat, Nov 29, 2008 at 08:14:32PM -0000, Stephan Trebels wrote:
> For whatever reason I did not find it in intrepid-proposed, so I downloaded the deb and installed it.

Hmm ... maybe due to a non-primary mirror? Which mirror are you using?

 - Alexander

set to verification-done based on comment https://bugs.edge.launchpad.net/ubuntu/intrepid/+source/network-manager/+bug/291564/comments/42 and https://bugs.edge.launchpad.net/ubuntu/intrepid/+source/network-manager/+bug/291564/comments/43

This bug was fixed in the package network-manager - 0.7~~svn20081018t105859-0ubuntu1.8.10.1

---------------
network-manager (0.7~~svn20081018t105859-0ubuntu1.8.10.1) intrepid-proposed; urgency=low

  * fix LP: #292054 - Some drivers take too long to associate (Was:
    network-manager 0.7 always asks for WPA passphrase); we workaround
    this driver/wpasupplicant bug by giving association more time
    (e.g. 60sec instead of 25sec)
    - add debian/patches/lp292054_tune_supplicant_timeout_60s.patch
    - update debian/patches/series
  * fix LP: #256905 - dbus policy file (nm-avahi-autoipd.conf) not properly
    deployed in package; install nm-avahi-autoipd.conf
    - update debian/network-manager.install
  * fix LP: #282207 - [Sierra] NM 0.7 does not set APN for AT&T 3G connection;
    apply fix from Jerone Young
    - add debian/patches/lp282207_set_apn_at_syntax.patch
    - update debian/patches/series
  * fix LP: #268667 - not all required ppp options get set on command line
    which makes ppp use bad values from /etc/ppp/options; we backport upstream
    fix
    - add debian/patches/lp268667_more_ppp_default_options.patch
    - update debian/patches/series
  * fix LP: #291564 - ifupdown network manager does not blacklist/unmanage
    mapped devices in managed=false mode; thanks to Stephan Trebels for the
    patch
    - add debian/patches/lp291564_ifupdown_unmanage_mapping_and_iface.patch
    - update debian/patches/series
  * fix LP: #291902 - ifupdown plugin should not export any parsed connection
    configuration when running in managed=false mode; we fix this by exporting
    empty connection list in unmanaged mode
    - add debian/patches/lp291902_IFUPDOWN_dont_export_connection_in_unmanaged_mode.patch
    - update debian/patches/series
  * belt-and-braces fix LP: #290468 VPN fails, "/usr/bin/nm-ppp-starter
    missing"; we remove obsolete conffiles in -pptp .preinst; in case user
    modified them they will be renamed to .dpkg-bak; this patch takes care that
    NM doesn't consider files in /etc/NetworkManager/VPN that don't have a
    .name filename suffix.
    - add debian/patches/lp290468_only_consider_name_suffix_VPN_service_files.patch
    - update debian/patches/series
  * fix LP: #303142 - 3G [Option] some modems take a while time to register on
    network (CREG); we use g_timeout_add instead of _idle_add to give the
    modem some rest during registration phase.
    - add debian/patches/lp303142_more_time_for_automatic_registration.patch
    - update debian/patches/series

 -- Alexander Sack <email address hidden> Fri, 28 Nov 2008 13:48:34 +0100

this bug is still there in jaunty