(oneiric) wired connection unnaturally slow

Bug #856333 reported by Jane Silber
34
This bug affects 5 people
Affects Status Importance Assigned to Milestone
NetworkManager
Fix Released
High
libnl3 (Ubuntu)
Invalid
High
Unassigned
network-manager (Ubuntu)
Fix Released
High
Mathieu Trudel-Lapierre

Bug Description

After an update today (22 Sept) my wired connection is so slow as to be unworkable. I have a Thinkpad X201s, so this could be a dupe of bug #836250, but that bug was reported a month ago and my machine has worked fine much more recently than that. I feel this is something that was introduced in an update sometime in the last 2-3 days.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: network-manager 0.9.0-0ubuntu3
ProcVersionSignature: Ubuntu 3.0.0-11.18-generic 3.0.4
Uname: Linux 3.0.0-11-generic x86_64
ApportVersion: 1.23-0ubuntu1
Architecture: amd64
CRDA: Error: [Errno 2] No such file or directory
Date: Thu Sep 22 12:46:39 2011
InstallationMedia: Ubuntu 10.10 "Maverick Meerkat" - Release amd64 (20101007)
IpRoute:
 default via 10.45.43.1 dev eth0 proto static
 10.45.43.0/24 dev eth0 proto kernel scope link src 10.45.43.13
 10.45.43.0/24 dev wlan0 proto kernel scope link src 10.45.43.94
 169.254.0.0/16 dev eth0 scope link metric 1000
NetworkManager.state:
 [main]
 NetworkingEnabled=true
 WirelessEnabled=true
 WWANEnabled=true
ProcEnviron:
 PATH=(custom, no user)
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: network-manager
UpgradeStatus: Upgraded to oneiric on 2011-09-07 (14 days ago)

Revision history for this message
Jane Silber (silbs) wrote :
Pete Graner (pgraner)
Changed in network-manager (Ubuntu):
importance: Undecided → Medium
Revision history for this message
Pete Graner (pgraner) wrote :

@Jane,

I looks like you have both your wireless & wired interfaces running at the same time.

Try going to the network-manger applet and:

1. Left click on the icon
2. Click "Disconnect" under "Wireless Networks"
3. Click "Auto eth0" under "Wired Networks"

That will disconnect your wifi and force your wired to reconnect, ensure all routing etc is flushed.

Try again and see if you performance gets better.

Revision history for this message
Jane Silber (silbs) wrote :

ifconfig -a output attached (taken with wired network unplugged)

Revision history for this message
Jane Silber (silbs) wrote :

Another ifconfig -a, this time with wired and wireless.

I have tested with (a) wired alone, (b) wired & wireless, and (c) wireless alone. This morning it seemed as though any configuration with wired was a problem (i.e., a and b). But having retested, now I only see the problem in the (b) scenario. Not sure what's happening, and I haven't done any updates since I first experienced the problem.

Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

I think I know what this is, but it's a little tricky to test/prove. With both wired and wireless connected, could you do the following:

1) Capture the output of two separate commands (in separate terminals) running roughly at the same time:

sudo tcpdump -i eth0 icmp or arp
sudo tcpdump -i wlan0 icmp or arp

2) As these are running, start in a third terminal: "ping 8.8.8.8"

They don't need to be started exactly at the same time; and we only need the terminal output because they can include sensitive data if run while other things are going on using the network. This should only need to run for a short while: in ping, you'll likely see no responses for a while and short bursts of actually getting responses, or the other way around. If you always get responses or never get any, then it's something different (but the output of tcpdump will still help).

And the highly technical explanation for all this:
I think traffic is being routed incorrectly; as if for a while eth0 sends the traffic, and it comes back through wlan0 (which would cause it to be "lost"). At some point once other network devices have their arp cache expire; depending on how fast arp replies will get to the server; eth0 will start to both send and receive traffic and connections will complete. Then another arp request will come by and break it again.

However, I don't see how this would be tied to an update unless there was a kernel change very recently. Wired is not affected by many things except NM to establish connections and the driver/kernel to push data. The latest upload of NM doesn't appear to be installed on your system yet (I uploaded it just a few hours ago). The last kernel upload was 9 days ago. I'm not sure what else may have affected the device.

Changed in network-manager (Ubuntu):
status: New → Incomplete
Revision history for this message
Jane Silber (silbs) wrote :

Output of the above cmds sent privately to Mathieu because of confidentiality concerns.

Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

Turns out this is provably an issue with libnl. Routes are being given priorities by NetworkManager but the function that does this isn't seeing all the routes it should when it tries to apply the priorities.

You can "easily" fix this by removing the routes and re-adding them with a "metric N" value attached. NM usually uses 1 for ethernet and 2 for wireless (and 3 for mobile, etc.)

E.g.
ip route del <network> dev eth0 scope link proto kernel
ip route add <network> dev eth0 scope link proto kernel metric 1
ip route del <network> dev wlan0 scope link proto kernel
ip route add <network> dev wlan0 scope link proto kernel metric 2

Except these depend on the actual network being set.

I've opened a bug upstream about the issue (linked), and I'm continuing to work to fix this.

Changed in network-manager (Ubuntu):
status: Incomplete → In Progress
assignee: nobody → Mathieu Trudel-Lapierre (mathieu-tl)
Changed in network-manager:
importance: Unknown → High
status: Unknown → New
tags: added: rls-mgr-o-tracking
Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

Okay, I believe I've finally made good progress.. Even if it's quite sad it happened so late ;)

To get more information, I decided to give the Fedora 16 Live CD a try, since it was likely broken there too in which case I could provide useful information to upstream to help with the debugging -- or not broken, in which case we'd have another path to investigate.

Turns out it works properly there, or at least I can definitely see the "metric 2" tag I'm expecting on wifi routes (since I don't currently have everything I need to properly test this at this moment). I checked and it had libnl1.1 installed. I didn't think of checking whether libnl3 was also installed or what version of NetworkManager was used (though I'm highly confident it's the same as what we have in Oneiric, for having compared packaging before).

So; I was able to locally rebuild NetworkManager against libnl1 (libnl-dev) instead of libnl3 (libnl3-dev), and I can now see the metric on wifi routes on my system in Oneiric. Once I'm back home I'll properly test this with wired connected (to ensure proper behavior and that there are no other side-effects), then most probably upload NM with the updated Build-Depends early tomorrow morning, EST.

Regardless, I think it's extremely good news (despite the work that was done for the libnl3 migration). Reassigning to the 'libnl3' package.

Changed in libnl3 (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Changed in network-manager (Ubuntu):
importance: Medium → High
Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

So it was a painful issue, caused by the rtnl_route_alloc_cache() call in... NetworkManager. Passing NL_AUTO_PROVIDE as a flag for the cache creation causes issues, switching that to no flags (0) fixes the issue.

Changed in libnl3 (Ubuntu):
status: Confirmed → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package network-manager - 0.9.1.90-0ubuntu2

---------------
network-manager (0.9.1.90-0ubuntu2) oneiric; urgency=low

  * debian/patches/libnl3-cache-args.patch: unbreak setting route priorities,
    which will also unbreak cases where connections fails when wired and wifi
    are connected to the same network. (LP: #856333)
  * debian/patches/dhcpv6-duid-support.patch: support handling DUIDs for DHCPv6
    requests, as defined by RFC 3315, section 9. (LP: #849994)
 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 28 Sep 2011 11:59:27 -0400

Changed in network-manager (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Till Kamppeter (till-kamppeter) wrote :

After the fix coming with the updates and restarting NetworkManager (via "sudo restart network-manager") I got bug 862052 while the system staying idle during the night. Perhaps caused by the fix for this bug?

Revision history for this message
Matteo (gmatteo-abis+sites) wrote :

My wired connections is still very slow, and freezes every few seconds, which is very annoying.

Changed in network-manager:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.