[mako/flipped] 3g Data Connections Fail

Here's a log of all the Network Manager messages grep'd from /var/log/syslog on a flipped system, where 3g fails.

Note that early in the log, NM ignores all of the rmnet* devices, and then later you see it try to work with the rmnet_usb* devices.

Here's the NM log output for today's non-flipped image.

So far I've tried to workaround which involve modifying /lib/udev/rules.d/65-android.rules.

The first modification was to add:

ACTION=="add", KERNEL=="rmnet_mux_ctrl", OWNER="radio", GROUP="radio", MODE="0640"

This has no effect on the presence of the rmnet_usb* devices.

The second attempted workaround was to add the following additional rules to 65-android.rules:

ACTION=="add", KERNEL=="smd7", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl0", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl1", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl2", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl3", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl4", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl5", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl6", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smdcntl7", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="smuxctl32", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="rmnet_mux_ctrl", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="hsicctl0", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="hsicctl1", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="hsicctl2", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="hsicctl3", OWNER="radio", GROUP="radio", MODE="0640"
ACTION=="add", KERNEL=="mdm", OWNER="radio", GROUP="radio", MODE="0640"

These also have no effect on the additional rmnet_usb* devices.

Note today's testing was with the latest daily ( 20130621.1 ).

Note, I found no differences in /dev or /sys/devices/virtual/net.

In /dev, I see rmnet_mux_ctrl both inside and outside the container in both models.

In /sys/devices/virtual/net, I see the same device directories ( rmnet0...rmnet7 ) both inside and outside the container in both models.

I guess the next step would be to determine why Network Manager sees a difference when running as flipped. It could be that the attributes exposed in the sysfs devices are different when flipped vs. non-flipped.

there were issues with the creation of subdirs under /dev/socket that should be fixed with the 20130622 image.

additionally lxc-android-config now ships a complete set of device specific udev rules derived from the android uevent.rc (as well as chown/chmod lines from init.rc) the radio related ones are identical to thr list from comment #3 though.

have a look if this image improves things (once it built)

The changes definitely help, and I'm now able to activate a connection on mako with the today's (20130624) flipped image plus an updated lxc-android-config package (0.35).

That said, there are still issues with NM trying to use the rmnet devices as wired Ethernet devices still. Again, as above, running nmcli dev on the flipped images shows the following devices:

root@ubuntu-phablet:/# nmcli dev
DEVICE TYPE STATE
rmnet_usb2 802-3-ethernet unavailable
rmnet_usb3 802-3-ethernet unavailable
rmnet_usb0 802-3-ethernet unavailable
rmnet_usb1 802-3-ethernet unavailable
/ril_0 gsm disconnected
wlan0 802-11-wireless disconnected

So, I still think we need to understand why this is so, and fix...

Also one other thing I noticed is that our current NM/oFono integration leads to a growing list of oFono data contexts ( just run the ofono-scripts list-contexts ). It seems that every time NM creates a new context, oFono persists it in it's gprs settings file.

it could either be that there are udev rules that recognize the device on the USB bus but have no special rule to handle rmnet, or that the device specific rules file is run to early

a) try to move /lib/udev/rules.d/75-net-description.rules out of the way for a test ...

b) if a) improves anything, try putting it back and move /lib/udev/rules.d/70-mako.rules to run after 75-net-description.rules (i.e. move it to 80-mako.rules)

also check what is in /etc/udev/rules.d/70-persistent-net.rules ...

http://paste.ubuntu.com/5798095/ has /var/log/udev from a mako install and there are obviously rmnet devices attached to the android usb gadget driver...

@Oliver

Please see my previous comment #7 where I said that I was able to make the 3g connection work.

That said, NM is still trying to configured these devices *before* the 3g connection is activated as they appear to be wired Ethernet devices.

We need to understand *why* NM is now all of a sudden trying to use these devices and figure out how to prevent. I've asked Mathieu to get involved and help us understand what's happening, and figure out a way to prevent it.

This is still broken with today's image (20130626).

After running 'nmcli con up id "3g"', the phone is briefly brought online ( note, I have no active Wi-Fi configuration ), however it seems NM then tries to separately active rmnet_usb0, and screws things up. Here's the actual sequence of commands:

root@ubuntu-phablet:/# nmcli con up id "3g"
root@ubuntu-phablet:/# ping yahoo.com
PING yahoo.com (206.190.36.45) 56(84) bytes of data.
64 bytes from ir1.fp.vip.gq1.yahoo.com (206.190.36.45): icmp_seq=1 ttl=43 time=218 ms
64 bytes from ir1.fp.vip.gq1.yahoo.com (206.190.36.45): icmp_seq=2 ttl=43 time=175 ms
64 bytes from ir1.fp.vip.gq1.yahoo.com (206.190.36.45): icmp_seq=3 ttl=43 time=220 ms
^C
--- yahoo.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 175.278/204.913/220.753/20.975 ms
root@ubuntu-phablet:/# nmcli dev
DEVICE TYPE STATE
rmnet_usb2 802-3-ethernet unavailable
rmnet_usb3 802-3-ethernet unavailable
rmnet_usb1 802-3-ethernet unavailable
rmnet_usb0 802-3-ethernet connecting (getting IP configuration)
/ril_0 gsm connected
wlan0 802-11-wireless disconnected
root@ubuntu-phablet:/# ping yahoo.com
ping: unknown host yahoo.com

@Oliver

I added manual iface statements for all of the devices to /etc/network/interfaces:

iface rmnet_usb0 inet manual
iface rmnet_usb1 inet manual
iface rmnet_usb2 inet manual
iface rmnet_usb3 inet manual

...and this fixes the problem, however I'm not sure this is the correct solution as this is a specific workaround for mako.

no, it definitely isn't a solution, but a good temporary workaround until NM learns to ignore these devices, i will look how to inject that workaround into lxc-android-config for now

This bug was fixed in the package lxc-android-config - 0.36

---------------
lxc-android-config (0.36) saucy; urgency=low

  * on mako devices add rmnet_usb[0-3] to /etc/network/interfaces as a
    temporary workaround for LP: #1193161
 -- Oliver Grawert <email address hidden> Wed, 26 Jun 2013 20:38:11 +0200

Oh, i actually thought we'd leave the meta task open until NM was fixed and the hack can be dropped ... did the NM fix happen already ?