Device validation failing when connected to an usb hub

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1866926/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

There is no SCSI attribute named ref but there is one named rev.

This is likely to be a typo from ubuntu rewrite of the original usb_modeswitch_dispatcher.tcl which reads the rev attribute.

Yes. I mistyped the attribute name. Exact log is:
$ cat /var/log/usb_modeswitch.log

USB_ModeSwitch log from Thu Mar 12 19:25:36 2020

Use global config file: /etc/usb_modeswitch.conf

Raw args from udev: 2-1.2

Bus ID for device not given by udev.
 Trying to determine it from kernel name (2-1.2) ...
Use top device dir /sys/bus/usb/devices/2-1.2

USB dir exists: /sys/bus/usb/devices/2-1.2

SCSI dir exists: /sys/bus/usb/devices/2-1.2
Warning: SCSI attribute "vendor" not readable.
Warning: SCSI attribute "model" not readable.
Warning: SCSI attribute "rev" not readable.
Check class of first interface ...
 Device is in install mode.
 No access to interface 8. Exit

After that the lsusb is:
Bus 002 Device 004: ID 0bda:0138 Realtek Semiconductor Corp. RTS5138 Card Reader Controller
Bus 002 Device 016: ID 2001:a706 D-Link Corp.
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 004: ID 04f2:b289 Chicony Electronics Co., Ltd
Bus 001 Device 003: ID 24ae:2003
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The udevadm monitor reads:
$ udevadm monitor
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent

KERNEL[2853.694871] add /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2 (usb)
KERNEL[2853.695241] add /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0 (usb)
KERNEL[2853.701296] add /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/host6 (scsi)
KERNEL[2853.701409] add /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/host6/scsi_host/host6 (scsi_host)
KERNEL[2853.701545] bind /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0 (usb)
KERNEL[2853.701645] bind /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2 (usb)
UDEV [2853.730704] add /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2 (usb)
KERNEL[2853.733186] add /kernel/slab/:A-0000256/cgroup/filp(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733231] add /kernel/slab/dentry/cgroup/dentry(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733253] add /kernel/slab/:A-0000128/cgroup/pid(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733272] add /kernel/slab/inode_cache/cgroup/inode_cache(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733292] add /kernel/slab/sock_inode_cache/cgroup/sock_inode_cache(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733310] add /kernel/slab/:A-0001024/cgroup/PING(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733326] add /kernel/slab/skbuff_head_cache/cgroup/skbuff_head_cache(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733343] add /kernel/slab/kmalloc-512/cgroup/kmalloc-512(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733361] add /kernel/slab/:A-0000192/cgroup/cred_jar(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL[2853.733384] add /kernel/slab/kmalloc-64/cgroup/kmalloc-64(1373:usb_modeswitch@2-1.2.service) (cgroup)
KERNEL...

On Kubuntu @ Dell with the same modem:
cat /var/log/usb_modeswitch.log

USB_ModeSwitch log from Thu Mar 12 20:09:49 2020

Use global config file: /etc/usb_modeswitch.conf

Adjust delay for USB storage devices ...
Error: could not access delay_use attribute: No such file or directory
Raw args from udev: 3-4

Bus ID for device not given by udev.
 Trying to determine it from kernel name (3-4) ...
Use top device dir /sys/bus/usb/devices/3-4

USB dir exists: /sys/bus/usb/devices/3-4

SCSI dir exists: /sys/bus/usb/devices/3-4
Warning: SCSI attribute "vendor" not readable.
Warning: SCSI attribute "model" not readable.
Warning: SCSI attribute "rev" not readable.
Use interface /sys/bus/usb/devices/3-4/3-4:1.0
----------------
USB values from sysfs:
  idVendor 2001
  idProduct a706
  manufacturer D-Link,Inc
  product D-Link DWM-156
  serial 536591500189710
  bNumConfigurations 3
  bConfigurationValue 1
  devnum 3
  busnum 3
----------------
Found packed config collection /usr/share/usb_modeswitch/configPack.tar.gz
Searching entries named: /usr/share/usb_modeswitch/2001:a706*
Searching overriding entries named: /etc/usb_modeswitch.d/2001:a706*
SCSI attributes not needed, move on.

Extract config 2001:a706 from collection /usr/share/usb_modeswitch/configPack.tar.gz
config: TargetVendor set to 2001
config: TargetProduct set to 7d01
Driver module is "option", ID path is /sys/bus/usb-serial/drivers/option1
! matched, now switching
Device may have an MBIM configuration, check driver ...
 no MBIM driver found, switch to legacy modem mode
Unbinding driver
Command to be run:
/usr/sbin/usb_modeswitch -W -D -s 20 -c /run/usb_modeswitch/current_cfg -b 3 -g 3 -v 2001 -p a706 2>&1

Verbose debug output of usb_modeswitch and libusb follows
(Note that some USB errors are expected in the process)
--------------------------------

Read config file: /run/usb_modeswitch/current_cfg

 * usb_modeswitch: handle USB devices with multiple modes
 * Version 2.5.2 (C) Josua Dietze 2017
 * Based on libusb1/libusbx

 ! PLEASE REPORT NEW CONFIGURATIONS !

DefaultVendor= 0x2001
DefaultProduct= 0xa706
TargetVendor= 0x2001
TargetProduct= 0x7d01

StandardEject=1
Success check enabled, max. wait time 20 seconds
System integration mode enabled

Use given bus/device number: 003/003 ...
Look for default devices ...
 bus/device number matched
  found USB ID 2001:a706
   vendor ID matched
   product ID matched
 Found devices in default mode (1)
Get the current device configuration ...
Current configuration number is 1
Use interface number 0
 with class 8
Use endpoints 0x01 (out) and 0x81 (in)

USB description data (for identification)
-------------------------
Manufacturer: D-Link,Inc
     Product: D-Link DWM-156
  Serial No.: 536591500189710
-------------------------
Sending standard EJECT sequence
Looking for active drivers ...
 OK, driver detached
Set up interface 0
Use endpoint 0x01 for message sending ...
Trying to send message 1 to endpoint 0x01 ...
 OK, message successfully sent
Read the response to message 1 (CSW) ...
 Response successfully read (13 bytes), status 0
Trying to send message 2 to endpoint 0x01 ...
 OK, message successful...

Make the comment on Kubuntu irrelevant.
I have launched Kubuntu Live CD daily and the same behavior can be observed.
The laptop is Toshiba Satellite L755.
Let me know if I can provide any further useful information.

Without being familiar with the Ubuntu flavour of usb_modeswitch - can this be a permission issue?

Does the usb_modeswitch wrapper and program run in the proper user / group context? It needs to be able to read attributes in the "sys" tree.

If the permissions are not correct, that would explain why the manual run with "sudo" works as expected.

Please note that on a Dell laptop everything works as expected. The difference most probably is caused by hardware in the laptop. Two different modems do not work with Toshiba and both of them work with Dell. The same up-to-date "focal fossa" system.

I am likely experiencing the same issue, yet on other hardware: Ubuntu 20.04 with a ZTE MF100 modem but only on Raspberry Pi. The good news: I have triaged the issue.

The bug is in "usb_modeswitch_dispatcher.c", a C rewrite of the upstream Tcl script. The C dispatcher receives the modem's kernel_name. If it contains the characters dot ('.') and dash ('-') but no colon (':'), the device is marked for some extended "interface validation" (see lines 343ff of "usb_modeswitch_dispatcher.c"). A bit further down at lines 388ff, the "iface" is initialized to 0, but the interface validation updates it to 8. The value 8 stems from an atoi() invocation on the contents of "bInterfaceClass". For this updated iface value, the C dispatcher tries to find the corresponding (sub) interface directory and this fails because my modem does not have 8 sub devices. Until here, 18.04 and 20.04 behavior is identical. However, the following new code was introduced in 20.04 right after the directory lookup attempt is made:

----------8<----------
if (ifdir == NULL) {
  modeswitch_log(" No access to interface %d. Exit\n", iface);
  return 1;
}
----------8<----------

This is causing the early exit for Mariusz and me. If I remove this code, the rest of the C dispatcher works fine (even with a wrong iface value of 8 - this is the same behavior as it was on 18.04). In a scenario where no interface validation takes place (e.g. because of a colon in the kernel_name or the absence of a dot - such as Ubuntu 20.04 on x86_64 VM), the iface value of 0 remains unchanged and the interface directory is looked up successfully.

I have tried to match the C dispatcher code with the upstream Tcl script. The Tcl script initializes iface to 0 and eventually updates it to "1.0" when doing the interface validation branch.

Another thing that I noticed is that the C dispatcher erroneously uses atoi() on the contents of bInterfaceClass. The content is a hexadecimal string such as "00", "08" or "ff" while atoi() expects a decimal string. If bInterfaceClass contains "ff", atoi() would return 0.

I have solved this problem by replacing "usb_modeswitch_dispatcher" with the upstream Tcl script, pointing its interpreter to jimsh and installing jimsh. I think it would be a wise choice to ditch the C dispatcher, use the upstream Tcl script and make the package depend on jimsh. Reasons:
- The C dispatcher does not implement the same functionality as the Tcl script in 2.5.2.
- Implementing changes from the Tcl script in 2.6.0 would be even more painful.
- The usb_modeswitch author provides flavors that eliminate the Tcl dependency.
- I fail to see how the "binary size" argument still applies in 2020 to machines that are capable of running Ubuntu.

To further clarify the impact of different hardware on bug occurrence: The difference in Mariusz' hardware (Dell vs. Toshiba) leads to different kernel names ("3-4" on Dell vs. "2-1.2" on Toshiba). So "device validation" only happened on his Toshiba (thereby updating iface to the wrong value of 8), but not on Dell. Similarly, I had a problem on RPi 3 (kernel_name = "1-1.3") but no problem on an x86_64 VM (kernel_name = "1-1"). My hardware and VM both ran on Ubuntu 20.04 LTS.

This is the commit that fixed a segfault in the C dispatcher but introduced the problem for Mariusz and me: https://git.launchpad.net/ubuntu/+source/usb-modeswitch/commit/?id=4ef455d6d34fa465945db2b18f5dcabc604a9888

Status changed to 'Confirmed' because the bug affects multiple users.

The dot in the kernel name of a USB device indicates that a device is connected via a hub.
In theory, this can even be cascaded, leading to names like "2-1.2.1.3.4"

So the hardware difference is that the ports in the Toshiba are not 'primary' but exposed through an internal hub as opposed to the Dell.

I just spent a few hours trying to figure out why my Alcatel modem wasn't establishing a connection...and ran across this bug. Thank you Hanno for the workaround!

I uninstalled the usb_modeswitch package and installed the one from Debian Bullseye, and my modem shows up now! After adding entries to my netplan configuration to enable DHCP, I can access the interfaces.

The Ubuntu package has been updated to remove the C rewrite and use the upstream tcl version now which should fix the issue
https://launchpad.net/ubuntu/+source/usb-modeswitch/2.6.0-2ubuntu1

We are not going to add tcl as a depends in a stable update though so the C code bug is still going to need to be resolved in focal

Hi all,

I'm the one who accidentally introduced the bug when fixing https://bugs.launchpad.net/ubuntu/+source/usb-modeswitch/+bug/1676763. I'm sorry for causing the problem, I will try to find a fix that will work in both scenarios.

Best regards,

Jakub

I'm starting to feel that trying to fix the C rewrite is futile.

On line 392 of usb_modeswitch_dispatcher.c, the iface variable is overwritten. Unfortunately the value being assigned is not an interface number, it is rather a USB class number. Fixing this might help with some part of this, but it also might break something else. I was also trying to map between the C rewrite and the 2.2.5 version of the tcl script (as stated in the header of the C version), but it seems that there are functional differences (for example the if_chk mode seems to trigger on different ocassions).

I'd vote for going with the Tcl version for Focal as well (I'm ignoring any existing SRU rules through). The dependency isn't a full Tcl interpreter, only a small library-based variant of it called Jim (http://jim.tcl.tk/index.html/doc/www/www/index.html). According to the package description, the library size is only 100-200 kB, which seems negligible compared to the current ISO sizes.

Hmmm, the Ubuntu version does depend on Tcl. I think a better solution would be to go with Jim (as Debian does: https://packages.debian.org/buster/usb-modeswitch) for both Focal and Groovy.

Let me just explain shortly why I'm sticking with Tcl for the dispatcher.

That script language is perfectly fitted for string and file parsing. At the same time, it's much better human-readable than a Bash or Perl script. I had people adding fixes to the wrapper script who weren't even familiar with the language.

Execution time is not really relevant regarding the mode-switching process, because its duration predominantly depends on the device's internal workings until it 'reappears' in the new mode.

Both the bare interpreter tclsh *and* the 'pocket version' (jimtcl) have everything on board for the task. No loading of additional extensions or modules required.

Besides, I somehow can't warm up to Python but that's my personal issue :-)

@Jakub, pulling it an extra interpreter in a stable update is probably not something we want to do so we are going to either fix the issue in the C code or let <= focal versions have issues in some cases. I do agree that starting doing changes to the logic might be something we want to avoid, there is probably an easier way to restore the previous behaviour but in a way that avoid the segfault right?

Also not that Jim is currently in Universe and not something Canonical has commited to maintain, which is why we are using tcl (which is in Ubuntu main)

@Josua, the problem there isn't the language or startup performance but having to maintain one extra language/interpreter for us (Canonical/Ubuntu), it means we have to keep someone on team familiar with tcl and to commit to fix any security problem find in the tclsh (or jimtcl if we decide to promote this one) which is adding maintenance load of a small team already having lot to do

Sebastien, I understand your concerns with regard to jimtcl. However, the tcl package seems to be well maintained, with very few bugs (2) in the current version, one of low urgency and one undecided.

Anyway, it's not my decision to make obviously. I trust that you know what's best for Ubuntu.

@Josua, agreed and as mentioned on comment #13 the package has been made to use tcl again now, in an ideal world we would prefer to have one less interpreter on the default installation to maintain but in practice the cost is low so it will do