usb-modeswitch can't apply Configuration=0 to Snapdragon X12 LTE

Please describe the problem here, rather than in forum posts. It's not helpful to have to go through various pages and posts from different people, with information that is potentially irrelevant to the bug. To compound to the situation, pastebins have disappeared by now.

Please include all the debugging information you can here, and a clear description of the problem, and then it should be relatively easy to fix the bug.

From my understating this issue boils down to not dropping new line symbol from kernel version string when function in C program check presence of cdc_mbim kernel module. It can’t find cdc_mbim kernel module because it looking in folder that doesn’t exist, so automatic changing LTE modem mode to MBIM doesn’t happen and it stays in Ethernet mode, which doesn’t work.

> Please include all the debugging information you can here

check_mbim_available function was modified like this:

check_mbim_available (void)
{
   int kver_fd = -1;
   ssize_t read_count;
   char kversion[100];
   char *cdc_mbim_module_path;
   struct stat buf;

   kver_fd = open(OSRELEASE_FILE, O_RDONLY);
   if (kver_fd > -1) {
      read_count = read(kver_fd, &kversion, 100);
      if (read_count > 0) {
         asprintf(&cdc_mbim_module_path, "%s/%s/%s",
                  MODULES_PATH,
                  kversion,
                  CDC_MBIM_DRIVER_PATH);
+ modeswitch_log("\n -> Let's check cdc_mbim_module_path : %s\n", cdc_mbim_module_path);
         if (!stat(cdc_mbim_module_path, &buf))
            goto success;
         if (!stat(CDC_MBIM_SYS_PATH, &buf))
            goto success;
      }
   }

   free(cdc_mbim_module_path);
   return 0;
success:
   free(cdc_mbim_module_path);
   return 1;
}

In result log which was on expired pastebin it was visible that patch that was printed after "Let's check cdc_mbim_module_path" message include line break. It was right after kernel version, that filled by #define OSRELEASE_FILE "/proc/sys/kernel/osrelease"

Example:
~$ cat /proc/sys/kernel/osrelease
4.19.0-041900-generic
~$ cat /proc/sys/kernel/osrelease | hexdump
0000000 2e34 3931 302e 302d 3134 3039 2d30 6567
0000010 656e 6972 0a63
0000016

Problem is that string in /proc/sys/kernel/osrelease include line break symbol, and nothing in dispatcher-c-rewrite.patch drop this new line symbol, so when check_mbim_available trying to check if cdc_mbim kernel module is present, it looking for "4.19.0-041900-generic\n" (\n is not visible but it's part of the string) instead of "4.19.0-041900-generic" folder.

I built usb-modeswitch with tcl8.6-dev and dispatcher-c-rewrite.patch disabled, and mobile broadband still works with my old USB thing.

No extra runtime depends.

https://launchpad.net/~gunnarhj/+archive/ubuntu/usb-modeswitch

@Mathieu: Would appreciate your input. Would this be a suitable step to take?

@RussianNeuroMancer: Does this possibly address the issue you are having?

On 2019-08-26 23:42, Gunnar Hjalmarsson wrote:
> I built usb-modeswitch with tcl8.6-dev and
> dispatcher-c-rewrite.patch disabled, and mobile broadband still works
> with my old USB thing.
>
> No extra runtime depends.

That does not make sense, apparently.

Changed it so the usb-modeswitch binary depends on tcl8.6, and the TCL dispatcher is run using /usr/bin/tclsh8.6.

The tcl8.6-dev build depend proved to not be needed, so I dropped it. Actually there is not much building happening. Not sure if that is an indication that something is wrong...

The upstream source has a jim/ folder with a lot of stuff, but that folder is excluded in the Debian/Ubuntu source.

As RussianNeuroMancer correctly pointed out the bug is in the concatenation of the strings
MODULES_PATH, kversion, and CDC_MBIM_DRIVER_PATH into cdc_mbim_module_path.

The ending linefeed char in kversion is the culprit and it makes the dispatcher unable to detect the presence of a cdc-mbim module.
usb_modeswitch will therefore not be able to automagically select the mbim configuration in the modem, usb_modeswitch thinks that the mbim driver is not present in the system.

> Does this possibly address the issue you are having?

Yes, I can confirm that with Configuration=0 in /usr/share/usb_modeswitch/03f0:0857 and usb-modeswitch package from ppa:gunnarhj/usb-modeswitch usb-modeswitch service automatically switch bConfigurationValue of HP lt4220 (Snapdragon X12 LTE) to 3. In result - modem works.

@RussianNeuroMancer: Thanks for confirming.

Then we can fix this bug either by modifying dispatcher-c-rewrite.patch or by replacing it with upstream's TCL code (as is done in the PPA version).

I'm hoping for Mathieu's input on the latter option. It would make maintenance easier going forward, but the question is if the tcl8.6 dependency is ok.

Otherwise, given your analysis and Lars' confirmation, fixing the patch should be easy for a C programmer.

Am 27. August 2019 04:04:34 GMT+02:00 schrieb Gunnar Hjalmarsson <email address hidden>:
>
> The upstream source has a jim/ folder with a lot of stuff, but that
> folder is excluded in the Debian/Ubuntu source.

The inclusion of the jim shell code is just meant to be a suggestion for users with space restraints who need something smaller than the full-featured TCL shell/package. It can be dropped of course if separate packages of tcl or jim are present.

@Josua: Thanks for that explanation.

Since you are here, may I take the opportunity to ask a question. I think that Mathieu would prefer that TCL somehow is compiled into a binary, and that's what README indicates will happen with the installed-shared option. But Debian builds with installed-shared, and they still depend on libjim.

So my question would be: Is it possible with any of the prepared install options to compile TCL into an executable binary (comparable with what Ubuntu is currently doing using the patch)?

Am 27. August 2019 19:17:09 GMT+02:00 schrieb Gunnar Hjalmarsson <email address hidden>:
>@Josua: Thanks for that explanation.
>
>Since you are here, may I take the opportunity to ask a question. I
>think that Mathieu would prefer that TCL somehow is compiled into a
>binary, and that's what README indicates will happen with the
>installed-
>shared option. But Debian builds with installed-shared, and they still
>depend on libjim.

That's what the "install static" target is supposed to provide. It should link jim statically, resulting in a reasonably sized binary with no special dependecies.

To be honest, I haven't tested this for a long time since Debian decided to use the shared libjim from its own package.

There is no enthusiasm among the Ubuntu developers to go back to a TCL dispatcher, so I have made an alternative fix via the attached debdiff where I simply modified the C patch by removing the problematic newline. Built with the modified patch in this PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/usb-modeswitch2

@RussianNeuroMancer: Any chance you can test this variant too, before I ask a sponsor to upload the patch?

@Josua: Thanks for your answer! (I didn't see it before posting the debdiff.)

I may make an attempt with install-static later then. Possibly a better alternative for Ubuntu (easier maintenance) than keeping the C patch.

> Any chance you can test this variant too, before I ask a sponsor to upload the patch?

This variant fail to apply configuration with following error:

> usb_modeswitch_dispatcher[2373]: Could not read attribute: No such file or directory

Probably there is other issues besides new line.

@RussianNeuroMancer: Thanks for testing! Is that the same error you get when you run the usb-modeswitch version in the Ubuntu archive?

Attached please find a test script I put together. Can you please run test:

./test

and let us know what it outputs.

When I run it, it says:

File "/lib/modules/5.0.0-25-generic/kernel/drivers/net/usb/cdc_mbim.ko" found!

(It's installed by the linux-modules-extra-5.0.0-25-generic package.)

> Is that the same error you get when you run the usb-modeswitch version in the Ubuntu archive?

Yes, same error: "usb_modeswitch_dispatcher[3587]: Could not read attribute: No such file or directory"

Installed version is: 2.5.2+repack0-2ubuntu1

> Can you please run test

Sure, output is:

~$ ./test
File "/lib/modules/5.2.10-050210-generic/kernel/drivers/net/usb/cdc_mbim.ko" found!

So to summarize you found the newline issue, but that proved to not be related to the error message you get, even if fixing it may be necessary as a partial solution. If I modify the test script by commenting the newline fix I get:

File "/lib/modules/5.0.0-25-generic
/kernel/drivers/net/usb/cdc_mbim.ko" not found.

I.e. no error message.

So we are kind of stuck. There is a working TCL version in the first PPA, but I'm not permitted to propose it for upload.

@Mathieu, @Steve: Are the reasons for carrying the dispatcher as an Ubuntu specific C program still strong enough?

Rather than pasting just a single error line, please include the full debug log for running usb-modeswitch. You can do that by modifying /etc/usb_modeswitch.conf:

EnableLogging=1

And looking under /var/log/ for the log file written by usb-modeswitch after it has run.

Having the full file means we know exactly what happens as usb-modeswitch tries to probe the device, it will be much easier to figure out what goes wrong and why.

The "Could not read attribute" might not be fatal by itself; something else is likely making it impossible to figure out what type of device exactly it is and whether it is in the right mode.

Log of usb-modeswitch with disabled dispatcher-c-rewrite.patch and without /usr/share/usb_modeswitch/03f0:0857

Log of usb-modeswitch with fixed dispatcher-c-rewrite.patch and without /usr/share/usb_modeswitch/03f0:0857

Log of usb-modeswitch with fixed dispatcher-c-rewrite.patch and with /usr/share/usb_modeswitch/03f0:0857 (Configuration=0)

So last time (#16) usb-modeswitch service failed for me because there was no configuration for this modem in usb-modeswitch-data - result log is in #20
It's possible to workaround this be creating configuration manually (#21) but version without dispatcher-c-rewrite.patch is more convenient - it configured modem even without configuration file (#19).

@RussianNeuroMancer: Thanks for the additional info. From what you now say, and unlike how I understood comment #14, the newline fix *does* make a difference, i.e. it addresses the problem description in the bug summary: "usb-modeswitch can't apply Configuration=0 to Snapdragon X12 LTE". Is that correct?

(But even if it is, for your device there is indeed a usability difference between the C and TCL dispatcher. Advanced and patient users like yourself may be able and willing to deal with it, while less tech-savvy users may well give up.)

> i.e. it addresses the problem description in the bug summary: "usb-modeswitch can't apply Configuration=0 to Snapdragon X12 LTE". Is that correct?

Yes.

@RussianNeuroMancer: With your log files as guidance, Mathieu has made several modifications of dispatcher-c-rewrite.patch, which I put in the second PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/usb-modeswitch2

Any chance you can try that too?

Just in case I want to clarify that I did mistake in #19 and #22 - I totally forgot that besides configuration file I also added couple of lines with modem id's to /lib/udev/rules.d/40-usb_modeswitch.rules:

# HP lt4210
ATTR{idVendor}=="03f0", ATTR{idProduct}=="0857", RUN+="usb_modeswitch '/%k'"

Obviously, without this two lines even usb-modeswitch with disabled dispatcher-c-rewrite.patch will not able to switch modem mode automatically, as nothing will run usb_modeswitch against device path.

So, I wrote this message to clarify that all usb_modeswitch runs above and below happened with modified /lib/udev/rules.d/40-usb_modeswitch.rules

Log of usb-modeswitch 2.5.2+repack0-2ubuntu2~mtrudel1 and without /usr/share/usb_modeswitch/03f0:0857

Log of usb-modeswitch 2.5.2+repack0-2ubuntu2~mtrudel1 and with /usr/share/usb_modeswitch/03f0:0857 (Configuration=3)

Log of usb-modeswitch 2.5.2+repack0-2ubuntu1 from Ubuntu eoan repository and with /usr/share/usb_modeswitch/03f0:0857 (Configuration=3)

Shortly, 2.5.2+repack0-2ubuntu2~mtrudel1 version does not work with or without /usr/share/usb_modeswitch/03f0:0857

I created /usr/share/usb_modeswitch/03f0:0857 with Configuration=3 specifically to check if 2.5.2+repack0-2ubuntu2~mtrudel1 can behave at least like usb-modeswitch 2.5.2+repack0-2ubuntu1.

Thanks! I'll leave it to Mathieu to ask possible follow-up questions.

However, there is one thing I wonder ATM: When you say "with disabled dispatcher-c-rewrite.patch" above, are you then talking about not using a dispatcher at all, or are you referring to the variant with the TCL dispatcher in the first PPA, i.e. <https://launchpad.net/~gunnarhj/+archive/ubuntu/usb-modeswitch>?

I referring to the variant with the TCL dispatcher in the first PPA. You described this version in #4 as "dispatcher-c-rewrite.patch disabled" so this is why I call it like this.

So with my PPA version, what do you have in /run/usb_modeswitch/current_cfg then?

I suspect the program is crashing, I probably made a small mistake in the patch?

Currently I doesn't have access to hardware. I will try to check /run/usb_modeswitch/current_cfg content with latest build from PPA this Friday, or next week.

This bug was fixed in the package usb-modeswitch - 2.6.0-2ubuntu1

---------------
usb-modeswitch (2.6.0-2ubuntu1) groovy; urgency=medium

  * Resynchronize on Debian
  * Remove dispatcher-c-rewrite, it has issues and no active maintainer
    (LP: #1800794)
  * debian/rules:
    - don't use the dynlink option, we use tcl rather libjim which
      is in Universe

  [ Gunnar Hjalmarsson ]
  * debian/control:
    - Drop build-deps libpipeline-dev and libudev-dev
    - Depends on tcl8.6
  * debian/rules:
    - Set TCL=/usr/bin/tclsh8.6

usb-modeswitch (2.6.0-2) unstable; urgency=medium

  * Bump S-V to 4.4.1 without changes needed
  * Set Rules-Requires-Root to no

usb-modeswitch (2.6.0-1) unstable; urgency=medium

  * Update to new upstream version 2.6.0; the repack is not needed anymore as
    the embedded 'jim' library has been taken out
  * Convert to git-debrebase workflow
  * Refresh patches
  * d/rules: Update make targets
  * Enable the Salsa CI Team standard pipeline

 -- Sebastien Bacher <email address hidden> Tue, 30 Jun 2020 16:37:41 +0200

Issue is no longer reproducible on Linux 5.8, although on 5.9 there is regression: https://bugzilla.kernel.org/show_bug.cgi?id=209469

*on Ubuntu 20.10 with Linux 5.8

@RussianNeuroMancer: This bug is closed, so please file a new bug if you have found a new issue. OTOH, an issue with Linux 5.9 isn't reasonably an Ubuntu bug yet. If you think there is a need to modify usb-modeswitch to work with Linux 5.9, then I think it's best to start a new topic in the upstream forum.