multipath-tools update introduced syslog messages about partx

Status changed to 'Confirmed' because the bug affects multiple users.

Hello Matt,

Are there any other error messages available when this happens ?

I'm looking specifically for Illegal SCSI requests similar to this one:

[ 1923.361604] sd 7:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 1923.361621] sd 7:0:0:0: [sdb] tag#0 Sense Key : Illegal Request [current]
[ 1923.361628] sd 7:0:0:0: [sdb] tag#0 Add. Sense: Invalid field in cdb
[ 1923.361635] sd 7:0:0:0: [sdb] tag#0 CDB: Write same(16) 93 08 00 00 00 00 00 00 29 10 00 00 0f f8 00 00
[ 1923.361641] blk_update_request: critical target error, dev sdb, sector 10512
[ 1923.362078] blk_update_request: critical target error, dev dm-2, sector 10512
[ 1923.363433] blk_cloned_rq_check_limits: over max size limit.

Here is an example of a bunch of failover events from a rebooted storage controller. You can see that there are others similar to what you mentioned above.

Hi Matt-

do you mind posting your "/etc/multipath.conf" from the host you provided the last syslog.

And also if you can provide the output of "sudo multipath -ll" from the same host.

Thank you

I have no multipath.conf, we just use the defaults that are built in for our storage.

multipath -ll for that particular host is not going to look good right now as I'm having an issue with an Emulex device. Until I get my hardware issue resolved I don't want to send you an output that will be confusing as I only have one path right now :)

This is the output of multipathd show config, which is the effective multipath.conf file.

Thanks for the info Matt. I am looking into the case.

If you replace:
uid_attribute "ID_SERIAL"

with:
getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"

in your multipath.conf and restart your multipathd those messages should go away.

The line should be in your "defaults" section of the config.

Hi Matt-

just wanted to give you a quick update on the issue. We've isolated the problem and are now working with upstream developer to push the patch.

Patch has been submitted upstream on:

1. Thu, 23 Jun 2016 16:06:24 -0500
[dm-devel] Subject: [PATCH 1/1] multipath-tools: fix dm- device filtering
https://www.redhat.com/archives/dm-devel/2016-June/msg00359.html

2. Tue, 5 Jul 2016 15:46:57 -0500
[dm-devel] [PATCH 1/1] multipath-tools: fix dm- device filtering
https://www.redhat.com/archives/dm-devel/2016-July/msg00086.html

I has been accepted upstream on:
Fri, 8 Jul 2016 09:42:43 +0200
https://www.redhat.com/archives/dm-devel/2016-July/msg00159.html

Here is the debdiff for the fix. Working on up-streaming this fix to debian.

The attachment "Fix debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

Do I have to rebuild the package with this debdiff before it can be accepted?

Hi Mike,

I already built the PPA for this change. You can get it by running following commands:

add-apt-repository ppa:dragan-s/lp1570093
apt update
apt install multipath-tools

With this fix the errors like the ones below should be gone:
Apr 13 11:21:00 ICTM1618S01H4 multipath: dm-15: failed to get udev uid: Invalid argument
Apr 13 11:21:00 ICTM1618S01H4 multipath: dm-15: failed to get sysfs uid: Invalid argument
Apr 13 11:21:00 ICTM1618S01H4 multipath: dm-15: failed to get sgio uid: No such file or directory

Err... -ENONAME sorry meant to type Matt

Fix has been submitted upstream into Debian:

Tue, 12 Jul 2016 11:30:15 -0500
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830882

I can confirm that the messages seem to be cleaned up with your fix.

Great, thank you Matt. Independent verification of the fix will now allow it to move upstream.

This bug was fixed in the package multipath-tools - 0.5.0+git1.656f8865-5ubuntu4

---------------
multipath-tools (0.5.0+git1.656f8865-5ubuntu4) yakkety; urgency=medium

  [ Dragan Stancevic ]
  * multipath-tools: fix dm- device filtering
   - Allow filtering of DEV_UEVENT dev_type
   - Resolves LP: #1570093

 -- Louis Bouchard <email address hidden> Mon, 18 Jul 2016 12:37:04 +0200

This bug needs SRU information before the fix can be uploaded to Xenial please. Some dep3 headers in the patch would be nice too.

rbasak: the uploader is to blame on this one :-/

For some reason, I uploaded the package with only a changelog entry ! I'll respin the upload while dragan-s completes the SRU template.

Hello Matt, or anyone else affected,

Accepted multipath-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.5.0+git1.656f8865-5ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hi Brian-

I can confirm that after:
apt-get install multipath-tools/xenial-proposed

the messages below went away:

Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 systemd[1]: Started Device-Mapper Multipath Device Controller.
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-3: failed to get udev uid: Invalid argument
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-3: failed to get sysfs uid: Invalid argument
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-3: failed to get sgio uid: No such file or directory
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-4: failed to get udev uid: Invalid argument
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-4: failed to get sysfs uid: Invalid argument
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-4: failed to get sgio uid: No such file or directory
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-5: failed to get udev uid: Invalid argument
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-5: failed to get sysfs uid: Invalid argument
Aug 16 15:05:08 ubuntu-SRV-16-04-lp1570093 multipath: dm-5: failed to get sgio uid: No such file or directory

Package version tested:

ubuntu-SRV-16-04-lp1570093:~$ apt-cache policy multipath-tools
multipath-tools:
  Installed: 0.5.0+git1.656f8865-5ubuntu2.2
  Candidate: 0.5.0+git1.656f8865-5ubuntu2.2
  Version table:
 *** 0.5.0+git1.656f8865-5ubuntu2.2 400
        400 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     0.5.0+git1.656f8865-5ubuntu2.1 500
        500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
     0.5.0+git1.656f8865-5ubuntu2 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

This bug was fixed in the package multipath-tools - 0.5.0+git1.656f8865-5ubuntu2.2

---------------
multipath-tools (0.5.0+git1.656f8865-5ubuntu2.2) xenial; urgency=medium

  * multipath-tools: fix dm- device filtering
   - Allow filtering of DEV_UEVENT dev_type
   - Resolves LP: #1570093

 -- Dragan Stancevic <email address hidden> Thu, 21 Jul 2016 11:16:44 +0200

The verification of the Stable Release Update for multipath-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

test, ignore