99-vmware-scsi-udev.rules has no effect - timeout by 30 not 180

Bug #1790145 reported by T. Scheller
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
open-vm-tools (Debian)
Fix Released
Unknown
open-vm-tools (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

 * Debian delta we had carried renders the udev rules of upstream
   dysfunctional.

 * Drop the Debian Delta to follow upstream more closely and fix this bug.

[Test Case]

 * Set up a VMWare guest
 * boot into that guest
 * The disks appear as scsi disks in the guest and should have a
   /sys/<DEVPATH>/device/timeout of 180 (instead of the default of 30)

[Regression Potential]

 * There could be other systems using our Deb (people backporting prior to
   Bionic) where the older style is needed and there in turn the timeout
   would no more be set correctly. But our concern is the actual target
   release of a package and for that we have to change it to get it right.

[Other Info]

 * n/a

---

Dear all,

System:
$ lsb_release -rd
Description: Ubuntu 18.04.1 LTS
Release: 18.04

$ apt-cache policy open-vm-tools
open-vm-tools:
  Installiert: 2:10.3.0-0ubuntu1~18.04.1
  Installationskandidat: 2:10.3.0-0ubuntu1~18.04.1
  Versionstabelle:
 *** 2:10.3.0-0ubuntu1~18.04.1 500
        500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2:10.2.0-3ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages

Description:
After the update of open-vm-tools and a reboot today the timeout for device sda any VM with open-vm-tools is set to a default timeout at 30.

The rule is made executable also.

/lib/udev/rules.d$ ls -la 99*.rules
-rw-r--r-- 1 root root 4363 Jul 20 19:13 99-systemd.rules
-rwxr-xr-x 1 root root 368 Aug 1 08:16 99-vmware-scsi-udev.rules

$ cat /sys/block/sda/device/timeout
30

Device timeout should be reported as 180.

Can be reproduced on other 18.04.1 after the package update.

Related branches

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Timo,
thanks you for the report! That is an interesting one for sure ...

I know that upstream modified these rules quite a lot for different things not working as inteneded. I will not go ahead and try to modify it right now and re-doing all the same errors over and over again.

Instead I subscribed Oliver Kurth (Upstream open-vm-tools) to take a look and will try to make VMware aware.

Once we have a fix via that route that we know will not kill other things we can look into adding that to the package.

Revision history for this message
Oliver Kurth (okurth-1) wrote :

Hm, again.

I filed an internal bug. Thanks for reporting.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Timo,
I can't reproduce this (I'm sure the bug is right, but I lack the env to do so), but maybe you could quickly check this for me.

Could you check if reverting the following change:

-ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="VMware*", ATTRS{model}=="Virtual disk*", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo 180 >/sys$DEVPATH/device/timeout'"
-ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="VMware*", ATTRS{model}=="VMware Virtual S", ENV{DEVTYPE}=="disk", RUN+="/bin/sh -c 'echo 180 >/sys$DEVPATH/device/timeout'"
+ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="VMware*", ATTRS{model}=="Virtual disk*", ENV{DEVTYPE}=="disk", ATTR{timeout}="180"
+ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="VMware*", ATTRS{model}=="VMware Virtual S", ENV{DEVTYPE}=="disk", ATTR{timeout}="180"

in the file:
/lib/udev/rules.d/99-vmware-scsi-udev.rules

Would resolve the situation?

Revision history for this message
T. Scheller (tschell) wrote :

Hi Christian,

changed the files on the same two systems again.
After reboot, the timeout is again 180.

So yes, confirmed.
The change will resolve the situtaion.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks for the quick turnaround.

That is a change that came in by Debian, which it seems we should remove when on version 10.3 and >=Bionic.

@Oliver - I think you can abort your investigation as upstream is ok.

I have another fix for open-vm-tools in the pipe, I think I'll try to address both in one upload.

@Timo - could you help to ouline the best way to check this?
If I just start a vmware guest, are the default disks already ok to check this?
Or would I need to set up some special disks to get the right type for the rules to apply at all?

description: updated
Revision history for this message
T. Scheller (tschell) wrote :

Hi Christian,

no problem at all.

To your questions, yes with pleasure.

Any disk for a vmware guest should be/is (normally) affected by this rule.
You don`t need to do anything special at all.

You can verify the Type and Vendor with "lshw -C disk" and see if there is a "Virtual disk" by vendor "VMware" which should refer back to the rule definition:
ATTRS{vendor}=="VMware*", ATTRS{model}=="Virtual disk*", ENV{DEVTYPE}=="disk"

And thats just it.

Check the parameter is again just a "cat /sys/block/{{disk}}/device/timeout"

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI: Reported to Debian as well, linked the bug here

Changed in open-vm-tools (Debian):
status: Unknown → New
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hmm,
the Debian maintainer said it works fine for him.

I OTOH finally got a VM System as well and I can confirm the 30 seconds.

$ cat /sys/block/sd[a-c]/device/timeout
30
30
30

Reverting the udev rule change helps me just as much as it helped you Timo.

I verified and wonder where these rules are from.
The upstream rules work.
The Debian Rules work (and look nicer)
But ours don't

FYI: current Debian rules:
ACTION=="add", SUBSYSTEMS=="scsi", ENV{DEVTYPE}=="scsi_device", ATTRS{vendor}=="VMware*" , ATTRS{model}=="Virtual disk*", ATTRS{timeout}=="?*", ATTR{timeout}="180"
ACTION=="add", SUBSYSTEMS=="scsi", ENV{DEVTYPE}=="scsi_device", ATTRS{vendor}=="VMware*" , ATTRS{model}=="VMware Virtual S", ATTRS{timeout}=="?*", ATTR{timeout}="180"

I'll tomorrow sort out where in the merge that was lost and prep a fix

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

But merging we tried to follow upstream more, so we dropped
  ATTRS{timeout}=="?*"
but keeping the nicer Debian style
  ATTR{timeout}="180"
But while not perfect that would still work.

But the other part was about DEVTYPE which upstream has adopted.
So we exchanged Debians
  ENV{DEVTYPE}=="scsi_device"
with upstreams
  ENV{DEVTYPE}=="disk"

And this is what breaks the case here.
So in my initial try to fix it I went the wrong way by following upstream "even more" dropping the rest of d/p/debian/scsi-udev-rule.
Instead I'll pick the latest version and things seem good again with that.

Changed in open-vm-tools (Debian):
status: New → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package open-vm-tools - 2:10.3.0-0ubuntu2

---------------
open-vm-tools (2:10.3.0-0ubuntu2) cosmic; urgency=medium

  * d/p/ubuntu/lp-1791220-Disable-hgfsServer-not-VMware.patch: avoid crashing
    with segfaults when force starting the service in non VMWare environments.
    (LP: #1791220)
  * d/p/debian/scsi-udev-rule: fix applying of the scsi timeout (LP: #1790145)
    - revert "update to match changes in 10.3 (DEVTYPE checkadded upstream)"
    - refresh patch context to match upstream version 10.3

 -- Christian Ehrhardt <email address hidden> Mon, 10 Sep 2018 09:33:13 +0200

Changed in open-vm-tools (Ubuntu):
status: New → Fix Released
Changed in open-vm-tools (Ubuntu Bionic):
status: New → Triaged
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

- SRU Template complete
- Cosmic fixed and confirmed (I retested on vSphere)
- Since other than the version in changelog latest -dev and LTS are kept in sync the MP we had for cosmic essentially applies here as well.

Pushing the same fix to Bionic, now available in -unapproved for the SRU Teams consideration.

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello T., or anyone else affected,

Accepted open-vm-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-vm-tools/2:10.3.0-0ubuntu1~18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in open-vm-tools (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The following log shows the issue before the fix, an upgrade to the version in proposed and then the fixed situation after the fix.

We checked the same code generally from the identical PPA we built before.

ubuntu@node-1:~$ for i in /sys/block/sd*/device/timeout; do echo $i; cat $i; done
/sys/block/sda/device/timeout
30
/sys/block/sdb/device/timeout
30
/sys/block/sdc/device/timeout
30
/sys/block/sdd/device/timeout
30
ubuntu@node-1:~$ sudo apt install open-vm-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  open-vm-tools-desktop
The following packages will be upgraded:
  open-vm-tools
1 upgraded, 0 newly installed, 0 to remove and 58 not upgraded.
Need to get 542 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 open-vm-tools amd64 2:10.3.0-0ubuntu1~18.04.2 [542 kB]
Fetched 542 kB in 1s (524 kB/s)
(Reading database ... 102200 files and directories currently installed.)
Preparing to unpack .../open-vm-tools_2%3a10.3.0-0ubuntu1~18.04.2_amd64.deb ...
Unpacking open-vm-tools (2:10.3.0-0ubuntu1~18.04.2) over (2:10.3.0-0ubuntu1~18.04.1) ...
Processing triggers for ureadahead (0.100.0-20) ...
Setting up open-vm-tools (2:10.3.0-0ubuntu1~18.04.2) ...
Processing triggers for libc-bin (2.28-0ubuntu1) ...
Processing triggers for systemd (237-3ubuntu10.3) ...
Processing triggers for man-db (2.8.3-2) ...
ubuntu@node-1:~$ sudo reboot
Connection to 10.245.205.25 closed by remote host.
Connection to 10.245.205.25 closed.
[✗=255]─[paelzer@lap:~]──[42736]──[18:05 Do Sep 20]──
$ ssh ubuntu@10.245.205.25
Welcome to Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-34-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

  System information as of Thu Sep 20 16:41:11 UTC 2018

  System load: 0.02 Processes: 102
  Usage of /: 20.9% of 29.40GB Users logged in: 0
  Memory usage: 1% IP address for ens160: 10.245.205.25
  Swap usage: 0%

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

 * Canonical Livepatch is available for installation.
   - Reduce system reboots and improve kernel security. Activate at:
     https://ubuntu.com/livepatch

58 packages can be updated.
16 updates are security updates.

Last login: Thu Sep 20 16:34:38 2018 from 10.172.196.172
ubuntu@node-1:~$ for i in /sys/block/sd*/device/timeout; do echo $i; cat $i; done
/sys/block/sda/device/timeout
180
/sys/block/sdb/device/timeout
180
/sys/block/sdc/device/timeout
180
/sys/block/sdd/device/timeout
180

Overall: verified

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for open-vm-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package open-vm-tools - 2:10.3.0-0ubuntu1~18.04.2

---------------
open-vm-tools (2:10.3.0-0ubuntu1~18.04.2) bionic; urgency=medium

  * d/p/ubuntu/lp-1791220-Disable-hgfsServer-not-VMware.patch: avoid crashing
    with segfaults when force starting the service in non VMWare environments.
    (LP: #1791220)
  * d/p/debian/scsi-udev-rule: fix applying of the scsi timeout (LP: #1790145)
    - revert "update to match changes in 10.3 (DEVTYPE checkadded upstream)"
    - refresh patch context to match upstream version 10.3

 -- Christian Ehrhardt <email address hidden> Mon, 10 Sep 2018 09:33:13 +0200

Changed in open-vm-tools (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.