udevd inotify_add_watch no such directory error for physical volumes

Confirming this bug, similar situation here with multiple partitions defined as LVM physical volumes belonging to a volume group.

> sudo fdisk -l /dev/sda
>
> Disk /dev/sda: 320.1 GB, 320072933376 bytes
> 255 heads, 63 sectors/track, 38913 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk identifier: 0xae329394
>
> Device Boot Start End Blocks Id System
> /dev/sda1 * 1 6332 50861758+ 7 HPFS/NTFS
> Partition 1 does not end on cylinder boundary.
> /dev/sda2 6333 38913 261706882+ 5 Extended
> Partition 2 does not end on cylinder boundary.
> /dev/sda5 6333 7087 6056473+ 83 Linux
> /dev/sda6 7087 10734 29294496 83 Linux
> /dev/sda7 28470 33691 41945683+ 8e Linux LVM
> /dev/sda8 33692 38913 41945683+ 8e Linux LVM
> /dev/sda9 23248 28469 41945683+ 8e Linux LVM
> /dev/sda10 18026 23247 41945683+ 8e Linux LVM
> /dev/sda11 12804 18025 41945683+ 8e Linux LVM
> /dev/sda12 10735 12803 16619211 83 Linux
>
> Partition table entries are not in disk order

bug 441367 could be related, although it does not seem to be a real duplicate

This bug makes my boot-process stop for about 10-15 seconds, so it is really "bugging" me - what can be done to improve the situation?

Same problem on a fresh kubuntu lucid install. The messages seem to randomly oscilate between real devices (not partitions) though, /dev/sda /dev/sdf /dev/sde etc. Makes the boot look ugly and somewhat slower but does not seem to cause other issues

sorry, my pc as crashed... is death definitelly...
now i can buy a new pc
new pc= news bug
this bug is not confirmed a in to the new pc...

good work friends...

hem, sorry for my english...

Same problem here. During the boot process, before the Ubuntu boot logo appears, the following message appears:

udevd-work [XXX]: inotify_add_watch(6, /dev/sdb9, 10) failed: no such file or directory

(Where XXX might be a process pid number, its value varies). Sometimes a second message appears along width this one, it complains about /dev/sda2. Both sda2 and sdb9 are physical volumes and are part of the same volume group. In this volume group there is a logical volume which I use as my /home partition.

Apart from these error messages I experienced no further problems so far, not even the 10-15 sec boot delay mentioned in msg #4. The screen is blank with only these errors on it for a few seconds, but the boot process continues in the background, so I think the empty screen is due to another problem with plymouth and the binary NVIDIA driver (at least in my case).

I don't know what informations I should provide, if there's something to add please let me know.

I recently update from 8.04LTS up to date to the new 10.04LTS.
Same problem for me when rebooting after doing a grub update.
udevd-work [XXX]: inotify_add_watch(x /dev/sda7, x) failed: no such file or directory
/dev/sda7 is my main lvm partition :
Périphérique ..... Id Système
/dev/sda7 .... 8e Linux LVM

To reproduce :

$ sudo update-grub
error: cannot open `/dev/sdb' while attempting to get disk size.
error: cannot open `/dev/sdb' while attempting to get disk size.
Generating grub.cfg ...
error: cannot open `/dev/sdb' while attempting to get disk size.
error: cannot open `/dev/sdb' while attempting to get disk size.
...... cutted ....
error: cannot open `/dev/sdb' while attempting to get disk size.
error: cannot open `/dev/sdb' while attempting to get disk size.
Found linux image: /boot/vmlinuz-2.6.32-22-generic
Found initrd image: /boot/initrd.img-2.6.32-22-generic
error: cannot open `/dev/sdb' while attempting to get disk size.
...... etc

Anyway, grub is well updated.
This bug doesn't seems impacting anything in my case as my /dev/sdb is not a bootable disk

I had the same problem. After installing cryptsetup the problem no longer occurred. I assume that the cryptdisks-udev files in the /etc/init/ and /etc/init.d somehow influence the boot process in a positive way.

I'm curious to know if this solves the problem for you too.......

Greetings,
    Frank

Same problem here in 3 servers after upgrading from 9.10 to 10.04.
All 3 servers have LVM with multiple physical volumes

I make the Frank test and after installing cryptsetup the problem no longer occurred for me too.
I think that cryptsetup disables something about udev.

Hi all,

I have not been able to reproduce the error by grub-update as Nico suggested.

Have followed tuxfrank's advice and - indeed - error disappears after installing cryptsetup

Cheers

Frank Claessen

installing cryptsetup fixed it for me too.
But the bug remains. Now somebody needs to know WHAT cryptsetup is fixing exactly and have this fix pushed upstream.

Installing cryptsetup fixed it for me too, but I don't really consider this to be a fix. I don't need cryptsetup, and don't like the idea that I'm loading something I don't need upon every boot to mask a bug. I notice this bug is confirmed but still unassigned. Is it considered low priority? I've just installed Ubuntu 10.04, but am considering wiping it out and switching to Debian because of this and another (login screen freeze) issue I've had since installing Ubuntu.
Thanks,
Brett Stahlman

this bug affected me too and installing cryptsetup fixed the bug. now i can see my splash :P

I had the same issue, and the solution suggested by tuxfrank worked for me, too. But I agree with Sic Volo and Brett Slahlman -- the bug remains, we just covered it for a while by a workaround.
Besides, after this bug appeared, my mother filesystem started acting weird, announcing severe errors and suddenly becoming a read-only file system until a reboot. I'm not sure if these issues are connected in any way, but they really seem to be.

I'm fairly certain that this happens because the kernel device-mapper removes the devices associated with the partitions that make up the LVM set *before* udev has an opportunity to process the original event. In essence, udev is racing.

GOOD NEWS EVERYONE!

I did it! I mean track it down ... it IS indeed weird and took me 4 hours... :/

Ultrashort FIX (if not boot disk is affected):
- sudo aptitude purge dmraid

YES! It is the f*ke-Raid module, that causes such headache.....is it?
NO! It is the Ubuntu Installer...

Reproduce:
- Start your LIVECD (install or rescue)
- While HD-Setup answer "Enable SATA Raid devices" once with YES and once with NO

1st: This is a very poor option description! - also with 10 years of HW-experience - till now i did not fully understand this option!
Please add some additonal text!
2nd: So - in my case, i was installing WITHOUT this option - everything went fine. And now i did find out, that dmraid is INSTALLED and ENABLED at startup - WHY?!

For dmraid developers:

-- Board: http://www.supermicro.com/products/motherboard/PD/E7230/PDSME.cfm
-- 2x Seagate HD 200GB on sata PORTS
-- FastTrak/Promise SATA-Raid Controller

yes, with ONLY 2 HDs on internal board ports (and empty raid-controller), the partiton nodes are removed if dmraid is enabled...

WOW...
T.Y. for your work...
(^_^)

now i test it...

Good Work

Christian

Many thanks for tracking this down.

The only partitions that generated errors on my system were the LVM managed extended partitions.

Thanks Christian for you work.
I quite forgot this issue as it was not really a problem for me.
In my case I never dmraid installed.
When I use "sudo update-grub" now, this error is not displayed anymore.

Hey Folks,
This might well be a bug in dmraid, but purge dmraid is not a solution either.
Because I DO use dmraid for a Raid 1 Array and get the Same error.

udevd-work [xx]: inotify_add_watch(6, /dev/dm-0, 10) failed: no such file or directory

I don't think this is a dmraid issue (since I've removed dmraid and the bug persists). However, since my LVM partitions are generating these errors I wonder if it could be that certain partition formats (LVM and dmraid) are unreadable by inotify, hence the error?

I have similar problems. Since I can't see relevant output in my log files, I am attaching screenshot which seems to be more verbose.

Same as Simon Simon Rodan - I need dmraid since I want to boot from my (damned expensive) Revodrive X2, which is in fact nothing else but a dmraid connection 4 Sandforce SSDs.

I have these error messages with Maverick running on a Sony VGN-AR590E laptop with Intel Mobile 82801 SATA RAID Controller. The first time I installed Maverick, it was causing random boot failures. After installing cryptsetup, it fixed it, but I still experience the error messages (udevd-work [xx]: inotify_add_watch(6, /dev/dm-x, 10) failed: no such file or directory) on boot.

@Mark Wilmoth

Experienced the same. After reading this thread I tried installing 10.10 on the Revo with 'encrypt home directory' which, obviously, will install cryptsetup the startup process went one stop further and then hang up while showing the splash-screen (it didn't show that before).
Starting in 'recovery mode' showed that there still appeared the same errors. Still not able to boot Ubuntu from the thing :(

oh, also I would like to add this, cuz maybe it could help find the problem or a solution to anyone who is more skilled than I am:
Installing the dmraid modul on my Ubuntu 10.04 setup (which is installed on the same computer, but on a 'normal' HDD, and which does NOT touch the SSDs) this system wouldn't boot either.
I had to take out the card, start the computer, uninstall dmraid, shut down the machine, re-plug the card and the system is now booting fine again.
This is pretty hard, cuz just installing dmraid will cause the system not to boot even though it does not touch the raid.

@bash.vi

The reason that dmraid will halt your system from booting is that the raid metadata will still be in your disks.
To stop dmraid from recognising your disks as a raid you have to purge this data from your disks.

Dmraid can only identify disks as in a raid set by this metadata.

I get these udevd error every-time I boot but it doesn't seem to effect my raid which has been running perfectly on 10.04, haven't tried 10.10 yet. The dm-xx nodes aren't even associated with dmraid so this bug is irrelevant from dmraid anyway.

The installer in 10.04 would leave you with a broken install if you let it do the partitioning, I manually create my partition using gparted then install using ubiquity. I'm not sure if ubiquity is fixed yet or not as I have fallen behind in my Ubuntu testing.

So I would not use the dmraid controller then and instead create a software raid by using the Linux capabilities for that? Did I get that right?
Think I can manage to do that.

But how can I get rid of the dmraid metadata on the disks?

I had this issue relating to my LVM volume (I don't have any dmraid setup).

I tried removing the dmraid package but this didn't help.

I then saw that installing 'cryptsetup' helps - http://ubuntuforums.org/showthread.php?t=1500937

So I installed 'cryptsetup' and the error went away. No idea why!

My english is bad, i have solution for this, in reality this isn't a bug, the problem comes from bios (i believe this is due from a old pc) you must go to bios and move from apic to PIC, for me this is impossible do this, because i have dual boot and windows does not boot if i do this, if someone can help me, for setting apic on system i am there be grateful.

you can write to <email address hidden>

Excume

My english is bad, i have solution for this, in reality this isn't a bug, the problem comes from bios (i believe this is due from a old pc) you must go to bios and move from apic to PIC, for me this is impossible do this, because i have dual boot and windows does not boot if i do this, if someone can help me, for setting apic to off, on my system i am there be grateful.

you can write to <email address hidden>

Some *possibly* better way to fix this error message :

* What I have done :

  - boot from the latest linux kernel version installed on my system (2.6.32-32)

* The highly probable reason why this fix the error boot message :

  - I, previously, upgraded both dmsetup + dmraid + libdmraid and linux kernel (to the latest version
    available for my original lucid 10.04 ubuntu install)

  - *BUT* I switched back grub to booting, by default, with original kernel version 2.6.32-21

  => So, my conclusion, is that there was a conflict between linux kernel (mainly dm-raid45 module ?)
        and dmraid tools / libs, leading to unpredictable situation

* Possible fix :

  - insure to have linux kernel and dmraid tools in compatible versions
    - eg. either original fresh installed versions
    - or all ones upgrade to latest possible version suitable for your ubuntu distribution

* Anyway, in my situation :

  - both have fixed all error message of type "inotify_add_watch ... no such directory"
  - *AND* the extra boot delay (traced by dmesg / syslogd)

* My software config :

  - Ubuntu 10.04 x64
  - Linux kernel 2.6.32-21 + 2.6.32-32
  - dmraid 1.0.0-rc16-4.1
  - libdmraid 1.0.0-rc16
  - dmsetup 2:1.02.39-1ubuntu4.1

* My hardware config :

  - AMD64 opteron
  - 1 internal PATA hdd
  - 1 faked RAID-0 of 2 SATA hdds with nvidia driver
  - 1 faked RAID-0 of 2 SATA hdds with jmicron driver

* Note :

  - I don't know if those solutions and my situation are exactly matching this bug report,
    but, if not, I'm sure my message will be redirected to a more suitable others bug report.
  - I hope not, because this was the top ranked bug report when I looked for
    "inotify_add_watch ... no such directory"

One whole year and the bug is still unassigned? Is there nobody in charge of the udevd module?
I have installed 10.04LTS server lately as a home server with LVM but no Raid as using Rsync is sufficient for me.

My system says there are no pending updates but nonetheless I do see the udevd-work bug too.

Linux homeserver 2.6.32-33-server #72-Ubuntu SMP Fri Jul 29 21:21:55 UTC 2011 x86_64 GNU/Linux

apt-get dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Same here:
$ uname -ra
Linux DELLka 3.2.0-17-generic #27-Ubuntu SMP Fri Feb 24 22:03:50 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

$ sudo fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b7be8

   Device Boot Start End Blocks Id System
/dev/sda1 * 2048 103407615 51702784 7 HPFS/NTFS/exFAT
/dev/sda2 103410466 976771071 436680303 5 Extended
/dev/sda5 103410468 794218495 345404014 7 HPFS/NTFS/exFAT
/dev/sda6 794220544 796317695 1048576 83 Linux
/dev/sda7 796319744 804708351 4194304 82 Linux swap / Solaris
/dev/sda8 804710400 859584511 27437056 83 Linux
/dev/sda9 859586560 976771071 58592256 83 Linux

Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... done.
fsck from util-linux 2.19.1
fsck from util-linux 2.19.1
fsck from util-linux 2.19.1
udevd[501]: inotify_add_watch(6, /dev/dm-0, 10) failed: No such file or directory
udevd[499]: inotify_add_watch(6, /dev/dm-0, 10) failed: No such file or directory

/dev/sda9: recovering journal
/dev/sda8: clean, 280432/1716960 files, 1993649/6859264 blocks
/dev/sda6 was not cleanly unmounted, check forced.
/dev/sda6: 433/524288 files (3.5% non-contiguous), 98864/1048576 blocks
mountall: fsck /boot [417] terminated with status 1
/dev/sda9: clean, 24343/3662848 files, 1944713/14648064 blocks

I have found this fault too with Ubuntu Server 12.04LTS. I had installed LVM2 non-raid (it is a VM anyway) for a 2nd drive to be mounted at '/home'.

The boot-time message I saw was:
udevd[252]: inotify_add_watch(6, /dev/dm-0, 10) failed: Invalid argument

As others have noted, installing cryptsetup makes the boot message (and delay) go away, but it is evidently a workaround not a real fix.

Same problem and solution as mentioned in the previous comment.

the solution in the previous comment does not work for me...

we are using ubuntu12.04 as xen dom0, and lvm as xen storage. when shutting down a domU we got these error in the syslog。and I found udev using the lvm LV device and prevent me from removing it some seconds afterward
--------------- /var/log/syslog ----------
Nov 15 21:34:09 e6 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/vif/4/0
Nov 15 21:34:12 e6 udevd[4964]: inotify_add_watch(6, /dev/dm-3, 10) failed: No such file or directory
Nov 15 21:34:12 e6 udevd[4964]: inotify_add_watch(6, /dev/dm-3, 10) failed: No such file or directory
Nov 15 21:34:12 e6 udevd[4972]: inotify_add_watch(6, /dev/dm-4, 10) failed: No such file or directory
Nov 15 21:34:12 e6 udevd[4972]: inotify_add_watch(6, /dev/dm-4, 10) failed: No such file or directory
Nov 15 21:34:12 e6 udevd[4972]: inotify_add_watch(6, /dev/dm-2, 10) failed: No such file or directory
Nov 15 21:34:12 e6 udevd[4972]: inotify_add_watch(6, /dev/dm-2, 10) failed: No such file or directory
---------------------------
I don't know why udev is atempting to watch the dm (LV) device.

I found in /lib/udev/rules.d/60-persistent-storage-dm.rules and edit out or comment the line :
-------------
OPTIONS+="watch"
-------------
it seemed the problem is gone, and I found no side-effect at the present

for the record, I am using udev-175-0ubuntu9.2

Neptune Ning's solution worked for me. Thank you !

I'm using Ubuntu12.04 with updated kernel 3.2.0-34-generic.
The system is installed by Ubuntu-12.04-alternate-amd64 ISO file with LVM partition:

root@Ubuntu1204:~# uname -a
Linux Ubuntu1204 3.2.0-34-generic #53-Ubuntu SMP Thu Nov 15 10:48:16 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

root@Ubuntu1204:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 14G 3.6G 9.7G 27% /
udev 2.0G 4.0K 2.0G 1% /dev
tmpfs 791M 720K 791M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 2.0G 144K 2.0G 1% /run/shm
/dev/mapper/LVM-local 50G 180M 47G 1% /local

Bootup error message:
udevd[254]: inotify_add_watch(6, /dev/dm-0, 10) failed: Invalid argument

I have tried "update-grub", "update-grub2" solutions,
and also Neptune Ning's solution by comment out the line (OPTIONS+="watch")
in "/lib/udev/rules.d/60-persistent-storage-dm.rules" file.

But I still see the line below during bootup:
udevd[254]: inotify_add_watch(6, /dev/dm-0, 10) failed: Invalid argument

Hi,

I am also getting the same, on a kvm hypervisor running Ubuntu 12.04 LTS :

Sep 6 01:19:31 ns5004890 kernel: [188914.076691] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null)
Sep 6 01:19:32 ns5004890 udevd[30252]: inotify_add_watch(6, /dev/dm-8, 10) failed: No such file or directory
Sep 6 01:19:32 ns5004890 udevd[30252]: inotify_add_watch(6, /dev/dm-8, 10) failed: No such file or directory
Sep 6 01:19:34 ns5004890 kernel: [188916.283025] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null)
Sep 6 01:19:34 ns5004890 udevd[30252]: inotify_add_watch(6, /dev/dm-8, 10) failed: No such file or directory
Sep 6 01:19:34 ns5004890 udevd[30252]: inotify_add_watch(6, /dev/dm-8, 10) failed: No such file or directory
Sep 6 01:19:36 ns5004890 kernel: [188918.465621] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null)
Sep 6 01:19:36 ns5004890 udevd[30252]: inotify_add_watch(6, /dev/dm-8, 10) failed: No such file or directory
Sep 6 01:19:36 ns5004890 udevd[30252]: inotify_add_watch(6, /dev/dm-8, 10) failed: No such file or directory
Sep 6 01:19:37 ns5004890 kernel: [188920.129626] device viif1007 entered promiscuous mode
Sep 6 01:19:38 ns5004890 kernel: [188920.191607] viifbr0: port 8(viif1007) entering forwarding state
Sep 6 01:19:38 ns5004890 kernel: [188920.191618] viifbr0: port 8(viif1007) entering forwarding state

Got this while reinstalling the operating system inside a vps.

I don't need cryptsetup on this. ...might be better to provide a fix for this issue. Ionotify should be working even without cryptsetup : i see no reason why it should be necessary to install it.

Although it's true : it really fix this issue!

I am also affected by this bug. At start up I get the message "udevd[135] inotify_add_watch (6, /dev/dm-0, 10) no such file or directory". I have a fake raid which is probably causing the problem. I also put this in my /etc/rc.local to discover and "activate" my raid disks: sudo dmraid -r, sudo dmraid -ay. This was when I first experienced this bug.

Hope this helps.

This version is now outdated and no more supported

From 14.04.3 LTS,

[ 2.643930] systemd-udevd[186]: inotify_add_watch(7, /dev/sda1, 10) failed: No such file or directory

Never fixed doesn't mean invalid.