Nautilus does not remove usb drive made with USB-Creator after unmounting it

Can you please make sure you have the latest usb-creator, then run sudo /usr/share/usb-creator/usb-creator-helper > /tmp/usb-creator-helper.log and usb-creator-gtk until the install finishes and incorrectly leaves the source ISO mounted. Attach /tmp/usb-creator-helper.log and ~/.usbcreator.log to this bug report.

Thanks!

Tried a 10.04 build from April 5th 2010. Still showing the issue.

I have tested and verified no issue here, using usb-creator-gtk version 0.2.21.

Can you run usb-creator-gtk -v from command line and verify it is actually version 0.2.21. If not, please update. If so, see below...

Also, can you run the mount command from the comand kline to verify that from the command kine you see the mounted loop device? I have seen issues with Ext4 and (for example) Dolphin under KDE not updating the directory view unless I forcably reload/refresh the current view. So, for example, I would see files which were no longer there. I suspect it to be an Ext4 issue with delayed writes (possibly). So, if you do not see the loop device running the mount command, then it's not an issue with usb-creator, but rather nautalis and/or Ext4.

It's something usb creator is doing. Maybe it's holding up dbus messages.

Just did a "fresh" install of the latest build:
http://cdimage.ubuntu.com/daily-live/20100406.1/

You don't even have the loop device in nautilus while usb creator is creating the usb stick. It's only after it is done does it show up and stay there. I have more pictures. Showing Nautilus and output from the "mount" command on the side.

Also this build is when runnig usb-creator-gtk --version is 0.2.21

And as I expected, you do not see the loop device anymore when issuing the mount command.

As I explained above, this is likely an issue with Nautilus and/or Ext4. Notice that the loop device never showed up immediately in Nautilus and was visible via the mount command. I'm not so sure this is a usb-creator issue.

If usb-creator creates a device and then removes it, the system should pick this up. This is all done as a popen call to the mount and umount command, and does not involve dbus at this point. So, perhaps the issue is around the filesystem, as we are dealing direclty with mount.

What happens if you issue the 'sync' command from a consle, or tell Nautilus to reload/refresh its view?

When I issue sync does nothing. Please try this out your self. This doesn't seem to be a Nautilus issue, but something else and usb-creator is triggering it.

Jerone,

Sorry for making you have to run another command. Unfortunately I cannot reproduce this myself and the suggestion I gave you before had a typo in it. Please open a terminal and run:
sudo /usr/share/usb-creator/usb-creator-helper > /tmp/usb-creator-helper.log 2>&1

Then start usb-creator-gtk. At the very least, you should have "DEBUG:root:Shutting down." in /tmp/usb-creator-helper.log when you're done. Attach both /tmp/usb-creator-helper.log and ~/.usbcreator.log. Just to save further run around, please also attach the output of `udisks --dump` and mount.

I suspect the dbus backend is throwing an exception while trying to unmount the loopback device.

Thanks for your patience and persistence.

Issue is still happing on fresh install of 10.04 RC.

DEBUG:root:['mount', '-o', 'loop', '/home/ubuntu/Desktop/lucid-desktop-i386.iso', '/tmp/tmpJdWnQQ']
...
DEBUG:root:['umount', dbus.String(u'/tmp/tmpJdWnQQ')]

Showing information for /org/freedesktop/UDisks/devices/loop0
  native-path: /sys/devices/virtual/block/loop0
  device: 7:0
  device-file: /dev/loop0
...
  is mounted: 0
...
  loop:
    filename: /home/ubuntu/Desktop/lucid-desktop-i386.iso

This would indicate that it's not mounted. Can you check against /proc/self/mounts to be sure?

@Evan
   Sorry for the long delay. Nope .. nothing in /proc/self/mounts

   I'm seeing this issue on every 10.04 machine that uses startup disk creator. You should easily be able to reproduce this issue locally.

This issue is still happening in Maverick builds as well. Tested with 6/24 build still happening.

Can someone re-confirm this bug under a recent version of lucid and maverick? I can not reproduce this bug anymore.

@Ayan
        Issue is still happening. Just tried my 10.04 box with all the latest updates. Not 100% sure how you are not able to reproduce.

1) Use usb creator
2) Try a 10.04 or 10.10 iso
3) Stick in usb stick
4) Wait while it installs iso to usb stick
5) When done .. check Nautilus .. you will see a device that is the ISO. It's a invalid device. You have to wait till the usb creator tool is completely done.

Issue still exists in 10.10. Was easily able to reproduce the issue with Aug 17 2010 build of 10.10.

I can confirm existence of this issue as well in 10.10. If I click on the "Ubuntu 10.10 i386" icon in Nautilus, I'm prompted for root password to mount it, when I enter my password, I receive an error message:
"Error mounting: mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       (could this be the IDE device where you in fact use
       ide-scsi so that sr0 or sda or so is needed?)
       In some cases useful info is found in syslog - try
       dmesg | tail or so"

Perhaps this is a bug in Nautilus, not usb-creator?

The core issue seems to be that 'mount -o loop' creates an udev event, but unmounting such a loop mount creates none, misleading Nautilus to offer an non-existent device to the user. The bug description is wrong anyway, as there is no loop device left after usb-creator.

See also https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/608390

Could you get a gvfs-mount -li log after unmount the device? and a udisks --dump one?

I'm not quite sure about the steps prior to unmounting the device in respect to information you request. 'gvfs-mount -li' and 'udisks --dump' don't report a loop device after

1. mkdir /tmp/tmpmount
2. sudo mount -o loop ubuntu-10.04-desktop-i386.iso /tmp/tmpmount/
3. sudo umount /tmp/tmpmount

They do falsely report a loop device after a run of usb-creator though, while 'losetup -a' clearly indicated that there is none. This is the relevant part of 'udisks --dump' with usb-creator running:

========================================================================
Showing information for /org/freedesktop/UDisks/devices/loop0
  native-path: /sys/devices/virtual/block/loop0
  device: 7:0
  device-file: /dev/loop0
    presentation: /dev/loop0
  detected at: Di 14 Sep 2010 11:42:16 CEST
  system internal: 1
  removable: 0
  has media: 1 (detected at Di 14 Sep 2010 11:42:16 CEST)
    detects change: 0
    detection by polling: 0
    detection inhibitable: 0
    detection inhibited: 0
  is read only: 0
  is mounted: 1
  mount paths: /tmp/tmpLMNiRK
  mounted by uid: 0
  presentation hide: 0
  presentation nopolicy: 1
  presentation name:
  presentation icon:
  size: 733419520
  block size: 512
  job underway: no
  usage: filesystem
  type: iso9660
  version:
  uuid:
  label: Ubuntu 10.04 LTS i386
  loop:
    filename: /home/ilja/ubuntu-10.04-desktop-i386.iso
  drive:
    vendor: Linux
    model: Loop: ubuntu-10.04-desktop-i386.iso
    revision:
    serial:
    WWN:
    detachable: 0
    can spindown: 0
    rotational media: Yes, unknown rate
    write-cache: unknown
    ejectable: 0
    adapter: Unknown
    ports:
    similar devices:
    media:
      compat:
    interface: (unknown)
    if speed: (unknown)
    ATA SMART: not available

========================================================================

And this is after usb-creator finished its job and was closed:

========================================================================
Showing information for /org/freedesktop/UDisks/devices/loop0
  native-path: /sys/devices/virtual/block/loop0
  device: 7:0
  device-file: /dev/loop0
    presentation: /dev/loop0
  detected at: Di 14 Sep 2010 11:42:16 CEST
  system internal: 1
  removable: 0
  has media: 1 (detected at Di 14 Sep 2010 11:42:16 CEST)
    detects change: 0
    detection by polling: 0
    dete...

Note: It might be that we need this kernel patch: http://lkml.org/lkml/2010/5/3/76

.. which was accepted upstream here: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=c3473c63542d53740f175f3a515257ae159e998b

@Martin
          The patch is now in 2.6.35-22 kernel. But even with it, problem still there.

I even run the latest -git kernel on top of maveric, and this problem exists.

Also having this problem in 10.10. I just created a USB startup disk and the image is still there.

Release noted for 10.10.

Now I'm seeing that GVolumeManager is emitting the wrong signals when a loopback device is mounted or unmounted -- it is emitting exactly the opposite signals it should.

First run this test program:
  https://code.launchpad.net/~ayan/+junk/gio-test

This is what I get:

$ ./gio-test &
$ sudo losetup /dev/loop0 /your/favorite.iso
$ sudo mount -t iso9660 /dev/loop0 /mnt
** (process:28520): DEBUG: received drive-disconnected event for '731 MB File'.
** (process:28520): DEBUG: received volume-removed event for 'Ubuntu 10.04 LTS amd64'.
$ sudo umount /mnt
** (process:28520): DEBUG: received drive-connected event for '731 MB File'.
** (process:28520): DEBUG: received volume-added event for 'Ubuntu 10.04 LTS amd64'.

Note that when you mount the device, we get a drive-disconnected and volume-removed event and when we unmount the device, we get a drive-connected and volume-added event.

But when I mount other devices, gio-test displays the expected output.

I think the bug is in libgdu. If the event custody chain is:

  kernel uevent -> udev -> udev uevent -> (udisks sends D-BUS signal) -> (libgdu + gvfs)

We can test udisk with:

$ udisks --monitor-detail

You'll see that the "is mounted" flag is set properly when you mount and unmount a loop device.

You can test gvfs with:

$ gvfs-mount -io

And you'll see the events are backwards there.

Running Ubuntu 10.10 Nautilus is sleeping when using the Places bookmarks. Lunch the folder and nothing happens

I used usb-creator yesterday to try ubuntu 11.04-beta1 (32bit). After successful creation of the bootable device I unmounted the volume and even chose safely remove after.
After I removed the USB stick, Nautilus still displays /dev/loop0 as File 732MB (ubuntu-11.04-beta1-i386.iso).

Trying ti mount it then gives this fs driver error:
Error mounting: mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       (could this be the IDE device where you in fact use
       ide-scsi so that sr0 or sda or so is needed?)
       In some cases useful info is found in syslog - try
       dmesg | tail or so

Workaround: checking it with gnome-disk-utility 2.30.1-1 returns the volume as not safe, but makes it disappear at last.

Up-to-date Lucid 64bit here.
usb-creator-gtk 0.2.22.1

Reproduced today after using usb-creator to test an ubuntu-based distro (elementary 0.1, based on maverick 32bit).

Request formatting of the ejected volume, listed in nautilus with the distro name, prompts an error message
Error creating partition table: helper exited with exit code 1: BLKRRPART ioctl failed for /dev/loop0: Invalid argument
but the ghost volume then disappears.
So this is another, if less clean, workaround.

bug seems to be reproducible at will

If I mount/unmount the loop device in nautilus (i. e. through gvfs), it seems to work well:

Volume changed: 'Ubuntu 11.04 amd64'
  Volume(0): Ubuntu 11.04 amd64
    Type: GProxyVolume (GProxyVolumeMonitorGdu)
    ids:
     unix-device: '/dev/loop0'
     label: 'Ubuntu 11.04 amd64'
    themed icons: [drive-removable-media-file] [drive-removable-media] [drive-removable] [drive]
    can_mount=1
    can_eject=0
    should_automount=0

Mount added: 'Ubuntu 11.04 amd64'
  Mount(0): Ubuntu 11.04 amd64 -> file:///media/Ubuntu%2011.04%20amd64
    Type: GProxyMount (GProxyVolumeMonitorGdu)
    default_location=file:///media/Ubuntu%2011.04%20amd64
    themed icons: [drive-removable-media-file] [drive-removable-media] [drive-removable] [drive]
    x_content_types: x-content/win32-software
    can_unmount=1
    can_eject=0
    is_shadowed=0

and the drive stays around. But if I do the same using mount/unmount on the command line, the drive and volume both go away, and gvfs does not know about the loop mount at all if I mount to /mnt. When mounting to /media/, it does work, though. This is desired behaviour, as otherwise the UI would be cluttered with loop mounts which are "system internal" (such as /mnt or temporary mounts from usb-creator).

I tried to reproduce this with

sudo losetup /dev/loop0 download/ubuntu/natty-desktop-amd64.iso

-> Volume/Drive created and appears in palimpsest and nautilus

sudo mount /dev/loop0 /mnt

-> Volume/Drive disappear in nautilus, as it's hidden from user sessions (not in /media); stays in palimpsest

sudo umount /mnt

-> Volume/Drive reappear in nautilus, as it's now umounted; still stays around in palimpsest

sudo losetup -d

-> volume/drive go away in nautilus/palimpsest

So it's something more specific to what usb-creator does. Need to investigate that in more detail then (debug cycles are quite long with a full usb-creator write turnaround).

Not important enough for an SRU.

I can still reproduce the bug under Maverick.

If one mounts a loopback filesystem under /media, things work fine.

Examining the gvfs code, it appears to explicitly ignores udisks signals for mounts outside of /media.

See _g_unix_mount_point_guess_should_display() @ line 631 of gvfs-1.6.4/monitor/gdu/ggduvolumemonitor.c -- notice that this function is used in should_volume_be_ignored() and should_drive_be_ignored().

I've tried a patch to usb-creator that uses udisks --mount to mount the temp drive under /media and it works fine under Maverick.

I'm still not familiar with all of the intricacies of how the mount and device events get propagated to nautilus. Please correct me if I'm wrong Martin.

Ayan George [2011-04-10 19:52 -0000]:
> I've tried a patch to usb-creator that uses udisks --mount to mount the
> temp drive under /media and it works fine under Maverick.

That might be a pointer to where the problem actually is, but I don't
think we should do this. As this is an internal helper mount, it
shouldn't be exposed to users in nautilus at all.

Ayan and I investigated this, and found out an interesting thing: When I explicitly use losetup/mount/umount/losetup -d as in comment 40, the aforementioned kernel patch [1] takes care to send a "change" uevent when the losetup -d happens.

However, when I use the approach that usb-creator uses, things still go wrong:

    sudo mount -o loop ~/download/ubuntu/natty-desktop-amd64.iso /mnt

This sends a "change" event on loop0 which causes udev/udisks/etc. to re-read the device, and see that there's an iso9660 file system on it.

    sudo umount /mnt

This does NOT send a change event on loop0, although it does tear down the loop device. So after that, udev/udisks (and thus the gvfs stack) still think that /dev/loop0 is a mountable iso9660 file system.

A workaround would be to poke the loop device in the usb creator backend:

     def UnmountFile(self, device, sender=None, conn=None):
         popen(['umount', device])
+ popen(['udevadm', 'trigger', '--verbose', '--sysname-match=loop0'])

Ayan is currently testing this.

But the proper bug fix would be to either fix umount to tear down loop0 "more properly" so that the kernel patch kicks in, or fix the kernel to also send a change event on the missing case that umount triggers if it unmounts a loop device that was mounted with "-o loop".

[1] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=c3473c63542d53740f175f3a515257ae159e998b

Sorry, in above patch the --verbose should be dropped, that was just for debugging.

I straced umount, and found that it actually does not do anything with the loop device itself, or calls losetup. I suppose what happens is that the kernel itself tears down the loop device when you umount() a device which was mounted with -o loop (I don't see a mount option in /proc/mounts which would indicate this, so there must be some internal housekeeping flag?). I guess this code path is simply missing the kobject_uevent() call.

Unfortunately the trick with the udevadm trigger doesn't work around the problem.

The patch doesn't quite fix the problem because it only generates the event when bdev != NULL, but the kernel auto detach machinery passes NULL for the bdev in loop.c:1525. In util-linux mount.c:1270 checks if the kernel version is >= 2.6.37 and if it is, enables SETLOOP_AUTOCLEAR which causes the kernel to auto detach instead of umount.

Fixed it. Attaching simple one line patch that seems to work for me on Natty.

That indeed looks straightforward, thanks! Kernel team, could we get that upstream and into oneiric?

Chris, why did you drop the status from Triaged to Confirmed?

@Phillip -- Fixed that up, was doing some batch updates across other projects.

Cheers,
Chris

The parameters taken by loop_clr_fd() are confusing -- the block device isn't necessary since struct loop_device has a block device member (lo_device) that gets set when loop_set_fd() is called. An instance of a struct loop_device is the only thing required to clear the fd.

Perhaps the attached patch fixes the API issue as well as the bug?

The patch to fix the issue currently is queued in linux-next:

commit 23e2632cc815fb6cc38560805a12be586e23a8f1
Author: Phillip Susi <email address hidden>
Date: Sat Jul 16 23:31:06 2011 +1000

    Fix the loopback device to emit a uevent on auto release. The loopback
    driver failed to emit the change uevent when auto releasing the device.
    Fix lo_release() to pass the bdev to loop_clr_fd() so it can emit the
    event.

    Cc: Jens Axboe <email address hidden>
    Signed-off-by: Phillip Susi <email address hidden>
    Signed-off-by: Andrew Morton <email address hidden>

As the merge window has opened, it should likely get into 3.1 from where we could pick it. I am no sure Ayan's patch would make it (sure one could try to send it upstream (after removing the unnecessary whiespace change)) upstream. From the lo_ioctl side bdev is the known (passed in argument) while lo is not. Theoretically clear could be called before set and that may or may not be checked for under all circumstances.

Patches made it into -mm branch this weekend.

The attachment "fixloop.patch" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

Commits 4c823cc and 8a9c594 in the current mainline linux kernel should fix this issue. I will work on backporting and testito lucid and maverick

Err -- I'll work on backporting and testing on lucid and maverick.

This bug is awaiting verification that the kernel for Oneiric in -proposed solves the problem (3.0.0-16.28). Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-oneiric' to 'verification-done-oneiric'.

If verification is not done by one week from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Verified fixed with linux-image-3.0.0-16-generic rev. 3.0.0-16.28 on Xubuntu 11.10. Thunar exposed the underlying issue with usb-creator-gtk and earlier kernels the same way Nautilus did. 3.0.0-16.28 fixes it, thank you.

This bug was fixed in the package linux - 3.0.0-16.28

---------------
linux (3.0.0-16.28) oneiric-proposed; urgency=low

  [Herton R. Krzesinski]

  * Release Tracking Bug
    - LP: #922692

  [ Upstream Kernel Changes ]

  * Revert "drm/i915/dp: Fix the math in intel_dp_link_required"
    - LP: #919350

linux (3.0.0-16.27) oneiric-proposed; urgency=low

  [Brad Figg]

  * Release Tracking Bug
    - LP: #920735

  [ Paolo Pisati ]

  * Revert "SAUCE: omap3: beagle: if rev unknown, assume xM revision C"
    - LP: #912199
  * Revert "SAUCE: omap3: beagle: detect new xM revision C"
    - LP: #912199
  * Revert "SAUCE: omap3: beagle: detect new xM revision B"
    - LP: #912199
  * Revert "SAUCE: omap3: beaglexm: fix DVI initialization"
    - LP: #912199
  * [Config] DEFAULT_MMAP_MIN_ADDR=32k on arm
    - LP: #903346

  [ Upstream Kernel Changes ]

  * Revert "rtc: Disable the alarm in the hardware"
    - LP: #913373
  * Support for Terratec G1
    - LP: #821061
  * drm/radeon/kms: fix DP detect and EDID fetch for DP bridges
    - LP: #825777
  * drm/radeon/kms/DCE4.1: fix Select_CrtcSource EncodeMode setting for DP
    bridges (v2)
    - LP: #825777
  * drm/radeon/kms: cleanup atombios_adjust_pll()
    - LP: #825777
  * drm/radeon/kms/atom: rework encoder dpms
    - LP: #825777
  * drm/radeon/kms: check for DP MST mode in a few more places (v2)
    - LP: #825777
  * drm/radeon/kms: rework DP bridge checks
    - LP: #825777
  * drm/radeon/kms: fix DP setup on TRAVIS bridges
    - LP: #825777
  * ALSA: sis7019 - give slow codecs more time to reset
    - LP: #907778
  * ALSA: hda/realtek - Fix Oops in alc_mux_select()
    - LP: #907778
  * alarmtimers: Fix time comparison
    - LP: #907778
  * ARM: davinci: da850 evm: change audio edma event queue to EVENTQ_0
    - LP: #907778
  * arm: mx23: recognise stmp378x as mx23
    - LP: #907778
  * ARM: at91: fix clock conid for atmel_tcb.1 on 9260/9g20
    - LP: #907778
  * ARM: davinci: dm646x evm: wrong register used in
    setup_vpif_input_channel_mode
    - LP: #907778
  * ASoC: Provide a more complete DMA driver stub
    - LP: #907778
  * fs/proc/meminfo.c: fix compilation error
    - LP: #907778
  * thp: add compound tail page _mapcount when mapped
    - LP: #907778
  * thp: set compound tail page _count to zero
    - LP: #907778
  * ptp: Fix clock_getres() implementation
    - LP: #907778
  * mm: Ensure that pfn_valid() is called once per pageblock when reserving
    pageblocks
    - LP: #907778
  * mm: vmalloc: check for page allocation failure before vmlist insertion
    - LP: #907778
  * fix apparmor dereferencing potentially freed dentry, sanitize
    __d_path() API
    - LP: #907778
  * target: Handle 0 correctly in transport_get_sectors_6()
    - LP: #907778
  * intel-iommu: fix return value of iommu_unmap() API
    - LP: #907778
  * intel-iommu: set iommu_superpage on VM domains to lowest common
    denominator
    - LP: #907778
  * intel-iommu: fix superpage support in pfn_to_dma_pte()
    - LP: #907778
  * percpu: fix chunk range calculation
    - LP: #907778
  * iwlwifi: do not re-configure HT40 after associated
    - LP: #907778
  * mac80211: fix race condition caused by late addBA respon...