Displayed progress not correct due to kernel fs cache. Need O_SYNC when copying files.

Bug #30640 reported by magilus
60
This bug affects 5 people
Affects Status Importance Assigned to Milestone
nautilus (Ubuntu)
Triaged
Wishlist
Unassigned

Bug Description

While copying files, the progress bar is wrong in all cases since it is only reporting progress of the kernel to accept the writes into its cache. As a result, it reaches 100% before the copy is actually finished, and then appears to wait for an fsync before going away. Nautilus should use O_SYNC on the destination file so it will be informed of the progress properly.

Revision history for this message
Loic Pefferkorn (loic) wrote :

Are you using, in media:/, "safely remove" (approx translation), to sync files on the key ?

Revision history for this message
magilus (magilus) wrote :

No, I don't use it, when I use it it seems to work. But shouldn't files being copied directly? I think that this makes more sense..

Revision history for this message
magilus (magilus) wrote :

I just added Nautilus to the package list because it appears in Gnome, too but I do not really know which package causes this.

Revision history for this message
Daniel Holbach (dholbach) wrote :

I'm inclined to close this bug. You have to be sure sync has to be called before you unplug it. Unmount usually does this (either via GUI or in the terminal). Please try this and report back.

Revision history for this message
magilus (magilus) wrote :

The problem is that there is no progress bar while syncing, you don't know when Ubuntu is done syncing the files, thus there may be a loss of data when you unplug your usb device too early while it syncs in the background.

In the operating system of a company headquartered in Redmond they automatically sync while copying the files and show a correct progress bar. Maybe something similar to that would be better in Ubuntu.

Revision history for this message
magilus (magilus) wrote :

Since this is not a real bugeport anymore I will change the severity to "Wishlist".

Revision history for this message
Sebastien Bacher (seb128) wrote :

Rejecting the nautilus task, we already have bug #32643 about that

Changed in nautilus:
status: Unconfirmed → Rejected
Kenny Duffus (kduffus)
Changed in kdebase:
assignee: nobody → kubuntu-team
Revision history for this message
Rocco Stanzione (trappist) wrote :

Copying files on the command line behaves similarly. The operation is not complete until well after you've been returned to the command line. I know the moment it's done by umounting the device and waiting for *that* to complete. I believe this to be a kernel issue, not a desktop interface issue.

Revision history for this message
Rocco Stanzione (trappist) wrote :

That does seem to be the case - the copy operation considers itself to be finished once the data has been successfully copied to the filesystem buffer. We could leave the progress indicator up until a successful 'sync', I suppose.

Revision history for this message
Yuriy Kozlov (yuriy-kozlov) wrote :

I am closing this. You need to unmount (Safely Remove) your USB drive properly. In Gutsy and Hardy there is a progressbar when unmounting to make sure everything gets synced before the drive is unplugged.

Changed in kdebase:
status: New → Invalid
Revision history for this message
magilus (magilus) wrote :

Well the progress bar does not show any progre, it just bounces left and right. Ultimately, the kernel should be changed so that when cp finishes, all data has been copied I think. Not invalid I think.

Revision history for this message
Yuriy Kozlov (yuriy-kozlov) wrote :

I think you can get that effect by mounting with the sync option, but there are good reasons for not doing so.

Changed in linux:
assignee: nobody → ubuntu-kernel-team
importance: Undecided → Wishlist
status: New → Triaged
Revision history for this message
Leann Ogasawara (leannogasawara) wrote :

The Ubuntu Kernel Team is planning to move to the 2.6.27 kernel for the upcoming Intrepid Ibex 8.10 release. As a result, the kernel team would appreciate it if you could please test this newer 2.6.27 Ubuntu kernel. There are one of two ways you should be able to test:

1) If you are comfortable installing packages on your own, the linux-image-2.6.27-* package is currently available for you to install and test.

--or--

2) The upcoming Alpha5 for Intrepid Ibex 8.10 will contain this newer 2.6.27 Ubuntu kernel. Alpha5 is set to be released Thursday Sept 4. Please watch http://www.ubuntu.com/testing for Alpha5 to be announced. You should then be able to test via a LiveCD.

Please let us know immediately if this newer 2.6.27 kernel resolves the bug reported here or if the issue remains. More importantly, please open a new bug report for each new bug/regression introduced by the 2.6.27 kernel and tag the bug report with 'linux-2.6.27'. Also, please specifically note if the issue does or does not appear in the 2.6.26 kernel. Thanks again, we really appreicate your help and feedback.

Revision history for this message
Launchpad Janitor (janitor) wrote : Kernel team bugs

Per a decision made by the Ubuntu Kernel Team, bugs will longer be assigned to the ubuntu-kernel-team in Launchpad as part of the bug triage process. The ubuntu-kernel-team is being unassigned from this bug report. Refer to https://wiki.ubuntu.com/KernelTeamBugPolicies for more information. Thanks.

Revision history for this message
dorpm (dorpmueller) wrote : Re: displayed progress not correct when copying files to an usb stick

This effect appears not only when copying any data to or from an USB stick. I observe this behaviour also when I zip an archive. For me this is very annoying because the progress bar gives no information about when any operation is finished. For what is such a progress bar useful? Where can I get the information about finishing the copy or archiving process? At the moment I press F5 in Dolphin every few seconds to see if the filesize is still changing...

Florian

Revision history for this message
magilus (magilus) wrote :

please report a bug against dolphin..

Revision history for this message
Jeremy Foshee (jeremyfoshee) wrote :

This bug report was marked as Triaged a while ago but has not had any updated comments for quite some time. Please let us know if this issue remains in the current Ubuntu release, http://www.ubuntu.com/getubuntu/download . If the issue remains, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

tags: added: kj-triage
Changed in linux (Ubuntu):
status: Triaged → Incomplete
Revision history for this message
Jonathan Formentin (jonathan-formentin) wrote :

This bug is still occuring (Ubuntu 9.10, 64bits). The progress bar reaches 100% while there are still write access to the device, no matter the usb key brand and write speed.

Changed in nautilus (Ubuntu):
status: Invalid → New
affects: kdebase (Ubuntu) → ubuntu
Changed in ubuntu:
assignee: Kubuntu Bugs (kubuntu-bugs) → nobody
Phillip Susi (psusi)
Changed in nautilus (Ubuntu):
importance: Wishlist → Low
status: New → Triaged
summary: - displayed progress not correct when copying files to an usb stick
+ Displayed progress not correct due to kernel fs cache. Need O_SYNC when
+ copying files.
description: updated
Revision history for this message
Phillip Susi (psusi) wrote :

Actually now in Maverick when I try this, the copy reaches 100% then sits there for several seconds while it appears that an fsync is done to flush the cache. I think that solves the original problem, though it would be nice if the progress did not reach 100% prematurely and then wait for the sync.

Changed in linux (Ubuntu):
status: Incomplete → Invalid
Changed in nautilus (Ubuntu):
importance: Low → Wishlist
description: updated
Revision history for this message
Pietrek B. (ptrbrzozowski) wrote :

Still seeing this on 12.10 and 12.04.

This is very annoying, I just tried copying a 800MB ISO file to my Cruzer Fit 16GB USB stick. It does like 10MB/s max and despite that Nautilus indicated average transfer speed ~40MB/s, the progress bar went up to 99% in a couple of seconds and then just stalled while waiting for the buffer to be written. This is wrong and misleading at best. Having a progress indicator that behaves like this is just pointless IMO. Any fixes, guys?

magilus (magilus)
Changed in linux (Ubuntu):
status: Invalid → Confirmed
Revision history for this message
Phillip Susi (psusi) wrote :

This does not affect the kernel.

no longer affects: linux (Ubuntu)
Revision history for this message
magilus (magilus) wrote :

what does it affect then? as it also happens when doing this in the shell (cp...... is done even though data is not completely copied) I am wondering which package should be affected by this bug, too.

Phillip Susi (psusi)
no longer affects: ubuntu
Revision history for this message
Phillip Susi (psusi) wrote :

It affects nautilus, which is why it is tasked there.

Revision history for this message
magilus (magilus) wrote :

so, shall i also file a bug against cp? dont you think that there is a underlying issue there?

Revision history for this message
Phillip Susi (psusi) wrote :

No.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.