Slow file dialogs, open and save

Status changed to 'Confirmed' because the bug affects multiple users.

Yes, please could the fix be included on Ubuntu 20.04 ?

I created a patch to 1.44.1-1ubuntu1 from the merge request on GNOME's gvfs gitlab (https://gitlab.gnome.org/GNOME/gvfs/-/merge_requests/96) commit dc21a0948bcbe8a6d79d674bd1e4d63ded57d340 (https://gitlab.gnome.org/GNOME/gvfs/-/merge_requests/96/diffs?commit_id=dc21a0948bcbe8a6d79d674bd1e4d63ded57d340)

I rebuilt the various gvfs debs, installed them from a local deb repository, and it does seem to fix the issue (it was happening quite frequently for me on Mint). I'll confirm in a few days that I haven't had the issue again.

Hopefully this is helpful in getting this fixed.

The attachment "gvfs.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

Just a quick confirmation that with the patch applied, I've had no gvfsd-trash deadlocks requiring a pkill in over a week. It was previously hourly or worse.

The debdiff here looks fine, although the debian/changelog should reference this bug report using the syntax (LP: #1927100). Because this fix is for a stable release of Ubuntu we need to follow the Stable Release Update policy (https://wiki.ubuntu.com/StableReleaseUpdates). Could you please add SRU information to the bug description? Thanks in advance!

Sure, I'll try to take care of that soon.

I was also a little unclear on what to do about the version number--I forced something non-standard (1.44.1-1ubuntu1-test) so that it would locally supersede the current ubuntu version but not block updates. Should the debdiff reflect what would be the next appropriately numbered package version? Or is that taken care of later?

Sorry for the probably-trivial question, I haven't done this before

I would have used a version number of 1.44.1-1ubuntu2~ppa1 as this is greater than 1.44.1-1ubuntu1 and less than 1.44.1-1ubuntu2.

$ dpkg --compare-versions 1.44.1-1ubuntu2~ppa1 lt 1.44.1-1ubuntu2; echo $?
0

$ dpkg --compare-versions 1.44.1-1ubuntu2~ppa1 gt 1.44.1-1ubuntu1; echo $?
0

This is easy for a sponsor (me) to change though, so you don't need to go through the effort of creating a new debdiff.

In the attached patch, I've edited debian/changelog to include the LP bug # as requested. I also went ahead and added some description/reference links to the patch itself--I ended up creating a new debdiff to do this since I wasn't sure how to manually update line numbers in a way that would be safe. I renamed the original patch to 1-gvfs for clarity. The substance of the patch is unchanged.

I'll update the bug description momentarily. Sorry for the delay!

The bug description is now updated. Apologies for anything I missed!

I've uploaded this now, thanks!

 $ dput gvfs_1.44.1-1ubuntu1.1_source.changes
Trying to upload package to ubuntu
Checking signature on .changes
gpg: /tmp/pkgs/focal/gvfs_1.44.1-1ubuntu1.1_source.changes: Valid signature from 1E918B66765B3E31
Checking signature on .dsc
gpg: /tmp/pkgs/focal/gvfs_1.44.1-1ubuntu1.1.dsc: Valid signature from 1E918B66765B3E31
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading gvfs_1.44.1-1ubuntu1.1.dsc: done.
  Uploading gvfs_1.44.1-1ubuntu1.1.debian.tar.xz: done.
  Uploading gvfs_1.44.1-1ubuntu1.1_source.buildinfo: done.
  Uploading gvfs_1.44.1-1ubuntu1.1_source.changes: done.
Successfully uploaded packages.

The patch and upload look good. Thanks!

> To my knowledge, there is not a 100% reproducer for gvfs from userspace as this is a probable lock-ordering issue in lower-level code. https://gitlab.gnome.org/GNOME/glib/-/issues/1941 contains a reproducer for the underlying glib issue.

I appreciate the difficulty, and we can be pragmatic about not requiring 100% verification. But then what exactly is the test plan? How do you propose to verify that the issue is at least _likely_ to be fixed before we release the fix into focal-updates? What steps, if any, are you proposing to take, exactly? Are you proposing not to test it at all? Or just to test that basic functionality in the affected areas still work, and if so, what would that cover exactly? Or are you going to attempt to trigger the race and if unsuccessful some number of times then call it good? Or some combination? I'd like for expectations to be clear, and the plan agreed, before accepting this into focal-proposed.

Once a test plan (or lack thereof) is agreed, then +1 to accept this into focal-proposed.

I can tell you anecdotally that I went from pkill-ing gvfsd-trash hourly or worse, to not once since I applied the patch locally in October. But even after all that, I'm not really sure what behavior of mine was triggering the lockup so frequently.

Is an anecdotal user-report like that enough? (obviously using -proposed instead of my local deb repo) FWIW, when committing this change to gvfs, the gnome folks agreed with this comment by their bug's original poster
"I think we should just merge this workaround, it's such an intermittent issue and we know exactly why it happens in glib, and it's low risk. We can't really test it any "better" than merging it and see if the issue ever happens again."

If you'd prefer more, I can rollback from my local deb repo and see if there's any way to passively detect/log the issue (i.e. rather than just noticing it incidentally when a file dialog doesn't appear quickly). I guess the goal there would just be to say the issue occurred X times in Y hours on 1.44.1-1ubuntu1, and zero times in Z hours on -proposed.

I should also note that while the original poster of this bug uses/used Ubuntu Mate 20.04, my own daily driver is actually Mint...so you'd have to judge whether I would be an acceptable tester for this

edit: some googling suggests that 'gio info trash:///' should hang the same way as file dialogs (source: https://forums.linuxmint.com/viewtopic.php?p=1876910&sid=6bb2886d4c755c55a4862eb7add064b9#p1876910). maybe a script that calls that in a loop and logs any time it takes longer than 10s to return would work as a passive test?

> I guess the goal there would just be to say the issue occurred X times in Y hours on 1.44.1-1ubuntu1, and zero times in Z hours on -proposed.

I think something like that would be fine, yes.

I'm not sure about using Mint for the test though. Are there any direct users of Ubuntu who could test in this fashion? If no direct Ubuntu users appear to be affected, then the race could be manifesting due to some difference in Mint, at which point there'd be limited benefit for the regression risk cost of making a change for direct Ubuntu users in a stable release. But if direct Ubuntu users _are_ affected, then I would expect at least one of them to be able to step up and help us test.

I've seen the same issue reported in gnome trackers under Nautilus+Fedora and Thunar+Arch, so I'm fairly sure it's not a Mint-specific bug (though those are obviously not Ubuntu-based).

The original report here was for Ubuntu Mate...that's "direct" Ubuntu, right? If there's anyone out there, I'd be happy to write a script for verification

Is there nobody using Ubuntu itself who will be able to help test at all?

Is anyone willing to fire up an Ubuntu live image to help test it there? If so, maybe it's worth seeing if the race reproduces in a live environment on Ubuntu first?

I created an (installed) VM of Ubuntu Mate, since that was the original report.

My main affected workstation sees long uptimes, many transient podman containers and fuse mounts, temporary files, etc. that are not easily duplicated in a VM--I'm not sure if all/any of that contributes to the hanging. I can at least leave the VM running with a) files in the trash, and b) `gio info trash:///` running in a loop to see if it reproduces at some point.

I haven't seen the issue occur in an (idle) Ubuntu Mate VM after a week or so. To be fair, I can't reproduce it in a clean Mint VM either. There's a user in the Mint/Nemo issue tracker that was seeing the issue using Ubuntu+Cinnamon, but they've since upgraded to 21.10 (with newer gvfs) and are no longer affected.

Not sure this will be verifiable without a user to test in their active/natural install--I can reproduce it readily on my workstation, but I'm really not sure which aspect of my setup/usage makes it occur.

Thank you for working with us to try and get a test plan for Ubuntu together.

How about this:

1) I'll accept the package (1.44.1-1ubuntu1.1) into focal-proposed.
2) You test using your Ubuntu VM as best as you can to ensure that the package hasn't regressed for Ubuntu users.
3) You verify the package fixes your race on Mint as pragmatically you've found no practical way to test on Ubuntu.
4) If we're good on both verifications, I'll publish the update to focal-updates.

Does this plan work for you?

This sounds fine to me--sorry that it has ended up this way, but thanks for offering this compromise.

Let me just first confirm that I can still reproduce it on my Mint install--I've been running from my locally-patched gvfs since October, so I should be sure it hasn't changed with any subsequent Mint/Nemo updates. I'll report back after that.

Hello sam, or anyone else affected,

Accepted gvfs into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gvfs/1.44.1-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

I've accepted this into -proposed on the basis that it'll make it possible for you to verify, and it's easy to reject if it turns out you can't verify it its easy to remove from proposed.

All autopkgtests for the newly accepted gvfs (1.44.1-1ubuntu1.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

gvfs/1.44.1-1ubuntu1.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#gvfs

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

I've had the fix for focal installed since yesterday and it seems to have fixed the bug without any adverse effects

Apologies for the delay, was just finally able to confirm yesterday that the bug still exists on unpatched-gvfs (my container/transient fuse mount workflow hadn't been active for a while, and that must've been a critical part of reproducing the bug for me)

Moving onto -proposed shortly

Confirming that
1. I noticed no regression in clean, fully-updated Ubuntu VMs over 48h (periodic file browser operations and open-file dialogs in gtk apps, looped `gio info trash:///` in terminal). Tested standard Ubuntu and Ubuntu MATE (per the original report here).
2. I did NOT experience the bug in Mint on -proposed under the same conditions that caused it on unpatched gvfs over the prior handful of days. I again noticed no regression

Robie--I think this covers the proposal that you outlined, so I'll retag as verification-done-focal per the instructions in Chris's comment. Hopefully that's the correct action

Thank you for taking the time to do this!

This bug was fixed in the package gvfs - 1.44.1-1ubuntu1.1

---------------
gvfs (1.44.1-1ubuntu1.1) focal; urgency=medium

  * debian/patches/fix-trashd-hang.patch: avoids deadlock in trashd that can
    cause gtk file dialogs to hang system-wide (LP: #1927100)

 -- David Buckmaster <email address hidden> Tue, 21 Dec 2021 01:04:36 -0500

The verification of the Stable Release Update for gvfs has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.