exe thumbnailer eats all memory while thumbnailing large files

Yes, it's a problem. The best fix would be for wrestool to not have to load the entire executable to find the icon itself, although I'm not familiar enough with wrestool to know if it's possible. See: https://bugs.launchpad.net/ubuntu/+source/icoutils/+bug/614918

Until that's fixed, a workaround would be to respect the Nautilus preferences checkbox you noted.

This bug was fixed in the package gnome-exe-thumbnailer - 0.6-0ubuntu1

---------------
gnome-exe-thumbnailer (0.6-0ubuntu1) maverick; urgency=low

  * New version (thanks Jan Nekvasil)
    - use of %i (local file with full path) instead of %u
    - avoid dependency on python
    - only thumbnail local files (LP: #602767)
    - better version number extraction pipeline (LP: #622295)
    - enhanced text positioning with rounded corners in version labels
 -- Scott Ritchie <email address hidden> Fri, 17 Sep 2010 02:04:27 -0700

Still a problem in 0.7-0ubuntu1.

Not sure why this bug is mark Fix Released. It should be reopened with a High priority.

100% reproducible, can kill systems in a matter of seconds and result in data loss.

I had marked it fixed because there were substantial performance improvements in the new wrestool, so absent further reports of failure it seemed right to give it the benefit of the doubt.

Data loss is a curious thing here and shouldn't be happening with just exe-thumbnailer -- is your whole memory literally running out and processes are getting killed? If so that means Gnome is getting a bit reckless about spawning multiple thumbnailer instances simultaneously.

Oops - sorry I didn't reply. Just happened again, so I refound this bug report.

Yeah, data loss happens when memory runs out and I have to forcibly restart. Still a significant issue for me.

I ran into this again today a few times, and it pissed me off enough to create a workaround. Attached is a patch intended for /usr/bin/gnome-exe-thumbnailer.

It prevents wrestool from being called on files above a certain size (hardcoded as 100MB). This solved the issue for me, although a more general solution would be preferred.

I think there might be a better workaround.

1) When memory is low, don't spawn a parallel thumbnail process for every executable in the folder -- do them one at a time instead. This probably accounts for most of the cases of deadlock.
2) Have the thumbnailer script determine how many other instances of itself are running, and if their total memory is too high wait until one finishes. Ideally this would happen in some sort of intelligent order, but that probably requires code in Nautilus.
3) If even one at a time is too many (eg multi-gigabyte file), then blacklist it. Rather than 100 mb I would set this at something dynamic like 1/2 the system ram.

Oops - either LP didn't send me a bug mail or I missed it.

>>
I think there might be a better workaround.

1) When memory is low, don't spawn a parallel thumbnail process for every executable in the folder -- do them one at a time instead. This probably accounts for most of the cases of deadlock.
2) Have the thumbnailer script determine how many other instances of itself are running, and if their total memory is too high wait until one finishes. Ideally this would happen in some sort of intelligent order, but that probably requires code in Nautilus.<<

This would probably be a welcome improvement, but I think can be considered separately. My problem is with large files - one at a time or parallel, they still hose the system.

>> 3) If even one at a time is too many (eg multi-gigabyte file), then blacklist it. Rather than 100 mb I would set this at something dynamic like 1/2 the system ram.<<

I didn't update my description of the patch when I posted a new one. It now reads the nautilus thumbnail limit from gconf and respects that.

Should I try to upstream? Change the patch? I'd like to get this into Natty so I don't have to keep a local diff lying around.

If you're talking about gnome-exe-thumbnailer upstream, I'm basically it. I'm putting a new version in natty momentarily though, so you'll need to rediff.

Again, sorry for the late response. I'm not sure why LP isn't emailing me. I've tried manually subscribing now, so hopefully it will work.

I see no changes to the Natty version. My patch still applies cleanly. It may not be the best solution, but it works for me.

New version uploaded to Natty, once it builds please forward port the patch and I'll integrate it :)

Status changed to 'Confirmed' because the bug affects multiple users.

I've added code that makes the g-e-t respect the org.gnome.nautilus.preferences thumbnail-limit value, please test.

6 years later, and this is still a problem.

Besides wrestool, might want to take a look at the md5sum command that creates an icon based on the file md5sum when there's no icon embedded - this also creates a lot of disk activity and cached RAM. I hacked up a solution with changing a line to 'HUE=$(head -c 16384 "$INPUTFILE" | md5sum | cut -c 1-2)', too lazy to make a proper patch.

Using CRC instead of md5sum would probably be sufficient for this purpose.

This bug was fixed in the package gnome-exe-thumbnailer - 0.9.4-1

---------------
gnome-exe-thumbnailer (0.9.4-1) unstable; urgency=medium

  [ Jan Nekvasil ]
  * New upstream version
     - respect org.gnome.nautilus.preferences thumbnail-limit (LP: #602767)
     - thumbnail templates for Captiva, Compass, Evolvere, NITRUX-Buttons,
       Paper, Square-Beam, and Vibrancy icon themes

  [ James Lu ]
  * Update my email.
  * Bump standards version to 3.9.8 - no changes needed.
  * Update watch file and homepage to the new repository on GitHub.
  * debian/patches: add fallback-thumbnail-limit.patch to fix compatibility
    with Cinnamon (and possibly other GNOME-like desktops), when Nautilus
    isn't installed.

 -- James Lu <email address hidden> Sat, 12 Nov 2016 21:12:32 -0800