Fails with SystemError when too many files are open

I ran into this bug today while cleaning up my system for an upgrade to quantal (it actually dup'd me on bug #894314 since it was oneconf crashing). I was running: apt-get purge package_name; relatively quickly.

This seems like a minimal reproducer in python:

import apt

while True:
  apt_cache = apt.Cache()

It will eventually consume all of the file descriptors and crash with the above SystemError. I was not really successful in reproducing with straight apt_pkg, though apt_pkg.PackageRecords(self._cache) seems to be the line opening the files. It seems like the issue is that the objects are not being garbage collected fast enough. If,

gc.collect()

is run each loop it seems oscillate between 59 and 114 fds, instead of increasing rapidly until python runs out. Even better is:

del apt_cache._records

which oscillates between 3 and 59 fds. So a fix might be to add an explicit close() method to apt.Cache which deletes at least the _records object, and perhaps other objects.

On Oct 03, 2012, at 11:56 PM, Jason Conti wrote:

>import apt
>
>while True:
> apt_cache = apt.Cache()
>
>It will eventually consume all of the file descriptors and crash with
>the above SystemError. I was not really successful in reproducing with
>straight apt_pkg, though apt_pkg.PackageRecords(self._cache) seems to be
>the line opening the files. It seems like the issue is that the objects
>are not being garbage collected fast enough. If,
>
>gc.collect()
>
>is run each loop it seems oscillate between 59 and 114 fds, instead of
>increasing rapidly until python runs out. Even better is:
>
>del apt_cache._records
>
>which oscillates between 3 and 59 fds. So a fix might be to add an
>explicit close() method to apt.Cache which deletes at least the _records
>object, and perhaps other objects.

It's generally a bad idea to rely on garbage collection to free up external
resources like file descriptors. Much better to add an explicit API for
releasing those resources. So your above example would be better written like
so (if the API were available in pyapt):

    while True:
        apt_cache = apt.Cache()
        apt_cache.close()

Ideally, that would run forever.

Another perhaps useful idea, or at least another way of thinking about it,
would be to implement the context manager protocol, so you could do something
like this:

    while True:
        with apt.Cache() as cache:
            # do something with cache
            pass

and this would also guarantee to free up those file descriptors.

Thanks Barry, I added both of those to python-apt at:

lp:~jconti/python-apt/closeable-cache

with fixes for both gdebi and oneconf:

lp:~jconti/gdebi/closeable-cache
lp:~jconti/oneconf/closeable-cache

Stuck with del self._records to clean up the file descriptors since I took a look at the apt code for PackageRecords and it follows the usual C++ idiom of closing files on destruct.

While debugging the gdebi code I found another way to reproduce this specific bug, just continue Refreshing (ctrl+r) until the eventual SystemError, though I'm not sure how one would run into it in normal circumstances.

Doesn't help with the memory issues listed in bug #894314 but maybe oneconf can do some explicit garbage collection to keep memory reasonable.

Any news on this? It occurs in the following cases for me:
-when trying to upgrade to a newer Ubuntu release
-when trying to open software-properties-gtk
-when running "apt-get build-dep"

@ Alin: i had some similar error in ubiquity and managed to write a hotfix. maybe you can adapt that to gdebi. https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1063186

@Marcel: I've already tried that, it doesn't work in my case...

Status changed to 'Confirmed' because the bug affects multiple users.

Refering to the list of dublicates this issue makes troble within the dist-upgrade process with ubuntu-relese-upgrader., especially a fast increasing report number on the distribution upgrade from quantal2raring is observable.

In respect to this the taga

dist-upgrade and quantal2raring were added.

@Jason, your branches look good to me. Would you mind submitting a merge proposal for them? Thanks!

Come to find out this was fixed in python-apt in raring.
python-apt (0.8.8ubuntu1) raring; urgency=low

  [ Michael Vogt ]
  * merged from the debian-sid branch:
    - do not build-depend on python-unittest2, only needed for python2.6
  * merged from the debian-experimental branch
  * python/tag.cc:
    - make TagSecString_FromStringAndSize, TagSecString_FromString
      static, thanks to jcristau
  * python/cache.cc:
    - add "Codename" to PackageFile object
  * add dep8 style autopkgtest support
  * build fixes for python3.3
  * data/templates/Ubuntu.info.in:
    - add raring

  [ Jason Conti ]
  * lp:~jconti/python-apt/closeable-cache:
    - add apt.Cache.close() method

 -- Michael Vogt <email address hidden> Tue, 23 Oct 2012 12:58:13 +0200

What? it looks like this was submitted September 2012
almost a year ago..?
I have moved on ubuntu 13.04 or whatever...

But I still cannot get Wine to work well with Windows applications
like Slingbox. I cannot even use Slingbox via the web browsers under
Linux. I have to use Windows or Mac OS... which is fine with me.
Ubuntu is just for play anyway, nothing serious.

On Tue, Aug 27, 2013 at 4:06 PM, Brian Murray <email address hidden> wrote:

> @Jason, your branches look good to me. Would you mind submitting a
> merge proposal for them? Thanks!
>
> ** Also affects: python-apt (Ubuntu Saucy)
> Importance: Medium
> Status: Triaged
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1186115).
> https://bugs.launchpad.net/bugs/1051935
>
> Title:
> Fails with SystemError when too many files are open
>
> Status in “python-apt” package in Ubuntu:
> Triaged
> Status in “python-apt” source package in Saucy:
> Triaged
>
> Bug description:
> This error occurs each time I try to install a .deb file with Gdebi in
> Ubuntu 12.10, after the deb installation finishes.
>
> ProblemType: Crash
> DistroRelease: Ubuntu 12.10
> Package: gdebi 0.8.5build1
> ProcVersionSignature: Ubuntu 3.5.0-14.19-generic 3.5.3
> Uname: Linux 3.5.0-14-generic x86_64
> ApportVersion: 2.5.1-0ubuntu7
> Architecture: amd64
> CrashCounter: 1
> Date: Mon Sep 17 15:33:34 2012
> ExecutablePath: /usr/share/gdebi/gdebi-gtk
> InstallationMedia: Ubuntu 12.10 "Quantal Quetzal" - Alpha amd64
> (20120914)
> InterpreterPath: /usr/bin/python2.7
> PackageArchitecture: all
> ProcCmdline: /usr/bin/python /usr/bin/gdebi-gtk --non-interactive
> /home/andrei/Downloads/gnomishdark-theme_201208014-1~webupd8_all.deb
> ProcEnviron:
> PATH=(custom, no user)
> LANG=en_US.UTF-8
> SHELL=/bin/bash
> TERM=unknown
> PythonArgs: ['/usr/bin/gdebi-gtk', '--non-interactive',
> '/home/andrei/Downloads/gnomishdark-theme_201208014-1~webupd8_all.deb']
> SourcePackage: gdebi
> UpgradeStatus: No upgrade log present (probably fresh install)
> UserGroups:
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/1051935/+subscriptions
>

Status changed to 'Confirmed' because the bug affects multiple users.

This bug was fixed in the package oneconf - 0.3.5

---------------
oneconf (0.3.5) saucy; urgency=low

  * oneconf/distributor/Ubuntu.py: close the apt cache when we are done using
    it. Thanks to Jason Conti for the patch. (LP: #1051935)
 -- Brian Murray <email address hidden> Tue, 27 Aug 2013 16:26:54 -0700

Status changed to 'Confirmed' because the bug affects multiple users.

Fixed in 0.8.7.

This bug was fixed in the package ubiquity - 2.17.1

---------------
ubiquity (2.17.1) trusty; urgency=low

  [ Dmitrijs Ledkovs ]
  * Drop ntfsprogs alternative depends.
  * Merge packaging change to add debian-menu entry.
  * Tidy up autopilot tests warnings.
  * Move autopilot tests to lp:ubiquity and python3.
  * Bump to automake-1.14
  * Automatic update of included source packages: flash-kernel
    3.0~rc.4ubuntu44, grub-installer 1.78ubuntu9, netcfg 1.112ubuntu1,
    partman-base 169ubuntu1, partman-basicfilesystems 86ubuntu2,
    partman-efi 25ubuntu6, partman-ext3 80ubuntu1, yaboot-installer
    1.1.29ubuntu1.

  [ Brian Murray ]
  * Use the apt cache with with so that we close the cache when we are
    done with it (LP: #1051935)

  [ Dan Chapman ]
  * Autopilot tests update.
  * Autopilot tests for Edubuntu's extra install pages

  [ Jean-Baptiste Lallement ]
  * Join auto-pilot runner.
  * fix tests to be pep8 compliant
  * add directory autopilot/ to tests/run-pep8

  [ Huan Peng ]
  * Replace "Ubuntu" with the correct flavour name in several templates
    (LP: #1197220).
 -- Dmitrijs Ledkovs <email address hidden> Mon, 09 Dec 2013 16:05:50 +0000

Status changed to 'Confirmed' because the bug affects multiple users.

Hello Alin, or anyone else affected,

Accepted ubiquity into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/ubiquity/2.10.29 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Stéphane,

I don't use Precise any more so I for one can't test this...

It appears to have regressed ubiquity, since lp:~jconti/python-apt/closeable-cache is not present in precise.
I'm marking bug #1272272 as a duplicate of this one, but with intent that python-apt needs fixing.

Hello Alin, or anyone else affected,

Accepted python-apt into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/python-apt/0.8.3ubuntu7.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I ended up here while working on verification of the fix for bug #947107.

Since there is no clearly written test-case I've drawn some of my own conclusions:

(1) Any use of apt, certainly installation or upgrade of any packages, but also something as mundane as running "apt-cache policy <any package name>" prior to installation would cause ubiquity to crash with either the Precise 20140123 or 20140124 images.

(2) I can not reproduce that crash using the 20140125 image no matter how hard I try.

So I think I could mark this "verification-done" but there are so many duplicates I'd like to get some feed-back from Dimitri to be sure I'm not overlooking something.

Thanks, yes, 20140125 image should have all components in to verify all ubiquity SRU bugs.

verification-done precise, python-apt & ubiquity.

This bug was fixed in the package python-apt - 0.8.3ubuntu7.2

---------------
python-apt (0.8.3ubuntu7.2) precise; urgency=medium

  * Cherry-pick fixes from 0.8.8ubuntu1 to close apt.Cache and free
    file-descriptors. (LP: #1051935).
  * Update list of mirrors from launchpad, using pre-build.sh hook.
 -- Dimitri John Ledkov <email address hidden> Fri, 24 Jan 2014 12:19:45 +0000

The verification of the Stable Release Update for python-apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

This bug was fixed in the package ubiquity - 2.10.29

---------------
ubiquity (2.10.29) precise; urgency=low

  [ Dimitri John Ledkov ]
  * Show labels in automatic side-by-side installation. Fixed by removing
    all cairo styling from partition boxes in automatic partitioning,
    instead use simple bg styling. (LP: #947107, #1240532)
  * Automatic update of included source packages: grub-installer
    1.68ubuntu5.3.

  [ Brian Murray ]
  * Use the apt cache with with so that we close the cache when we are done
    with it. (LP: #1051935)

  [ Hao-Ran Liu (Joseph Liu) ]
  * src/wallpaper/wallpaper.c: Cropping wallpaper image instead of
    stretching it if image's aspect ratio doesn't match monitor's aspect
    ratio (LP: #1207249)

  [ Colin Watson ]
  * Update translations from Launchpad.
 -- Dimitri John Ledkov <email address hidden> Wed, 22 Jan 2014 14:35:56 +0000

also got that issue on Utopic i386, but cant report it directly as apport says "duplicate of lp:869704
 which is itself a dupe of that one (what a mess !!!), so im joining the crash file.

Just occurred in Trusty as well.

guest@ubuntu:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Vivid Vervet (development branch)
Release: 15.04
Codename: vivid
guest@ubuntu:~$

guest@ubuntu:~$ uname -mrs
Linux 3.18.0-11-generic x86_64
guest@ubuntu:~$

I got this while $ sudo apt-get dist-upgrade. On my Wubi based Ubuntu installtion.

Got this crash (or his brother #869704) today again with Ubuntu 15.04!

Same here on 15.04 64

Continues to happen on 15.04 64 bit, initially brought me to bug #869704 as well.

I'm not sure, but I think the changes in python-apt 1.0 could improve the situation, as I changed it to avoid reference cycles in the apt.Cache class (and related classes).