/usr/lib/auto-apt/auto-apt-pkgdcb consumes over 100% RAM on 0.3.23

Bug #1300987 reported by iw2snl
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
auto-apt (Ubuntu)
Fix Released
Undecided
Unassigned
Trusty
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
* 'sudo auto-apt update' fills the RAM, causing the system to freeze for a few minutes
  when the SWAP starts being used.
  - Due to this, auto-apt is unusable in Trusty (and Saucy).
  - This bug can only be reproduced after fixing bug #1300991 by applying the workaround
    provided in the end of that bug report by the reported.
* The new version 0.3.24 fixes this bug, LP: #1300991 and a dpkg warning about an
  obsolete argument.

[Test Case]
* First, apply the fix/workaround provided in the end of the bug report #1300991:
  Open /usr/bin/auto-apt and remove the first "[[:space]]" from line 145, so that it
  starts like 'sed -ne 's=^[[:space:]]*deb[[:space:]]'.
  Without the fix/workaround, the next command wouldn't even work!
* Run the command 'sudo auto-apt update'.
  - With version 0.3.23, RAM is quickly filled and the system freezes for a few minutes
    (until the oom killer terminates the process).
  - With the new version 0.3.24, RAM is not filled and the system doesn't freeze.

[Regression Potential]
* auto-apt is already non-functional in Trusty (and Saucy), therefore there is no
  regression potential.

[Other Info]
* The original bug report for Saucy follows:

1)Description: Ubuntu 13.10
Release: 13.10

Note: Actually Kubuntu 13.10

2)auto-apt:
  Installed: 0.3.23
  Candidate: 0.3.23
  Version table:
 *** 0.3.23 0
        500 http://md.archive.ubuntu.com/ubuntu/ saucy/universe amd64 Packages
        100 /var/lib/dpkg/status

3) $ sudo auto-apt update
Should update the database

4) Running the update command (hadn't verified updatedb) causes the system to freeze (seemingly).

I monitored the process tree with htop, during this command, and noticed, that /usr/lib/auto-apt/auto-apt-pkgdcb takes 100% CPU (doesn't seem to affect how responsive is the system) and like 60% of RAM. And it's constantly growing. Till it takes 100% of RAM, and then 100% SWAP. Then the process is automatically killed. But if you run the command from a graphical interface, you won't know it, because it seems like completely frozen.

Workaround: use /usr/lib/auto-apt/auto-apt-pkgdcb from auto-apt 0.3.22.
(works for me)

Notes regarding ways to reproduce in Ubuntu 13.10:
There is another bug, the sed line parsing sources.list has to trailing spaces (^[:[space]:][:[space]:]*deb), which cause update and updatedb to fail (no repository contents files are downloaded, so 0 entries are created).
Fixing that will make it possible to reproduce the bug, otherwise update commands won't even work.

Related branches

iw2snl (iw2snl)
summary: - /usr/lib/auto-apt/auto-apt-pkgdcb consumes over 100% RAM
+ /usr/lib/auto-apt/auto-apt-pkgdcb consumes over 100% RAM (only 0.3.23)
summary: - /usr/lib/auto-apt/auto-apt-pkgdcb consumes over 100% RAM (only 0.3.23)
+ /usr/lib/auto-apt/auto-apt-pkgdcb consumes over 100% RAM on 0.3.23
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in auto-apt (Ubuntu):
status: New → Confirmed
Revision history for this message
Bruno Nova (brunonova) wrote :

I confirm this bug.
Both "update" and "updatedb" commands have frozen my computer (due to full RAM), and the second command was actually killed by the out-of-memory killer (/usr/lib/auto-apt/auto-apt-pkgdcb is indeed the "guilty" program).

The workaround mentioned by the bug reporter also works for me.
Actually, after applying this workaround and the workaround mentioned in Bug #1300991, auto-apt works correctly.

For those who want to apply the workaround, one way to get the 0.3.22 version of the auto-apt-pkgdcb file is to download the 32 or 64 bit DEB of auto-apt of Raring from here: http://packages.ubuntu.com/raring/auto-apt
Then open the DEB in the archive manager, extract the auto-apt-pkgdcb file from its usr/lib/auto-apt directory and replace the system /usr/lib/auto-apt/auto-apt-pkgdcb file with this one.

Revision history for this message
Bruno Nova (brunonova) wrote :

I've looked at the changes done to file auto-apt-pkgcdb.c in this commit: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/trusty/auto-apt/trusty/revision/6#auto-apt-pkgcdb.c
There is clearly a memory leak in the "while" loop that starts in line 220.

The get_line() function allocates and returns a pointer to a character array (a new allocation for each invocation).
Inside the loop, in line 225, that function is called ("buf = get_line(stdin);"), and the pointer stored is in "buf".
However, the related free(buf) function is only called outside of the loop. So, only the last allocation is freed.
The free(buf) call must be moved to inside the "while" loop!

Revision history for this message
Bruno Nova (brunonova) wrote :

I've linked a branch that fixes this bug, bug #1300991 and the warning I mentioned in that bug.

There was indeed a memory leak.

Bruno Nova (brunonova)
tags: added: trusty
Bruno Nova (brunonova)
tags: added: saucy
Bruno Nova (brunonova)
description: updated
description: updated
Bruno Nova (brunonova)
Changed in auto-apt (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Bruno Nova (brunonova) wrote :

Ubuntu Sponsors, could you "backport" version 0.3.24 of auto-apt to trusty-proposed?
That version of the package fixes both this bug and LP: #1300991.
auto-apt is not functional in the current version 0.3.23.

Important note: 0.3.24 was uploaded to Debian and will, eventually, be synced into Utopic.
It is not yet available there.

Revision history for this message
Robie Basak (racb) wrote :

Uploaded to trusty-proposed. Unsubscribing sponsors. Now awaiting SRU team approval. Thank you for your work!

Changed in auto-apt (Ubuntu Trusty):
status: New → Triaged
Revision history for this message
Stéphane Graber (stgraber) wrote : Please test proposed package

Hello iw2snl, or anyone else affected,

Accepted auto-apt into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/auto-apt/0.3.23ubuntu0.14.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in auto-apt (Ubuntu Trusty):
status: Triaged → Fix Committed
tags: added: verification-needed
Revision history for this message
Bruno Nova (brunonova) wrote :

I've installed the updated auto-apt package, version 0.3.23ubuntu0.14.04.1 from trusty-proposed, in a Ubuntu 14.04 virtual machine (I'm still using 13.10).
This version seems to work correctly. The auto-apt commands mentioned in the description work fine and without filling the RAM.

I've also tested using auto-apt to "help me" install 'nano' from source ('auto-apt run ./configure').
It installed all required dependencies. After that, 'make' compiled the program without problems.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Colin Watson (cjwatson) wrote : Update Released

The verification of the Stable Release Update for auto-apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package auto-apt - 0.3.23ubuntu0.14.04.1

---------------
auto-apt (0.3.23ubuntu0.14.04.1) trusty; urgency=medium

  * No change rebuild from Debian auto-apt 0.3.24, which fixes only these bugs:
    - Patches supplied by Bruno Nova, with thanks.
    - Plug a memory leak caused by not freeing lines read from stdin. (LP:
      #1300987)
    - Use dpkg --print-architecture instead of deprecated dpkg
      --print-installation-architecture (the latter causes a warning). (LP:
      #1300991)
    - Accept no leading spaces at the start of a line in sources.list. (LP:
      #1300991)
 -- Robie Basak <email address hidden> Tue, 03 Jun 2014 13:32:35 +0000

Changed in auto-apt (Ubuntu Trusty):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.