Ubuntu
apt package

"Building dependency tree" is very slow with compressed apt indexes

Bug #676790 reported by Scott Moser
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
apt (Ubuntu)
Invalid
Low
Unassigned

Bug Description

Binary package hint: apt

I was looking at uec build logs (http://uec-images..ubuntu.com), and noticed a several orders of magnitude regression in speed between '2010-11-15 01:47:01' (20101115 build) and '2010-11-16 01:31:52' (20101116 build).

The uec builds are basically a 'debootstrap' build with additional things done after bootstrap. One of the additional things is a fairly large 'apt-get install' (~250 packages)
  apt-get install -y --force-yes landscape-client 'standard^' 'uec^' 'linux-virtual'

In the 20101115 build, the 'Building dependency tree...' step of apt changed from taking something less than 10 seconds (usually about 3) to over 1 hour and 30 minutes.

I can produce build logs if necessary. I've verified on a separate system, and the nightly builds continue to show this, so its not a build machine configuration change.

I think this roughly correponds with the upload time of apt '0.8.8ubuntu3'.

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: apt 0.8.8ubuntu3
ProcVersionSignature: Ubuntu 2.6.35-22.35-generic 2.6.35.4
Uname: Linux 2.6.35-22-generic x86_64
Architecture: amd64
Date: Wed Nov 17 21:04:39 2010
EcryptfsInUse: Yes
InstallationMedia: Ubuntu 10.04 "Lucid Lynx" - Beta amd64 (20100318)
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: apt

Revision history for this message
Scott Moser (smoser) wrote :
Revision history for this message
Michael Vogt (mvo) wrote :

I'm pretty sure its the "use-compressed-indexes" patch.

Changed in apt (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Martin Pitt (pitti)
Changed in apt (Ubuntu):
assignee: nobody → Martin Pitt (pitti)
status: Confirmed → In Progress
Revision history for this message
Michael Vogt (mvo) wrote :

This affects software-center in various ways too, scrolling becomes slow in the channel subviews. There were also some startup speed oddness issues, but that might be something else.

Martin Pitt (pitti)
tags: added: apt-compressed-indexes
summary: - huge performance regression
+ "Building dependency tree" is very slow with compressed apt indexes
Revision history for this message
Martin Pitt (pitti) wrote :

Scott, if you could attach a complete build log that would be helpful. I'd like to reproduce this locally in a chroot, and seeing which steps are taken would help. Thanks!

Revision history for this message
Martin Pitt (pitti) wrote :

This gets mitigated by disabling compressed apt index by default again for now. It's easy enough to reenable locally for reproducing the bug, so lowering the severity.

apt (0.8.9ubuntu4) natty; urgency=low

  [ Michael Vogt ]
  * cmdline/apt-key:
    - set timeout of wget for net-update to 90 seconds (thanks to \sh)

  [ Martin Pitt ]
  * Revert r1819 and r1820 to disable compressed indexes by default again.
    Testing has brought up a few places where this seriously degrades
    performance, mostly in applications which iterate through all available
    package records, like update-apt-xapian-index or synaptic. See
    https://bugs.launchpad.net/ubuntu/+bugs?field.tag=apt-compressed-indexes

 -- Martin Pitt <email address hidden> Thu, 25 Nov 2010 08:50:37 +0100

Changed in apt (Ubuntu):
importance: High → Low
status: In Progress → Triaged
Revision history for this message
Scott Moser (smoser) wrote :

In this log, look at lines 843 and 3768 for i386 and amd64 large package set installs respectively.

There, each arch build does a:
  chroot /path/to/target apt-get install -y --force-yes \
     landscape-client 'standard^' 'uec^' linux-virtual

Martin Pitt (pitti)
tags: added: pet-bug
Martin Pitt (pitti)
Changed in apt (Ubuntu):
assignee: Martin Pitt (pitti) → nobody
Revision history for this message
dino99 (9d9) wrote :

https://wiki.ubuntu.com/Releases

Changed in apt (Ubuntu):
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.