precise cloud-images significantly larger than oneiric

Bug #926160 reported by Scott Moser
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-images
Won't Fix
Wishlist
Unassigned

Bug Description

See below for detail, but basically from oneiric released images to precise images, we've grown from root filesystem used of 762M to 973M. That is 27% size increase.

This compares what I found on
   ubuntu-oneiric-11.10-<arch>-server-20120108
   ubuntu-precise-daily-<arch>-server-20120202

The .img and .tar.gz files on cloud-images have grown also, but strangely it seems less. comparing alpha-2 from precise to last release of oneiric (20120108) shows:

-------------- | 11.10 | prec | %growth
amd64 disk.img | 218M | 228M | 4.8
amd64 .tar.gz_ | 207M | 217M | 4.6
amd64 file tot | 559M | 575M | 2.9
amd64 ebs df__ | 762M | 883M | 15.8
amd64 inst df_ | 655M | 823M | 25.6
i386_ disk.img | 198M | 210M | 6.0
i386_ .tar.gz_ | 188M | 200M | 6.3
i386_ file tot | 484M | 505M | 4.3
i386_ ebs df__ | 689M | 814M | 18.1
i386_ inst df_ | 582M | 753M | 29.3

(Note, precise df numers are from daily build 20120108, 1 day after alpha2)
'file tot' is result of 'du --total --one-file-system --apparent-size'.

I suspect that part of the discrepancy between instance and ebs is somehow related to how we populate them. In the build process, the instance store builds are done into a 1.4G filesystem, and then that is grown (resize2fs) to 10G before publishing to EC2. The EBS population script downloads the 1.4GB filesystem image, creates a filesystem on a 8G disk, then mounts it loopback and rsync's the data from the image to the new filesystem. (Essentially, mkfs is done on a 1.4G filesystem, and one is done on a 8G filesystem).

So i have the following separate issues raised from above:
 * why such a dramatic growth between 11.10 and 12.04 current
 * can we/should we get the EBS filesystem usage more in line with instance-store?
 * Why the change in increased usage of ebs for a given release-arch:
   oneiric-amd64: 16%
   oneiric-i386: 18%
   precise-amd64: 7.2%
   precise-i386: 8%

Revision history for this message
Scott Moser (smoser) wrote :
description: updated
Revision history for this message
Scott Moser (smoser) wrote :

Note, the automatic running of 'apt-get update' had run on the amd64 precise instance, so that data was tainted. I changed the bug description to accommodate and picked up a new instancen data to replace it.

That is in this tarball.

description: updated
Scott Moser (smoser)
description: updated
Scott Moser (smoser)
Changed in ubuntu:
assignee: Ubuntu Server Team (ubuntu-server) → Ben Howard (utlemming)
Changed in ubuntu:
importance: Medium → Wishlist
Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Differences between the disks show the delta:

< 589708 /
---
> 589674 /
267c267
< 589708 total
---
> 589674 total
433c433
< 165201 /var
---
> 165167 /var
442c442
< 94861 /var/lib
---
> 94853 /var/lib
450c450
< 8766 /var/lib/dpkg
---
> 8758 /var/lib/dpkg
478c478
< 187 /var/log
---
> 162 /var/log

The space is being consumed in the temporal and application caching directories.

Resolving as won't fix since it will take some monkeying with the build system to fix. This will be addressed early next cycle.

Changed in ubuntu:
status: Confirmed → Won't Fix
Revision history for this message
Scott Moser (smoser) wrote :

Ben,
  Did you have some explaination for why we see significant (15%+) growth while the apparent size reported by du grew only 5% or so?

Revision history for this message
Scott Moser (smoser) wrote :

Sorry to be more clear, the thing that I do not understand is why the significant difference between these two lines in the table:
amd64 file tot | 559M | 575M | 2.9
amd64 ebs df__ | 762M | 883M | 15.8
amd64 inst df_ | 655M | 823M | 25.6

ie, counting total of all files space grew 2.9% , but the space used on the disk grew 15.8% (or even worse, 25% for instance store).

Mathew Hodson (mhodson)
affects: ubuntu → cloud-images
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.