Support "fudge factor" for file system sizing

Mailing list thread: https://lists.snapcraft.io/archives/devices/2016-October/000089.html

Oli makes the case for a command line switch, from the mailing list:

"""
defining it in the gadget would be rather awful since that means if we
use an identical image for cloud, VM  and "normal PC install" we would
need a gadget per image just for that one difference ... (the pc image
that you can use for all three consists of: pc (gadget, pc-kernel
(obviously kernel) and core (rootfs).

also not every cloud provider or VM user might want to use the same
basic size so they would also need their own gadgets that set the image
size here, i think that makes it to complex, thus i would rather go
with a commandline option to the build tool.
"""

Motivating use case, also from Oli on the mailing list:

"""
on VMs the img file defines the "physical VM disk" while for an actual
install you write the img file to a pyhsical disk device ... the resize
code will find that the partition table does not cover all of the disk
before first boot and start the resize process so here the img size
wont be taken into account at all.

on a VM, all the resize code will see is the img file that pretends to
be a harddisk and it will find there is nothing to resize to.
"""

I plan on supporting this through a command line option that overrides the hardcoded 8MiB rootfs fudge factor.

For now, I won't change the 1.5x size calculation boost on all directory calculations, although this should certainly be addressed later.

Note that this is different than LP: #1632424

From the mailing list:

I don't think a 'fudge factor' option is particularly clean or intuitive. I
think what you actually want is to be able to declare the overall size of
the image that you're building. It's a waste of mental energy to think
about VM image sizes in increments less than 1G, especially given that these
are all still going to be sparse files.

So I would suggest:

 - a --size option to ubuntu-image to specify the full image size (needs
   some careful thought for the multiple-image case)
 - keep the 50% buffer on ext4 filesystem size as a floor (the '50%' can be
   refined over time with investigation into the actual size requirements
   for ext4 metadata, but this is not a high priority; highest priority is
   that it works reliably)
 - if --size is smaller than the minimum size needed to accommodate the
   contents, warn but build the image anyway - because all the expensive
   work of ubuntu-image happens /before/ we're able to calculate the final
   size, so throwing that all away at the end would be /very annoying/.

I want to be clear about one point: the size that this switch will control is the overall disk image size, not the size of any individual partition within the image. I.e. it's the size of the resulting file written to via the --output switch.

Thus all the discussion about fudge factors is off-topic, as those are applied to partition sizes. We will address those separately.

Here's how the image size is calculate (in prepare_filesystems()):

        self.image_size = ceil((self.rootfs_size + next_avail) /
                               1024 + 17) * 1024

I propose therefore to call the switch `--image-size` (no shortcut) which takes a value of the same format as the other size parameters in the gadget.yaml. A bare integer means bytes, and it accepts suffixes 'M' for MiB and 'G' for GiB.

Since u-i only handles single volume gadget.yamls atm, --image-size=10G would refer to the first and only image. In the future we can relax the "only" part, so that an unqualified value would refer to first volume. This would be equivalent to the (future) extended syntax --image-size=0:10G. Later we can accept multiple --image-size arguments, and/or a syntax such as --image-size=0:10G,1:1000M,2:1073741824

We'll check that --image-size is at least as big as the calculated size. I'd prefer to fail hard in that case, since it's generally easier to relax a constraint than to tighten it, but as Steve points out, that would throw away a lot of work and even though we support preserving the working directory, we don't currently have a way to resume after failure. So, if the --image-size is smaller than the calculated size, we'll warn and ignore it, going with the calculated size instead. That way, we'll preserve all of the work and won't generate a corrupted image.

Alternative to the indexes, we could support named volumes.

https://github.com/CanonicalLtd/ubuntu-image/pull/74

Please add an SRU test case and regression potential.

See LP: #1634218 and https://lists.ubuntu.com/archives/ubuntu-release/2016-October/003949.html for SRU