Add support for rootfs post-processing in l-m-c

This is a shell script I use which will loop-mount an image or device (e.g. MMC) and either run a command a command in it or spawn your default shell. I use it like this:
loop-mnt-do -o $((106496*512)) panda.img

(106496 is the start sector of the second partition as returned by "file panda.img")

As this issue continues to annoy me and deter me from updating to new images, I have created a proof of concept implementation of an option that adds simple customization support. It works by copying a command/script into the rootfs and executing it in a chroot. I did it this way because that is what I need more at the moment (eg no need to copy files into the chroot).

I see two ways to implement this feature properly in the context of l-m-c:

1. Provide a hook to run a script on the host, providing all relevant information (paths etc) and let it copy files, set up and use a chroot on its own etc
2. Provide two hooks, one for running on the host (for copying stuff into the chroot), one for running into the chroot (for installing etc).

If l-m-c developers are agreeable to the idea of providing such a customization feature, then I am willing to put in the time to discuss more about it and implement it properly (which shouldn't be hard).

The only problem I see with this is that it would make troubleshooting l-m-c bugs harder if a customized image won't boot. Then again, a user who supplies a customization script probably is advanced enough not to blame the tool then.

Thanks Alexandros, I've assigned this to Mattias to review your change and merge it if it looks good.

Thanks,

James

Customization of images in the manner eluded too is already available via live-build. I would recommend that linaro-media-create should not concern itself with this kind of activity but instead leave that as part of rootfs generation which supports it.

2c

The attached branch is just a proof-of-concept implementation, probably not suitable for merging as is. It runs a script inside the chroot, so it is limited in that it cannot copy files from the host into the rootfs. In comment #2 I have proposed some more flexible approaches.

@Tom:
Although both approaches will lead to the same result, I think they actually belong to different use cases.

My use case is that it is currently tedious to customize the produced images each time they are created, so I (as a developer) am deterred from trying them out on a regular basis. I think people would find it even more tedious to use live-build, so that they are able to install some additional development packages, set up the network, install synergy etc.

You are right that this isn't the right mechanism for producing "official" customized images and it is not my intention to promote this instead of the proper tool which is live-build. I believe that these approaches are complimentary, not conflicting.

> @Tom:
> Although both approaches will lead to the same result, I think they actually belong to different use cases.
...

In case it wasn't clear," both approaches" refers to live-build vs l-m-c customization, not the approaches mentioned in comment #2.

Live build has in the past been tedious and difficult to use. That has been changing.

I believe mere mortals ought to be able to easily build their own images as they see fit customized 9 ways till tuesday.

Really the most disappointing thing about live-build has been the time it takes to build an image, since you had to run it on native hardware. Even that has changed.

The reason for my comment is live-build already has the mechanisms for customization as part of the tools. Some things like say network are already done and it's just a matter of tweaking one file.

I really think by doing this we're putting too much into linaro-media-create.

Instead I would suggest it should be a stand alone tool that could either be part of or based on the live-build frameworks which are already there and generally run after the fact on images but if useful could be brought into live-build for use as part of official or not so official builds.

Perhaps we have a good LC topic :-)

> The reason for my comment is live-build already has the mechanisms for customization as part of the tools. Some things like say network are already done and it's just a matter of tweaking one file.

Assuming I haven't misunderstood you, this doesn't solve the problem of applying small customizations in the released linaro images, which is my use case. I don't think its reasonable to expect that anyone who needs to change something in the images run a full-blown live-build session and create the image from scratch.

> I really think by doing this we're putting too much into linaro-media-create.
> Instead I would suggest it should be a stand alone tool that could either be part of or based on the live-build frameworks which are already there and generally run after the fact on images

The only additional feature we are adding is the ability to run an arbitrary script after the rootfs has been populated, before writing it to the SD card. This could be anything, ranging from performing customizations using whatever infrastructure one wants to just gathering some useful statistics.

That being said, I understand your concern about overloading l-m-c, and I am not opposed to having an external tool to handle this, if this approach feels cleaner overall. One could even argue that there is no need for any of this, as it is straightforward to create a custom script that performs the post processing manually, and this is true to some extent (e.g. see Loïc's script in comment #1). However, the point of this report is that setting up a working environment after creating a new image is such a common task for developers, that it is worth providing support for it by default, in a straightforward and automated way.

Since the branch is a proof-of-concept and perhaps not ready for merging, I won't have time to bring it in for 2011.10 (today). I hope it's ok to push it to 2011.11.

> I hope it's ok to push it to 2011.11.

Sure. We can discuss more at the Connect about what's the best way to handle this (perhaps as part of an image customization session as Tom suggested?).

After reading this bug again, I believe it'd make a lot of sense for people to be able to hook scripts at the linaro-media-create itself.

Having the customization as part of the live-build process, when creating the image, is for sure the best thing to do, but we should also cover the use cases where the user doesn't want to create their own images (think about not even having Ubuntu/Debian around), and just want to customize the released images.

On the l-m-c create I believe we should have 2 hooks, one before installing the packages (so it can be used to automatically agree to any license that the hwpack needs, a valid use case for LAVA), and another after the flashing is done. This should probably cover all our current use cases for it.

Then to avoid having bugs at both the tools and images when the user customized a released image, l-m-c should simply just stamp the image saying which script was used while flashing the image, in a way we can easily see if this was the case when dealing with bugs.

On Wed, Dec 7, 2011 at 3:25 AM, Ricardo Salveti <email address hidden> wrote:
> After reading this bug again, I believe it'd make a lot of sense for
> people to be able to hook scripts at the linaro-media-create itself.

I added that during the Connect and Tom seemed to like it. It's just
hanging until he could test it. He might have better things to do atm.
:)

>
> Having the customization as part of the live-build process, when
> creating the image, is for sure the best thing to do, but we should also
> cover the use cases where the user doesn't want to create their own
> images (think about not even having Ubuntu/Debian around), and just want
> to customize the released images.
>
> On the l-m-c create I believe we should have 2 hooks, one before
> installing the packages (so it can be used to automatically agree to any
> license that the hwpack needs, a valid use case for LAVA), and another
> after the flashing is done. This should probably cover all our current
> use cases for it.

Can you have a look at my linked branch and see if something like that
would be ok? Perhaps we need to add another hook somewhere then.

>
> Then to avoid having bugs at both the tools and images when the user
> customized a released image, l-m-c should simply just stamp the image
> saying which script was used while flashing the image, in a way we can
> easily see if this was the case when dealing with bugs.

Good idea. That's not in the linked branch.

>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/751531
>
> Title:
>  Add support for rootfs post-processing in l-m-c
>
> Status in Linaro Image Tools:
>  In Progress
>
> Bug description:
>  It is currently tedious to customize the produced images each time
>  they are created. This deters developers from changing and testing
>  images often, especially when they want to use images for development
>  or complicated testing. Needed customizations include everything from
>  installing packages to adding configuration files (ssh, network,
>  editors).
>
>  A simple but effective solution is for l-m-c to allow the execution of
>  a user shell script after the rootfs has been populated. l-m-c should
>  pass all necessary information to the script (location of rootfs,
>  default user etc). The script can then do whatever it wants to
>  customize the image according to each user's needs (copy files into
>  the rootfs, chroot and install packages etc).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/linaro-image-tools/+bug/751531/+subscriptions

bumping prio to high ... as this blocks now using snowball ubuntu acceleration in the lab.

I think a dirty patch/branch would be ok to unblock lava+ubuntu as we just need to get this feature in the lab for now.

not having this feature in the lab in theory blocks snowball release for 12.01 ... adjusting priority/milestone accordingly.

adding lava-dispatcher target as that most likely need to pick up new lmc feature to accept license or do the seeding.

due to milestone overlap this is 2012.02 for lava software, however, the real goal is to be able to validate 12.01 release of linaro-ubuntu in the lab at best.

Great to see some interest in this again. See the linked fix for l-m-c, which was the agreed way forward during the last Connect. It's just waiting for someone to test and review it.

What is blocking us here is the implementation of the "lb modify" command that l-m-c is supposed to call to modify the rootfs. I am not sure how far the implementation has progressed.

In case "lb modify" implementation and deployment is too far away, and a solution is needed now, I think the most versatile way forward is option 1 as described in comment #2. That is, just call a user script providing all the related paths in some way, and let it do whatever it wants, including running "lb modify" if/when support for it lands.

... or perhaps something like Ricardo is suggesting in #12 if there is a demand for two separate hooks.

On Thu, Jan 19, 2012 at 11:55 PM, Alexandros Frantzis <email address hidden> wrote:
> What is blocking us here is the implementation of the "lb modify"
> command that l-m-c is supposed to call to modify the rootfs. I am not
> sure how far the implementation has progressed.

Oh, I didn't realize that 'lb modify' isn't available yet. You
probably told me at the Connect that it needs implementing, but I must
have forgotten that and thought this fix was good to use.

>
> In case "lb modify" implementation and deployment is too far away, and a
> solution is needed now, I think the most versatile way forward is option
> 1 as described in comment #2. That is, just call a user script providing
> all the related paths in some way, and let it do whatever it wants,
> including running "lb modify" if/when support for it lands.

So this should be changed to just running a generic script then by
just executing the file the user has supplied, would that be good
enough?

    sudo ${user-script} --chroot-path ${chroot_path}

I re-read Ricardo's comment now about adding two separate hooks and
that seems to be needed again. So we should be able to provide a
pre-hwpack-install script as well as a post-install script. Any other
options that we should pass to the hook scripts?

>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/751531
>
> Title:
>  Add support for rootfs post-processing in l-m-c
>
> Status in LAVA Dispatcher:
>  New
> Status in Linaro Image Tools:
>  Triaged
> Status in Linaro Ubuntu Evaluation Builds:
>  Confirmed
>
> Bug description:
>  It is currently tedious to customize the produced images each time
>  they are created. This deters developers from changing and testing
>  images often, especially when they want to use images for development
>  or complicated testing. Needed customizations include everything from
>  installing packages to adding configuration files (ssh, network,
>  editors).
>
>  A simple but effective solution is for l-m-c to allow the execution of
>  a user shell script after the rootfs has been populated. l-m-c should
>  pass all necessary information to the script (location of rootfs,
>  default user etc). The script can then do whatever it wants to
>  customize the image according to each user's needs (copy files into
>  the rootfs, chroot and install packages etc).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/lava-dispatcher/+bug/751531/+subscriptions

On Fri, Jan 20, 2012 at 9:08 AM, Mattias Backman
<email address hidden> wrote:
> On Thu, Jan 19, 2012 at 11:55 PM, Alexandros Frantzis <email address hidden> wrote:
>> In case "lb modify" implementation and deployment is too far away, and a
>> solution is needed now, I think the most versatile way forward is option
>> 1 as described in comment #2. That is, just call a user script providing
>> all the related paths in some way, and let it do whatever it wants,
>> including running "lb modify" if/when support for it lands.
>
> So this should be changed to just running a generic script then by
> just executing the file the user has supplied, would that be good
> enough?
>
>    sudo ${user-script} --chroot-path ${chroot_path}
>
> I re-read Ricardo's comment now about adding two separate hooks and
> that seems to be needed again. So we should be able to provide a
> pre-hwpack-install script as well as a post-install script. Any other
> options that we should pass to the hook scripts?

Don't think so, I believe we should just run the script as root at the
chroot, and that's about it.

Then to fix the snowball issue we'd just create a script to run as
pre-hwpack-install, that would create the needed license check files.

I would argue for not running the script inside the rootfs (although my initial branch did just this), as this decreases the generality of the feature. For example, if the script is running inside the chroot, how would you copy files into the chroot?

On Fri, Jan 20, 2012 at 12:07 PM, Alexandros Frantzis <email address hidden> wrote:
> I would argue for not running the script inside the rootfs (although my
> initial branch did just this), as this decreases the generality of the
> feature. For example, if the script is running inside the chroot, how
> would you copy files into the chroot?

Yeah, that would also be something to consider.

The good thing about running it in the chroot is that we don't need to
care about security issues at the host side that much. If we allow the
script to also touch the host system, we just need to be careful by
not allowing sudo access that easily.

On Fri, 20 Jan 2012 14:37:57 -0000, Ricardo Salveti <email address hidden> wrote:

> The good thing about running it in the chroot is that we don't need to
> care about security issues at the host side that much.

Apart from the bit where chroots do not provide security?

Cheers,
mwh

On Mon, Jan 23, 2012 at 7:56 PM, Michael Hudson-Doyle
<email address hidden> wrote:
> On Fri, 20 Jan 2012 14:37:57 -0000, Ricardo Salveti
> <email address hidden> wrote:
>
>> The good thing about running it in the chroot is that we don't need to
>> care about security issues at the host side that much.
>
> Apart from the bit where chroots do not provide security?

But it's safer at least.

What is the lava-dispatcher task about? I've worked around the license problems in a different way.

@mwhudson: we'll have some work to drop Snowball work around in LAVA and replace it by a hook to do the seeding

Fathi dislikes the currently proposed approach of using command line options. He mentioned how he might look into solving it in a different way (using directories and environment variables). In general, I am opposed to that since we are using command line options for l-m-c otherwise, but if that's the general consensus, we'd be happy to take a patch (with tests).

Just noticed that this was moved into "in progress" on 2013-06-11, seems like the query needs some help. sorry for any confusion.