lxc containers fail to start in trusty with newer kernels

Bug #1317179 reported by Seth Forshee
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Fix Released
Medium
Stéphane Graber
Trusty
Fix Released
Medium
Stéphane Graber

Bug Description

When running a trusty userspace with a newer kernel (e.g. 3.15-rc4), attempting to start a container fails:

$ lxc-start -n p2
lxc_container: call to cgmanager_create_sync failed: invalid request
lxc_container: Failed to create net_cls:p2
lxc_container: Error creating cgroup net_cls:p2
lxc_container: failed creating cgroups
lxc_container: failed to spawn 'p2'

Iiuc, this seems to be caused new cgroups in the kernel which aren't enabled for systemd but that lxc tries to configure anyway.

Seth Forshee (sforshee)
no longer affects: lxc (Ubuntu)
no longer affects: lxc
Seth Forshee (sforshee)
description: updated
Revision history for this message
Serge Hallyn (serge-hallyn) wrote :

The variable JoinControllers in /etc/systemd/system.cofn should include net_cls. Otherwise when newer kernel is used in trusty, users will not by default be able to use unprivileged containers.

Changed in systemd (Ubuntu):
importance: Undecided → Medium
status: New → Fix Released
Changed in systemd (Ubuntu Trusty):
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
Martin Pitt (pitti) wrote :

Seth, you marked this as fix released in utopic, but there was no new upload for this. Did some of the recent utopic changes "accidentally" fix this? I don't remember anything which was related to LXC. Thanks!

Revision history for this message
Serge Hallyn (serge-hallyn) wrote : Re: [Bug 1317179] Re: lxc containers fail to start in trusty with newer kernels

Now that's odd. This certainly seemed to be fixed in utopic, but
after a quick dist-upgrade from a trusty host net_cls in fact
does not appear to be there. So I think you're right - thanks!

 status: confirmed

Changed in systemd (Ubuntu):
status: Fix Released → Confirmed
Revision history for this message
Martin Pitt (pitti) wrote :

Serge,

we currently have this patch in our systemd:

http://anonscm.debian.org/gitweb/?p=pkg-systemd/systemd.git;a=blob;f=debian/patches/Enable-all-cgroup-controllers-in-logind.patch;h=d80d34d1a4534684edaff2a6db35572d971fcbde;hb=refs/heads/ubuntu

which adds those to /etc/systemd/logind.conf's "Controllers" stanza. Can you please double-check that you really mean /etc/systemd/system.conf and not logind.conf? The former shouldn't even be used on Ubuntu while running upstart.

Revision history for this message
Martin Pitt (pitti) wrote :

Seth, can you try to add net_cls to /etc/systemd/logind.conf 's "Controllers=" line and see if that helps?

Changed in systemd (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Seth Forshee (sforshee) wrote :

Martin: So I hadn't actually tested in utopic until late yesterday, and I did find it to be broken there. Serge had marked it fixed, but obviously we're getting different results.

Adding net_cls in logind.conf fixes the problem for me in both trusty and utopic.

Revision history for this message
Martin Pitt (pitti) wrote :

Thanks Seth. So I figure Serge actually meant logind.conf when he said systemd.conf. :-) All clear now.

Changed in systemd (Ubuntu):
status: Incomplete → Triaged
Changed in systemd (Ubuntu Trusty):
status: Confirmed → Triaged
Revision history for this message
Martin Pitt (pitti) wrote :
Changed in systemd (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
Stéphane Graber (stgraber) wrote :

Sorry Martin, I didn't see this bug until after I fixed it...

So I added both net_cls and net_prio to utopic (jso we have them all listed just in case), would be great if you could update the git branch to match.

I'm also doing the SRU to trusty now.

Changed in systemd (Ubuntu):
status: Fix Committed → Fix Released
Changed in systemd (Ubuntu Trusty):
assignee: nobody → Stéphane Graber (stgraber)
Changed in systemd (Ubuntu):
assignee: nobody → Stéphane Graber (stgraber)
Changed in systemd (Ubuntu Trusty):
status: Triaged → In Progress
Revision history for this message
Stéphane Graber (stgraber) wrote :

Uploaded to the queue.

Testcase:
 - Install current upstream kernel on trusty (3.15)
 - Confirm that /proc/self/cgroup is incorrect for the net_cls cgroup
 - Update systemd-services
 - Reboot
 - Confirm that /proc/self/cgroup now looks identical for net_cls as for other controllers.

Revision history for this message
Martin Pitt (pitti) wrote :

Adjusted git accordingly: http://anonscm.debian.org/gitweb/?p=pkg-systemd/systemd.git;a=commitdiff;h=4d696d . The changelog will differ a bit on next upload as there are other unstaged changes, but the actual content is the same.

Revision history for this message
Stéphane Graber (stgraber) wrote :

Cool, thanks Martin!

I sort of wish there were a magic "all" keyword or something in there but well, I don't expect the cgroup controller list to grow too much during the lifetime of the LTS and I suspect our actual 14.10 implementation will be quite different anyway...

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Seth, or anyone else affected,

Accepted systemd into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/systemd/204-5ubuntu20.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in systemd (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Seth Forshee (sforshee) wrote :

The packages from systemd 204-5ubuntu20.2 fix the problem.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 204-5ubuntu20.2

---------------
systemd (204-5ubuntu20.2) trusty; urgency=medium

  * Update Enable-all-cgroup-controllers-in-logind.patch to also include
    net_cls and net_prio. That should cover all existing cgroups.
    (LP: #1317179)
 -- Stephane Graber <email address hidden> Thu, 08 May 2014 18:32:05 -0500

Changed in systemd (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.