docker.io doesn't work with apparmor 3.0 RC1 kernel

a failure in change_profile from unconfined is NOT expected to log a message.

Can you please verify that the target profile is loaded. The only reason apparmor rejects change_profile for unconfined is that the profile could not be found.

Oh can we also test against the distro kernel that the RC1 patches are based on to ensure that there aren't other changes in play

The target profile is loaded:
$ sudo aa-status|grep docker
   docker-default

I tried this on the 3.16.0-9.14 and 3.16.0-16.22 distro kernels. The 'docker run' command succeeds. If I do this:
$ sudo docker run -i -t ubuntu:trusty /bin/sh

I can verify the container is launched under confinement here:
sudo aa-status|grep docker
   docker-default
   docker-default (2209)

$ ps -Z 2209
LABEL PID TTY STAT TIME COMMAND
docker-default 2209 pts/1 Ss+ 0:00 /bin/sh

Installing auditd does not help.

Adding the following to /etc/apparmor.d/docker does not help:
  audit unix,
  audit signal,
  audit ptrace,
  change_profile -> *,

This is being caused by a bug fix in apparmor that places tighter requirements on the use of change_onexec from a multi-threaded application.

How to best resolve this issue on the apparmor side is being investigated. It is very likely that docker is not using the change_profile api correctly, and should be audited as well.

After discussing on IRC, we will revert the patch enabling stricter requirements to restore previous behavior while we investigate the best approach to resolve the issue properly.

Specifically we will revert the patch to security/apparmor/domain.c

fix: only allow a single threaded process to change context on exec

I'll take a look at adding this to Debian & Ubuntu - can we get this
upstreamed too?

On Tue, Sep 23, 2014 at 3:01 PM, Jamie Strandboge <email address hidden> wrote:
> ** Tags added: apparmor
>
> --
> You received this bug notification because you are a member of Docker
> Ubuntu Maintainers, which is subscribed to docker.io in Ubuntu.
> https://bugs.launchpad.net/bugs/1371310
>
> Title:
> docker.io doesn't work with apparmor 3.0 RC1 kernel
>
> Status in “apparmor” package in Ubuntu:
> Invalid
> Status in “docker.io” package in Ubuntu:
> Invalid
> Status in “linux” package in Ubuntu:
> Triaged
>
> Bug description:
> Steps to reproduce (from
> https://wiki.ubuntu.com/Process/Merges/TestPlans/AppArmor):
>
> 1. sudo apt-get install docker.io # 1.2.0~dfsg1-1
>
> 2. sudo docker pull ubuntu:trusty
>
> 3. sudo docker run ubuntu:trusty uptime
> 2014/09/18 15:48:48 Error response from daemon: Cannot start container fcdfaaf7945bcd9455fb5e0bde9950451152af14556880033818df7b50ddb1f4: set apparmor profile docker-default: permission denied
>
> What is expected? uptime to return something like:
> $ sudo docker run ubuntu:trusty uptime
> 20:31:21 up 1 min, 0 users, load average: 0.09, 0.06, 0.03
>
> I set 'sudo sysctl -w kernel.printk_ratelimit=0' but there is nothing
> apparmor related in the logs. If I boot an earlier kernel without the
> 3.0 RC1 patches, it works.
>
> FYI, 3.16.0-17.23 is in utopic-proposed now and on its way to utopic,
> which will affect docker.io in Ubuntu. Workaround until this bug is
> fixed is to boot into 3.16.0-16.22 or earlier.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1371310/+subscriptions

--
All programmers are playwrights, and all computers are lousy actors.

#define sizeof(x) rand()
:wq

d'oh, lost context on this thread and thought it was a new bug against docker with a patch. my fault. carry on, thanks folks.

I just upgraded to Utopic final beta and am experiencing this bug. What is a workaround to get my docker containers working?

John, from the description: "Workaround until this bug is fixed is to boot into 3.16.0-16.22 or earlier." If you upgraded, you probably still have the trusty kernel installed, so you can boot into that. Otherwise, use https://launchpad.net/ubuntu/utopic/+source/linux/3.16.0-16.22

@jdstrand:

Sorry, I missed that bit. I rebooted into the old Trusty kernel and Docker works fine (although compiz/mesa don't like the old kernel, so I guess I'll try 3.16.0-16.22). Thanks.

I can confirm that reverting the patch John Johansen mentioned makes docker.io work as well as in previous releases. We will have this fixed before 14.10 release.

FYI, this should be in a new kernel soon: https://lists.ubuntu.com/archives/kernel-team/2014-October/049001.html

Workaroud: Updating to kernel 3.17 ( from here: http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.17-utopic/ ) solves this.

This bug was fixed in the package linux - 3.16.0-22.29

---------------
linux (3.16.0-22.29) utopic; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #1379321

  [ Andrew Morton ]

  * SAUCE: (no-up) mm-introduce-a-general-rcu-get_user_pages_fast-fix
    - LP: #1309221
  * SAUCE: (no-up) arm64-mm-enable-rcu-fast_gup-checkpatch-fixes
    - LP: #1309221

  [ Andy Whitcroft ]

  * [Config] CONFIG_PATA_MACIO=y
    - LP: #1378894
  * [Config] enable cloud tools on i386
    - LP: #1367399
  * SAUCE: scsi: hyper-v storsvc switch up to SPC-3
    - LP: #1354397
  * SAUCE: powerpc -- fix mm/slice.c switch include to linux/hugetlb.h

  [ dann frazier ]

  * [Config] CONFIG_HAVE_GENERIC_RCU_GUP=y
    - LP: #1309221

  [ Feng Kan ]

  * SAUCE: (no-up) power: reset: Add generic SYSCON register mapped reset
    - LP: #1284433
  * SAUCE: (no-up) arm64: dts: Add X-Gene reboot driver dts node
    - LP: #1284433

  [ Ian Munsie ]

  * SAUCE: (no-up) powerpc/cell: Move spu_handle_mm_fault() out of cell platform
  * SAUCE: (no-up) powerpc/cell: Move data segment faulting code out of cell platform
  * SAUCE: (no-up) powerpc/cell: Make spu_flush_all_slbs() generic
  * SAUCE: (no-up) powerpc/msi: Improve IRQ bitmap allocator
  * SAUCE: (no-up) powerpc/mm: Export mmu_kernel_ssize and mmu_linear_psize
  * SAUCE: (no-up) powerpc/powernv: Split out set MSI IRQ chip code
  * SAUCE: (no-up) cxl: Add new header for call backs and structs
  * SAUCE: (no-up) powerpc/powerpc: Add new PCIe functions for allocating cxl interrupts
  * SAUCE: (no-up) powerpc/mm: Add new hash_page_mm()
  * SAUCE: (no-up) powerpc/opal: Add PHB to cxl mode call
  * SAUCE: (no-up) powerpc/mm: Add hooks for cxl
  * SAUCE: (no-up) cxl: Add base builtin support
  * SAUCE: (no-up) cxl: Driver code for powernv PCIe based cards for userspace access
  * SAUCE: (no-up) cxl: Add userspace header file
  * SAUCE: (no-up) cxl: Add driver to Kbuild and Makefiles
  * SAUCE: (no-up) cxl: Add documentation for userspace APIs
  * SAUCE: (no-up) cxl: Fix afu_read() not doing finish_wait() on signal or non-blocking

  [ John Johansen ]

  * SAUCE: Revert: fix: only allow a single threaded process to ...
    - LP: #1371310

  [ Steve Capper ]

  * SAUCE: (no-up) mm: introduce a general RCU get_user_pages_fast()
    - LP: #1309221
  * SAUCE: (no-up) arm: mm: introduce special ptes for LPAE
    - LP: #1309221
  * SAUCE: (no-up) arm: mm: enable HAVE_RCU_TABLE_FREE logic
    - LP: #1309221
  * SAUCE: (no-up) arm: mm: enable RCU fast_gup
    - LP: #1309221
  * SAUCE: (no-up) arm64: mm: enable HAVE_RCU_TABLE_FREE logic
    - LP: #1309221
  * SAUCE: (no-up) arm64: mm: enable RCU fast_gup
    - LP: #1309221

  [ Tim Gardner ]

  * SAUCE: Added bnx2x/bnx2x-e1-7.8.19.0.fw
    - LP: #1378491
  * [Config] CONFIG_CXL=m
  * [Config] CONFIG_POWER_RESET_SYSCON=y for arm64
  * SAUCE: (no-up) Restrict CONFIG_POWER_RESET_SYSCON to arm64 only

  [ Upstream Kernel Changes ]

  * powerpc: implement vmemmap_list_free()
    - LP: #1378413
  * powerpc: implement vmemmap_remove_mapping() for BOOK3S
    - LP: #1378413
  * powerpc: implement vmemmap_free()
    - LP: #1378413
  * powerpc: start loop at section start of start in vm...

Yep, Docker works again with kernel 3.16.0-22.29. Thanks for fixing this so promptly.

I am getting the following still.
# docker run -i -t ubuntu /bin/bash
set apparmor profile docker-default: no such file or directory2014/10/13 11:05:41 Error response from daemon: Cannot start container 7981801146ab1771470fd72bc97e6582ff573fa159199f6cb86afaa7eaa74ea9: set apparmor profile docker-default: no such file or directory

Here's uname -a:
# uname -a
Linux braindead 3.16.0-22-generic #29-Ubuntu SMP Thu Oct 9 16:26:18 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

As seen above, I am running 3.16.0-22.29. Is there something I need to do manually to fix this?

I also tried the 3.17.0-031700-generic #2001410060605 linked above. It also does not work on that kernel.

# docker run -i -t ubuntu /bin/bash
Unable to find image 'ubuntu' locally
Pulling repository ubuntu
1357f421be38: Download complete
511136ea3c5a: Download complete
1de2bd7201a2: Download complete
b70ad18cfc2a: Download complete
3b39241dab7c: Download complete
23f1840ed5a5: Download complete
dfa12c637781: Download complete
set apparmor profile docker-default: no such file or directory2014/10/13 11:24:00 Error response from daemon: Cannot start container 37fdd08fdb5ef28e9a564eb25723acf2363478d2e188f57c7587d416a2ec7825: set apparmor profile docker-default: no such file or directory

# uname -a
Linux braindead 3.17.0-031700-generic #201410060605 SMP Mon Oct 6 10:07:09 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Just so you know, I did rm -rf /var/lib/docker and reinstalled the docker.io package between each try.

Just so you know, here's the docker.io package version also:
ii docker.io 1.2.0~dfsg1-1ubuntu1 amd64 Linux container runtime

I just found that the apparmor profile for docker was disabled for some reason. After removing the symlink from /etc/apparmor.d/disable/docker, it works. I don't remember ever adding docker to the apparmor disable links.

This bug was fixed in the package linux - 3.18.0-8.9

---------------
linux (3.18.0-8.9) vivid; urgency=low

  [ Leann Ogasawara ]

  * Release Tracking Bug
    - LP: #1407692
  * rebase to v3.18.1
  * ubuntu: AUFS -- Resolve build failure union has no member named
    'd_child'

  [ Upstream Kernel Changes ]

  * arm64: optimized copy_to_user and copy_from_user assembly code
    - LP: #1400349
  * x86, kvm: Clear paravirt_enabled on KVM guests for espfix32's benefit
    - LP: #1400314
    - CVE-2014-8134
  * rebase to v3.18.1
 -- Leann Ogasawara <email address hidden> Mon, 05 Jan 2015 09:12:32 -0800