Docker hangs with xfs using aufs storage driver

Bug #1709749 reported by Akihiro Suda
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
Undecided
Unassigned
Zesty
Fix Released
Medium
Seth Forshee

Bug Description

SRU Justification

Impact: Running the yum command in a Centos/RHEL container causes the process to hang enter uninterruptible sleep mode if /var/lib/docker is hosted on an XFS filesystem while the AUFS storage driver is used.

Fix: Commit e34c81ff96415c64ca827ec30e7935454d26c1d3 from upstream aufs-standalone. Requires prerequisite commit b4d3dcc92a13d53952fe6e9a640201ef87475302.

Test Case: Test scenario described above. Kernel with fixes applied has been confirmed by reporter to fix the issue.

Regression Potential: Both patches have been in upstream aufs-standalone for a while now and included in the artful kernel as well as currently being included in the upstream branch for 4.10. Therefore they are well tested and unlikely to cause regressions.

---

1) The release of Ubuntu you are using, via 'lsb_release -rd' or System -> About Ubuntu
Ubuntu 16.04 and 17.04

2) The version of the package you are using, via 'apt-cache policy pkgname' or by checking in Software Center
kernel 4.10.0-28-generic

3) What you expected to happen

Docker with AUFS on XFS works.

4) What happened instead

It hangs up

---

As reported in https://github.com/moby/moby/issues/34361 , AUFS on XFS needs this fix for kernel 4.10: https://github.com/sfjro/aufs4-standalone/commit/e34c81ff96415c64ca827ec30e7935454d26c1d3

CVE References

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1709749

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Seth Forshee (sforshee)
Changed in linux (Ubuntu Zesty):
assignee: nobody → Seth Forshee (sforshee)
importance: Undecided → Medium
status: New → In Progress
Changed in linux (Ubuntu):
status: Incomplete → Fix Released
Revision history for this message
Seth Forshee (sforshee) wrote : Re: AUFS on XFS needs fix for kernel 4.10

I've backported the fix and necessary prerequisite patches and prepared a test build so you can verify that these fix the issue. Can you please test the kernel at the link below and let me know your results?

http://people.canonical.com/~sforshee/lp1709749/

Let me know if you have questions about which packages you need to install, likely only linux-image-*-generic and linux-image-extra-*-generic but possibly others if you use dkms drivers or something like that. Thanks!

Changed in linux (Ubuntu Zesty):
status: In Progress → Incomplete
Revision history for this message
Seth Forshee (sforshee) wrote :

Oh and you can ignore the request for logs in comment #1.

tags: added: kernel-da-key
Revision history for this message
Akihiro Suda (suda-kyoto) wrote :

Yes, I confirmed that the lp1709749 kernel works as expected.
Thank you for the work.

Seth Forshee (sforshee)
Changed in linux (Ubuntu Zesty):
status: Incomplete → In Progress
summary: - AUFS on XFS needs fix for kernel 4.10
+ Docker hangs with xfs using aufs storage driver
Seth Forshee (sforshee)
description: updated
Revision history for this message
Scott Emmons (lscotte) wrote :

This appears to be a duplicate of https://bugs.launchpad.net/ubuntu/+source/docker.io/+bug/1679768 and also affects Xenial when using hwe backported kernel.

Changed in linux (Ubuntu Zesty):
status: In Progress → Fix Committed
Revision history for this message
Kleber Sacilotto de Souza (kleber-souza) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-zesty' to 'verification-done-zesty'. If the problem still exists, change the tag 'verification-needed-zesty' to 'verification-failed-zesty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-zesty
Revision history for this message
Kleber Sacilotto de Souza (kleber-souza) wrote :

Hello Akihiro,

Could you please verify the fix with the latest Zesty kernel in -proposed?

Thank you!

Revision history for this message
Akihiro Suda (suda-kyoto) wrote :

Confirmed 4.10.0-34-generic (x86_64) works, sorry for late reply

Revision history for this message
Kleber Sacilotto de Souza (kleber-souza) wrote :

Hello Akihiro,

Thank you verifying the fix.

tags: added: verification-done-zesty
removed: verification-needed-zesty
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (4.2 KiB)

This bug was fixed in the package linux - 4.10.0-35.39

---------------
linux (4.10.0-35.39) zesty; urgency=low

  * linux: 4.10.0-35.39 -proposed tracker (LP: #1716606)

  * kernel panic -not syncing: Fatal exception: panic_on_oops (LP: #1708399)
    - SAUCE: s390/mm: fix local TLB flushing vs. detach of an mm address space
    - SAUCE: s390/mm: fix race on mm->context.flush_mm

  * CVE-2017-1000251
    - Bluetooth: Properly check L2CAP config option output buffer length

linux (4.10.0-34.38) zesty; urgency=low

  * linux: 4.10.0-34.38 -proposed tracker (LP: #1713470)

  * Ubuntu 16.04.03: perf tool does not count pm_run_inst_cmpl with rcode on
    POWER9 DD2.0 (LP: #1709964)
    - powerpc/perf: Fix Power9 test_adder fields

  * HID: multitouch: Support ALPS PTP Stick and Touchpad devices (LP: #1712481)
    - HID: multitouch: Support PTP Stick and Touchpad device
    - SAUCE: HID: multitouch: Support ALPS PTP stick with pid 0x120A

  * igb: Support using Broadcom 54616 as PHY (LP: #1712024)
    - SAUCE: igb: add support for using Broadcom 54616 as PHY

  * RPT related fixes missing in Ubuntu 16.04.3 (LP: #1709220)
    - powerpc/mm/radix: Optimise tlbiel flush all case
    - powerpc/mm/radix: Improve _tlbiel_pid to be usable for PWC flushes
    - powerpc/mm/radix: Improve TLB/PWC flushes
    - powerpc/mm/radix: Avoid flushing the PWC on every flush_tlb_range

  * AMD RV platforms with SNPS 3.1 USB controller stop responding (S3 issue)
    (LP: #1711098)
    - usb: xhci: Issue stop EP command only when the EP state is running

  * dma-buf: performance issue when looking up the fence status (LP: #1711096)
    - dma-buf: avoid scheduling on fence status query v2

  * IPR driver causes multipath to fail paths/stuck IO on Medium Errors
    (LP: #1682644)
    - scsi: ipr: do not set DID_PASSTHROUGH on CHECK CONDITION

  * Disable CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE (LP: #1709171)
    - [Config] CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=n for ppc64el

  * memory-hotplug test needs to be fixed (LP: #1710868)
    - selftests: typo correction for memory-hotplug test
    - selftests: check hot-pluggagble memory for memory-hotplug test
    - selftests: check percentage range for memory-hotplug test
    - selftests: add missing test name in memory-hotplug test
    - selftests: fix memory-hotplug test

  * Ubuntu 16.04.3: Qemu fails on P9 (LP: #1686019)
    - KVM: PPC: Pass kvm* to kvmppc_find_table()
    - KVM: PPC: Use preregistered memory API to access TCE list
    - KVM: PPC: VFIO: Add in-kernel acceleration for VFIO
    - powerpc/powernv/iommu: Add real mode version of iommu_table_ops::exchange()
    - powerpc/powernv/ioda2: Update iommu table base on ownership change
    - powerpc/iommu/vfio_spapr_tce: Cleanup iommu_table disposal
    - powerpc/vfio_spapr_tce: Add reference counting to iommu_table
    - powerpc/mmu: Add real mode support for IOMMU preregistered memory
    - KVM: PPC: Reserve KVM_CAP_SPAPR_TCE_VFIO capability number
    - KVM: PPC: Book3S HV: Add radix checks in real-mode hypercall handlers

  * [SRU][Zesty] [QDF2400] pl011 E44 erratum patch needed for 2.0 firmware and
    1.1 silicon (LP: #1709123)
    - tty: pl011: fix initialization or...

Read more...

Changed in linux (Ubuntu Zesty):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.