Revert ESE DASD discard support

Bug #1846219 reported by Dimitri John Ledkov
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
Critical
Unassigned
linux (Ubuntu)
Fix Released
Critical
Unassigned

Bug Description

[Impact]

 * Under certain conditions, the discard operation on the new ESE DASD may lead to data corruption.

[Test Case]

 * Internal IBM testing

[Regression Potential]

 * ESE DASD / thin provisioning, is the a new subtype of DASD drives specific to s390 architecture. The fix is to not expose discard availability to the kernel, to prevent utilising discard.

[Other Info]

 * Comments from the thin-provisioning feature bug https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830731

Comment from Jan Hoeppner 2019-09-24 07:16:57 CDT

The thin provisioning feature introduces an IOCTL and the discard support to allow userspace tools and filesystems to release unused and previously allocated space respectively.

During some internal performance improvements and further tests, the release of allocated space revealed some issues that may lead to data corruption in some configurations when filesystems are mounted with discard support enabled.

As we're working on a fix and trying to clarify the situation, it is highly recommended at this point to _not_ enable discard for any filesystem
when working with ESE DASDs.
mkfs should also be called with discard disabled for ESE DASDs
(e.g. mkfs.ext4 -E nodiscard /dev/dasdX).

Fixes and clarifications will be provided as soon as possible.

Regards,
Jan

------- Comment From <email address hidden> 2019-10-01 09:15 EDT-------

In addition to Jan's update to _not_ enable discard support I would recommend to completely revert the patch:

7e64db1597fe114b83fe17d0ba96c6aa5fca419a s390/dasd: Add discard support for ESE volumes

to prevent the user from enabling the discard support for DASD devices.
This patch will also be reverted upstream until further clarification.

Regards,
Stefan

---- xnox ----

Ubuntu enables discard by default in all the installers, all filesystems, and all layers (e.g. mdadm/lvm/luks discard).

Assumption until now, has been that it is harmless, when underlying storage doesn't support discard and always non-harmful when it is supported.

I agree, we should revert discard support for ESE volumes in Ubuntu for now.

Tags: s390x
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1846219

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Dimitri John Ledkov (xnox) wrote :
Revision history for this message
Seth Forshee (sforshee) wrote : ACK/cmt: [PATCH][EOAN] UBUNTU: SAUCE: Revert "s390/dasd: Add discard support for ESE volumes"

On Tue, Oct 01, 2019 at 04:19:07PM +0100, Dimitri John Ledkov wrote:
> BugLink: https://bugs.launchpad.net/bugs/1846219
>
> This reverts commit 7e64db1597fe114b83fe17d0ba96c6aa5fca419a.
>
> Signed-off-by: Dimitri John Ledkov <email address hidden>

The commit message could be more informative ...

I think we should copy some of the information from the bug into the
commit message when applying. But given that it fixes data corruption,
scope is limited to s390, and IBM has tested internally:

Acked-by: Seth Forshee <email address hidden>

Revision history for this message
Seth Forshee (sforshee) wrote : APPLIED: [PATCH][EOAN] UBUNTU: SAUCE: Revert "s390/dasd: Add discard support for ESE volumes"

On Tue, Oct 01, 2019 at 04:19:07PM +0100, Dimitri John Ledkov wrote:
> BugLink: https://bugs.launchpad.net/bugs/1846219
>
> This reverts commit 7e64db1597fe114b83fe17d0ba96c6aa5fca419a.
>
> Signed-off-by: Dimitri John Ledkov <email address hidden>

Applied to eoan/master-next, thanks!

Changed in linux (Ubuntu):
status: Incomplete → Fix Committed
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → Fix Committed
importance: Undecided → Critical
Revision history for this message
Frank Heimes (fheimes) wrote :

Landed in eoan -proposed, hence set to Fix Committed.

Revision history for this message
Frank Heimes (fheimes) wrote :

Transferring comment #19 from LP 1830731 to this bug, too - mainly for sforshee's awareness:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830731

------- Comment From <email address hidden> 2019-10-07 12:18 EDT-------
(In reply to comment #24)
> Stefan, do you also plan to revert it from upstream - or what's the plan
> there?

I saw that you guys reverted the patch with LP1846219 already. But for the sake of completeness, Stefan reverted the patch for upstream as well: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=964ce509e2ded52c1a61ad86044cc4d70abd9eb8

Regards,
Jan

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (7.6 KiB)

This bug was fixed in the package linux - 5.3.0-17.18

---------------
linux (5.3.0-17.18) eoan; urgency=medium

  * eoan/linux: 5.3.0-17.18 -proposed tracker (LP: #1846641)

  * CVE-2019-17056
    - nfc: enforce CAP_NET_RAW for raw sockets

  * CVE-2019-17055
    - mISDN: enforce CAP_NET_RAW for raw sockets

  * CVE-2019-17054
    - appletalk: enforce CAP_NET_RAW for raw sockets

  * CVE-2019-17053
    - ieee802154: enforce CAP_NET_RAW for raw sockets

  * CVE-2019-17052
    - ax25: enforce CAP_NET_RAW for raw sockets

  * CVE-2019-15098
    - ath6kl: fix a NULL-ptr-deref bug in ath6kl_usb_alloc_urb_from_pipe()

  * xHCI on AMD Stoney Ridge cannot detect USB 2.0 or 1.1 devices.
    (LP: #1846470)
    - x86/PCI: Avoid AMD FCH XHCI USB PME# from D0 defect

  * Re-enable linux-libc-dev build on i386 (LP: #1846508)
    - [Packaging] Build only linux-libc-dev for i386
    - [Debian] final-checks -- ignore archtictures with no binaries

  * arm64: loop on boot after installing linux-generic-hwe-18.04-edge/bionic-
    proposed (LP: #1845820)
    - [Config] Disable CONFIG_ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT

  * Revert ESE DASD discard support (LP: #1846219)
    - SAUCE: Revert "s390/dasd: Add discard support for ESE volumes"

  * Miscellaneous Ubuntu changes
    - update dkms package versions

linux (5.3.0-16.17) eoan; urgency=medium

  * eoan/linux: 5.3.0-16.17 -proposed tracker (LP: #1846204)

  * zfs fails to build on s390x with debug symbols enabled (LP: #1846143)
    - SAUCE: s390: Mark atomic const ops always inline

linux (5.3.0-15.16) eoan; urgency=medium

  * eoan/linux: 5.3.0-15.16 -proposed tracker (LP: #1845987)

  * Drop i386 build for 19.10 (LP: #1845714)
    - [Packaging] Remove x32 arch references from control files
    - [Debian] final-checks -- Get arch list from debian/control

  * ZFS kernel modules lack debug symbols (LP: #1840704)
    - [Debian] Fix conditional for setting zfs debug package path

  * Use pyhon3-sphinx instead of python-sphinx for building html docs
    (LP: #1845808)
    - [Packaging] Update sphinx build dependencies to python3 packages

  * Kernel panic with 19.10 beta image (LP: #1845454)
    - efi/tpm: Don't access event->count when it isn't mapped.
    - efi/tpm: don't traverse an event log with no events
    - efi/tpm: only set efi_tpm_final_log_size after successful event log parsing

linux (5.3.0-14.15) eoan; urgency=medium

  * eoan/linux: 5.3.0-14.15 -proposed tracker (LP: #1845728)

  * Drop i386 build for 19.10 (LP: #1845714)
    - [Debian] Remove support for producing i386 kernels
    - [Debian] Don't use CROSS_COMPILE for i386 configs

  * udevadm trigger will fail when trying to add /sys/devices/vio/
    (LP: #1845572)
    - SAUCE: powerpc/vio: drop bus_type from parent device

  * Trying to online dasd drive results in invalid input/output from the kernel
    on z/VM (LP: #1845323)
    - SAUCE: s390/dasd: Fix error handling during online processing

  * intel-lpss driver conflicts with write-combining MTRR region (LP: #1845584)
    - SAUCE: mfd: intel-lpss: add quirk for Dell XPS 13 7390 2-in-1

  * Support Hi1620 zip hw accelerator (LP: #1845355)
    - [Config] Enable HiSilicon QM/ZIP as module...

Read more...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (linux-gcp-5.3/5.3.0-1008.9~18.04.1)

All autopkgtests for the newly accepted linux-gcp-5.3 (5.3.0-1008.9~18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

linux-gcp-5.3/unknown (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-gcp-5.3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.