xfrm interface: several kernel panic

Bug #1836261 reported by Nicolas Dichtel
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
Undecided
Unassigned
Disco
Fix Released
Medium
Connor Kuehl

Bug Description

BugLink: https://bugs.launchpad.net/bugs/1836261

[Impact]

Upstream has recently received a number of bug fixes that resolve kernel panics, memory leaks, and list handling for virtual xfrm interfaces that were debuted in 4.19.

[Test Case]

3 of the 5 patches have test cases in their commit message for reproducing the fault they address.

Another patch prevents including a stale name in the log files.

And the other patch that doesn't have an explicit test case improves list
handling.

[Regression Potential]

This patchset contains a nontrivial amount of changes. However, the heavier patches contain test cases that they resolve the regressions they were created for. They've been upstream since July and I don't see any follow up Fixes commits targeting these. The blast radius is "only" the xfrm interface but this is smoke tested and if any dependents rely on it for core functionality they might exercise it and would possibly notice any issues by now as well.

Original bug description follows:
--------------------------------------------

There was several problems reported upstream:

1/ 56c5ee1a5823: xfrm interface: fix memory leak on creation
https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=56c5ee1a5823

2/ xfrm interface: avoid corruption on changelink
http://patchwork.ozlabs.org/patch/1130240/

3/ xfrm interface: ifname may be wrong in logs
http://patchwork.ozlabs.org/patch/1130241/

4/ xfrm interface: fix list corruption for x-netns
http://patchwork.ozlabs.org/patch/1130385/

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1836261

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Nicolas Dichtel (nicolas-dichtel) wrote :

The series has been included in the ipsec tree:
https://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec.git/log/?h=22d6552f827e

It will hit linus tree soon.

Revision history for this message
Nicolas Dichtel (nicolas-dichtel) wrote :

Here are the official commits from Linus tree:

56c5ee1a5823 ("xfrm interface: fix memory leak on creation")
e9e7e85d75f3 ("xfrm interface: avoid corruption on changelink")
e0aaa332e6a9 ("xfrm interface: ifname may be wrong in logs")
c5d1030f2300 ("xfrm interface: fix list corruption for x-netns")
22d6552f827e ("xfrm interface: fix management of phydev")

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=56c5ee1a5823
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e9e7e85d75f3
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e0aaa332e6a9
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c5d1030f2300
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=22d6552f827e

The bugs have been introduced in v4.19 by commit f203b76d7809 ("xfrm: Add virtual xfrm interfaces"):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f203b76d7809

Connor Kuehl (connork)
Changed in linux (Ubuntu Disco):
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Connor Kuehl (connork)
Revision history for this message
Connor Kuehl (connork) wrote :
description: updated
description: updated
Changed in linux (Ubuntu Disco):
status: In Progress → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-disco
Revision history for this message
Nicolas Dichtel (nicolas-dichtel) wrote :

root@dut-vm:~# ip netns add foo
root@dut-vm:~# ip netns add bar
root@dut-vm:~# ip -n foo netns set bar 0
root@dut-vm:~# ip -n foo link add xfrmi0 link-netnsid 0 type xfrm dev lo if_id 23
root@dut-vm:~# ip -n bar link ls xfrmi0
Device "xfrmi0" does not exist.
root@dut-vm:~# ip -n foo link ls xfrmi0
2: xfrmi0@if1: <NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/none 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff link-netns bar

=> interface is correctly created and there is no backtrace in dmesg, OK

root@dut-vm:~# ip netns del foo
root@dut-vm:~# ip netns del bar
root@dut-vm:~# ip link add xfrm1 type xfrm dev lo if_id 1
root@dut-vm:~# ip link add xfrm2 type xfrm dev lo if_id 2
root@dut-vm:~# ip link set xfrm1 type xfrm dev lo if_id 2
RTNETLINK answers: File exists
root@dut-vm:~# ip -d link list dev xfrm1
7: xfrm1@lo: <NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/none 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 1500
    xfrm if_id 0x1 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

=> if_id 0x1 and no backtrace in dmesg, OK

root@dut-vm:~# ip link add dummy type dummy
root@dut-vm:~# ip link add xfrm1 type xfrm dev dummy if_id 1
root@dut-vm:~# ip l d dummy
root@dut-vm:~# ip -d l ls xfrm1
10: xfrm1@if9: <NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/none 3a:90:dc:59:76:c6 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 1500
    xfrm if_id 0x1 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
root@dut-vm:~#

=> no refcnt pb, OK

Our internal tests suite is also OK.

tags: added: verification-done-disco
removed: verification-needed-disco
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (22.6 KiB)

This bug was fixed in the package linux - 5.0.0-32.34

---------------
linux (5.0.0-32.34) disco; urgency=medium

  * disco/linux: 5.0.0-32.34 -proposed tracker (LP: #1846097)

  * CVE-2019-14814 // CVE-2019-14815 // CVE-2019-14816
    - mwifiex: Fix three heap overflow at parsing element in cfg80211_ap_settings

  * CVE-2019-15505
    - media: technisat-usb2: break out of loop at end of buffer

  * CVE-2019-2181
    - binder: check for overflow when alloc for security context

  * Support Hi1620 zip hw accelerator (LP: #1845355)
    - [Config] Enable HiSilicon QM/ZIP as modules
    - crypto: hisilicon - add queue management driver for HiSilicon QM module
    - crypto: hisilicon - add hardware SGL support
    - crypto: hisilicon - add HiSilicon ZIP accelerator support
    - crypto: hisilicon - add SRIOV support for ZIP
    - Documentation: Add debugfs doc for hisi_zip
    - crypto: hisilicon - add debugfs for ZIP and QM
    - MAINTAINERS: add maintainer for HiSilicon QM and ZIP controller driver
    - crypto: hisilicon - fix kbuild warnings
    - crypto: hisilicon - add dependency for CRYPTO_DEV_HISI_ZIP
    - crypto: hisilicon - init curr_sgl_dma to fix compile warning
    - crypto: hisilicon - add missing single_release
    - crypto: hisilicon - fix error handle in hisi_zip_create_req_q
    - crypto: hisilicon - Fix warning on printing %p with dma_addr_t
    - crypto: hisilicon - Fix return value check in hisi_zip_acompress()
    - crypto: hisilicon - avoid unused function warning

  * xfrm interface: several kernel panic (LP: #1836261)
    - xfrm interface: fix memory leak on creation
    - xfrm interface: avoid corruption on changelink
    - xfrm interface: ifname may be wrong in logs
    - xfrm interface: fix list corruption for x-netns
    - xfrm interface: fix management of phydev

  * shiftfs: drop entries from cache on unlink (LP: #1841977)
    - SAUCE: shiftfs: fix buggy unlink logic

  * shiftfs: mark kmem_cache as reclaimable (LP: #1842059)
    - SAUCE: shiftfs: mark slab objects SLAB_RECLAIM_ACCOUNT

  * Suspend to RAM(S3) does not wake up for latest megaraid and mpt3sas
    adapters(SAS3.5 onwards) (LP: #1838751)
    - PCI: Restore Resizable BAR size bits correctly for 1MB BARs

  * No sound inputs from the external microphone and headset on a Dell machine
    (LP: #1842265)
    - ALSA: hda - Expand pin_match function to match upcoming new tbls
    - ALSA: hda - Define a fallback_pin_fixup_tbl for alc269 family

  * Add -fcf-protection=none when using retpoline flags (LP: #1843291)
    - SAUCE: kbuild: add -fcf-protection=none when using retpoline flags

  * Disco update: upstream stable patchset 2019-09-25 (LP: #1845390)
    - bridge/mdb: remove wrong use of NLM_F_MULTI
    - cdc_ether: fix rndis support for Mediatek based smartphones
    - ipv6: Fix the link time qualifier of 'ping_v6_proc_exit_net()'
    - isdn/capi: check message length in capi_write()
    - ixgbe: Fix secpath usage for IPsec TX offload.
    - net: Fix null de-reference of device refcount
    - net: gso: Fix skb_segment splat when splitting gso_size mangled skb having
      linear-headed frag_list
    - net: phylink: Fix flow control resolution
    - net: s...

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

All these 5 commits could be found in Eoan as well. Close this with Fix Released.

Changed in linux (Ubuntu):
status: Incomplete → Fix Released
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (linux-bluefield/5.0.0-1003.12)

All autopkgtests for the newly accepted linux-bluefield (5.0.0-1003.12) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

fsprotect/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-bluefield

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.