NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 2009325

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

What I have noticed now only occurs at NFS clients, not with the NFS server.

NFS Client Server that create a large amount of simultaneous accesses have the problem.
NFS Client Server that create few accesses do not have the problem.

We have now booted our web servers on an older kernel version and the problem was gone.
Even if the NFS Server servers have the current kernel.

Same symptoms here.

In our case, five 22.04LTS servers with the 5.15.0-60-generic kernel running a web service, all five with an NFS-mounted volume.

After updating some of them with the 5.15.0-67-generic kernel, the traffic on the interface with which the NFS server is accessed increases for no apparent reason. Both output and input packets per second go from about 2k/s to a continuous 12k/s. The context switch rate per second increases by the same ratio. As a result, operations against the NFS volume are slowed down and the user experience is noticeably worse. In the meantime, systems running 5.15.0-60-generic are working as usual.

As in the case of the reporter, booting the systems with 5.15.0-60-generic returns us to the original state.

Hello,

We have the same behavior on 22.04 and 18.04 servers.

This behavior affect us on different on many of our customers web servers with different platforms AWS, Azure and Nutanix (KVM virt) VMs.

The used kernel was :

    linux-image-5.15.0-67-generic
    linux-image-5.4.0-1097-aws
    linux-image-5.4.0-1104-azure

We have also now booted our web servers on an older kernel version and the problem was mostly gone.

We have also tried with the 5.15.0-68-generic on a test server on the https://launchpad.net/%7Ecanonical-kernel-team/+archive/ubuntu/ppa this issue affect also this next kernel.

Best Regards,

We see this problem too on focal clients with ans without HWE.
The server seems to be not affected. Downgrading the clients to 5.15.0-60 or 5.4.0-139 solves the problem here.

Best regards.

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2003053/comments/21

We are seeing this issue with NFSv3. NFS ACCESS operations increased quadratically after upgrading to 5.4.0-144 and 5.15.0-67. Downgrading to either 5.4.0-139-generic or 5.15.0-60-generic resolves the issue again.

The commit you say that you have reverted references NFSv4, which we are not using. Therefore, combined with the massive increase in ACCESS operations, I am inclined to believe that the issue is related to the file access cache-related commits and not the retry lock commit.

In our case (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2009325/comments/25), we also use NFSv3, and like Eirik Rye, we've seen this massive increase in ACCESS operations, from averages of 700/s (approx.) to peaks of 27000/s with 5.15.0-67.

In case it helps.

If the symptom observed is a massive increase in ACCESS operations,
it is more likely that the root cause is related to file access cache-related commits.
I will build a test kernel without those commits for testing.
Thanks for your help.

I have prepared a test kernel that reverts the file access cache-related commits.
Please follow these steps and give it a try:

1. sudo add-apt-repository ppa:chengendu/jammy-nfs-test
2. sudo apt update
3. sudo apt install linux-headers-5.15.0-67-generic linux-image-unsigned-5.15.0-67-generic linux-modules-5.15.0-67-generic linux-modules-extra-5.15.0-67-generic
4. reboot
5. uname -rv
   5.15.0-67-generic #74+test20230307b2h2cbb6062f8eb-Ubuntu SMP Tue Mar 7 23:16:16 UT

In the meantime, we will also work on figuring out the error.

I have sent a patch to fix the upstream error (https://lkml.org/lkml/2023/3/8/107),
which is intended to resolve the issue of a massive increase in ACCESS operations.
I have also prepared a test kernel that includes this patch and would appreciate it if you could use it for testing.

To install the test kernel, please follow these steps:

1. Run the following command: sudo add-apt-repository ppa:chengendu/jammy-nfs-patch
2. Update the packages: sudo apt update
3. Install the kernel packages: sudo apt install linux-headers-5.15.0-67-generic linux-image-unsigned-5.15.0-67-generic linux-modules-5.15.0-67-generic linux-modules-extra-5.15.0-67-generic
4. Reboot your system.
5. To check the installed kernel version, run the command 'uname -rv', which should output '5.15.0-67-generic #74+test20230308b0h1a13a615ee32-Ubuntu SMP Wed Mar 8 09:24:49 UT'.

Hello ChengEn, Du ,

Thank you for the quick response and fix .

We have quickly tried the test kernel ( 5.15.0-67-generic #74+test20230307b2h2cbb6062f8eb-Ubuntu ) on a test webserver (ubuntu 22.04.2 lts ) with an impacted configuration with nfs and db.

The test kernel seems for us to fix the problem, as we have returned to a normal behavior as the previous kernel (5.15.0-60-generic).

Best regards,

The first test kernel (5.15.0-67-generic #74+test20230307b2h2cbb6062f8eb-Ubuntu) merely reverts file access cache-related commits.
However, the second test kernel (5.15.0-67-generic #74+test20230308b0h1a13a615ee32-Ubuntu) truly identifies and resolves the problem.

The file access cache, which belongs to the user, will be cleared once the user logs out and logs in again.
After that, all behavior will be the same as usual.
We would appreciate it if you could use the second test kernel for testing and confirm that it has resolved the performance issue.

Hello ChengEn, Du ,
Thanks for the test Kernel.
I can no longer find a problem with this Kernel and NFS client.
I also think that the kernel has fixed the problem.

Best regards,

Hello ChengEn, Du ,

I have tested also the 2nd.

Same as the previous test kernel , the 2nd (5.15.0-67-generic #74+test20230308b0h1a13a615ee32-Ubuntu) seems to work as intended .

Same as Alexander , this patch seems to fix the problem.

Best regards,

Hi,

I believe I have seen the same problem on 5.19.0-35-generic

It was not present on 5.19.0-32-generic

And the nfs_access_add_cache function looks similar to 5.15.0 so the same fix will probably work.

A problematic patch has been identified in a bug report, which can lead to issues.
The bug report can be found at https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2003053.

This patch can affect several kernels, including:

linux/4.15.0-206.217 for bionic
linux/5.4.0-144.161 for focal
linux/5.15.0-66.73 for jammy
linux/5.19.0-35.36 for kinetic

The solution for this issue is the same for all the kernels mentioned above.
We are currently waiting for upstream to merge the patch.

We apologize for any inconvenience caused.

I've had this issue within AWS using:
linux-aws: 5.15.0-1031
linux-image-5.15.0-1031-aws

Problem goes away with linux-image-5.15.0-1030-aws

This seems to also affect the 5.19.0 kernel train as well (linux-aws-edge on jammy)

This bug was fixed in the package linux - 5.19.0-38.39

---------------
linux (5.19.0-38.39) kinetic; urgency=medium

  * kinetic/linux: 5.19.0-38.39 -proposed tracker (LP: #2012088)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux (5.19.0-37.38) kinetic; urgency=medium

  * kinetic/linux: 5.19.0-37.38 -proposed tracker (LP: #2008305)

  * XPS 9320 screen flicker on UHD panel 3840x2400 (LP: #2007516)
    - SAUCE: drm/i915/display/psr: Disable PSR2 sel fetch on panel SHP 5457

  * Fix speaker mute hotkey doesn't work on Dell G16 series (LP: #2003161)
    - platform/x86: dell-wmi: Add a keymap for KEY_MUTE in type 0x0010 table

  * net:cmsg_ipv6.sh from ubuntu_kernel_selftests failed with K-5.19
    (LP: #2000709)
    - selftest: net: Improve IPV6_TCLASS/IPV6_HOPLIMIT tests apparmor
      compatibility

  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtcpie: Force passing unreliable subtest

  * CVE-2023-0469
    - io_uring/filetable: fix file reference underflow

  * Improve arp_ndisc_evict_nocarrier.sh test result processing (LP: #2006546)
    - selftests: net: return non-zero for failures reported in
      arp_ndisc_evict_nocarrier

  * Make cm32181 sensor work after system suspend (LP: #1981773)
    - iio: light: cm32181: Add PM support

  * Fix Thunderbolt device hotplug fail when connect via thunderbolt dock
    (LP: #1991366)
    - PCI: Fix used_buses calculation in pci_scan_child_bus_extend()
    - PCI: Pass available buses even if the bridge is already configured
    - PCI: Move pci_assign_unassigned_root_bus_resources()
    - PCI: Fix whitespace and indentation
    - PCI: Fix typo in pci_scan_child_bus_extend()

  * arp_ndisc_evict_nocarrier.sh in net from ubuntu_kernel_selftests failed on
    J-oem-5.17 / K (LP: #1968310)
    - selftests: net: fix cleanup_v6() for arp_ndisc_evict_nocarrier

  * Screen freeze after resuming from suspend (nvme0: I/O timeout)
    (LP: #1996048)
    - PCI: vmd: Disable MSI remapping after suspend

  * RaptorLake: Fix the Screen is shaking by onboard HDMI port in mirror mode
    (LP: #1993561)
    - drm/i915/display: Drop check for doublescan mode in modevalid
    - drm/i915/display: Prune Interlace modes for Display >=12

  * udpgro_frglist.sh in net from ubuntu_kernel_selftests failed with K-5.19
    (Missing nat6to4 helper) (LP: #2000708)
    - selftests/net: mv bpf/nat6to4.c to net folder

  * selftests/.../nat6to4 breaks the selftests build (LP: #1996536)
    - selftests: net: Add cross-compilation support for BPF programs
    - selftests: net: Fix O=dir builds

  * CVE-2023-0266 // CVE-2023-0266 was assigned for this issue.
    - ALSA: pcm: Move rwsem lock inside snd_ctl_elem_read to prevent UAF

  * CVE-2022-4382
    - USB: gadgetfs: Fix race between mounting and unmounting

  * CVE-2022-2196
    - KVM: VMX: Execute IBPB on emulated VM-exit when guest has IBRS

  * ubuntu_kernel_selftests: net:udpgso_bench.sh failed (LP: #1951447)
    - selftests: net: udpgso_bench: Fix racing bug between the rx/tx programs

  * Kinetic update: upstream...

This bug was fixed in the package linux-aws - 5.19.0-1022.23

---------------
linux-aws (5.19.0-1022.23) kinetic; urgency=medium

  * kinetic/linux-aws: 5.19.0-1022.23 -proposed tracker (LP: #2011996)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux-aws (5.19.0-1021.22) kinetic; urgency=medium

  * kinetic/linux-aws: 5.19.0-1021.22 -proposed tracker (LP: #2008287)

  [ Ubuntu: 5.19.0-37.38 ]

  * kinetic/linux: 5.19.0-37.38 -proposed tracker (LP: #2008305)
  * XPS 9320 screen flicker on UHD panel 3840x2400 (LP: #2007516)
    - SAUCE: drm/i915/display/psr: Disable PSR2 sel fetch on panel SHP 5457
  * Fix speaker mute hotkey doesn't work on Dell G16 series (LP: #2003161)
    - platform/x86: dell-wmi: Add a keymap for KEY_MUTE in type 0x0010 table
  * net:cmsg_ipv6.sh from ubuntu_kernel_selftests failed with K-5.19
    (LP: #2000709)
    - selftest: net: Improve IPV6_TCLASS/IPV6_HOPLIMIT tests apparmor
      compatibility
  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtcpie: Force passing unreliable subtest
  * CVE-2023-0469
    - io_uring/filetable: fix file reference underflow
  * Improve arp_ndisc_evict_nocarrier.sh test result processing (LP: #2006546)
    - selftests: net: return non-zero for failures reported in
      arp_ndisc_evict_nocarrier
  * Make cm32181 sensor work after system suspend (LP: #1981773)
    - iio: light: cm32181: Add PM support
  * Fix Thunderbolt device hotplug fail when connect via thunderbolt dock
    (LP: #1991366)
    - PCI: Fix used_buses calculation in pci_scan_child_bus_extend()
    - PCI: Pass available buses even if the bridge is already configured
    - PCI: Move pci_assign_unassigned_root_bus_resources()
    - PCI: Fix whitespace and indentation
    - PCI: Fix typo in pci_scan_child_bus_extend()
  * arp_ndisc_evict_nocarrier.sh in net from ubuntu_kernel_selftests failed on
    J-oem-5.17 / K (LP: #1968310)
    - selftests: net: fix cleanup_v6() for arp_ndisc_evict_nocarrier
  * Screen freeze after resuming from suspend (nvme0: I/O timeout)
    (LP: #1996048)
    - PCI: vmd: Disable MSI remapping after suspend
  * RaptorLake: Fix the Screen is shaking by onboard HDMI port in mirror mode
    (LP: #1993561)
    - drm/i915/display: Drop check for doublescan mode in modevalid
    - drm/i915/display: Prune Interlace modes for Display >=12
  * udpgro_frglist.sh in net from ubuntu_kernel_selftests failed with K-5.19
    (Missing nat6to4 helper) (LP: #2000708)
    - selftests/net: mv bpf/nat6to4.c to net folder
  * selftests/.../nat6to4 breaks the selftests build (LP: #1996536)
    - selftests: net: Add cross-compilation support for BPF programs
    - selftests: net: Fix O=dir builds
  * CVE-2023-0266 // CVE-2023-0266 was assigned for this issue.
    - ALSA: pcm: Move rwsem lock inside snd_ctl_elem_read to prevent UAF
  * CVE-2022-4382
    - USB: gadgetfs: Fix race between mounting and unmounting
  * CVE-2022-2196
    - KVM: VMX: Execute IBPB on emulated VM-exit when guest has IBRS
  * ubuntu_kernel_selftests: net:udpgso_bench.sh failed (LP: #1951447)
...

Any idea when a new kernel package for Focal (20.04) will be made available? Thanks!

@ChengEn, Du thank you for the quick fix commit!!

I was curious if ya'll had any idea on how long it will take this change to specifically hit the focal and bionic linux releases?

And if not, do we have a general idea on how long this process usually takes?

Thanks again!

Hello,

The Canonical Kernel Team is planning to release the affected kernels in Jammy (22.04), Focal (20.04) and Bionic (18.04) early next week.

Thank you everyone for helping with tests and additional information.

Why is this not recognised as critical? We identified this as a major incident, with severe loss of service (availability) and likewise opening a risk assessment in relation to our dependency on Ubuntu.

Not only is it about severe loss of service, but this also keeps numerous systems at risk because they cannot easily install a kernel with the security fixes.

For the future, please consider not bundling security releases with other fixes, as to avoid this situation where systems essentially remain unpatched for a month.

Thanks for the update @kleber-souza!

Would early next week be Monday/Tuesday or is it possible it bleeds into Wednesday/Thursday morning/etc? We are also doing a risk assessment and knowing this will greatly help our cause.

Thanks in advance!

Even worse, is it even possible to downgrade? I couldn't see any viable option via madison.

linux-image-aws | 5.15.0.1031.35~20.04.20 | http://us-east-1.ec2.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
linux-image-aws | 5.15.0.1031.35~20.04.20 | http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
linux-image-aws | 5.4.0.1009.11 | http://us-east-1.ec2.archive.ubuntu.com/ubuntu focal/main amd64 Packages

I do welcome the release next week though.

This bug was fixed in the package linux - 5.4.0-146.163

---------------
linux (5.4.0-146.163) focal; urgency=medium

  * focal/linux: 5.4.0-146.163 -proposed tracker (LP: #2012094)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux (5.4.0-145.162) focal; urgency=medium

  * focal/linux: 5.4.0-145.162 -proposed tracker (LP: #2008389)

  * [SRU]Update ice driver to support E823 devices (LP: #1986717)
    - ice: Add device ids for E822 devices
    - ice: add support for E823 devices

  * btrfs/154: rename fails with EOVERFLOW when calculating item size during
    item key collision (LP: #2004132)
    - btrfs: correctly calculate item size used when item key collision happens

  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtcpie: Force passing unreliable subtest

  * [UBUNTU 20.04] KVM: s390: pv: don't allow userspace to set the clock under
    PV - kernel part (LP: #1999882)
    - KVM: s390x: fix SCK locking
    - KVM: s390: pv: don't allow userspace to set the clock under PV

  * CVE-2021-3669
    - ipc: replace costly bailout check in sysvipc_find_ipc()

  * net:fcnal-test.sh 'nettest' command not found on F/K (LP: #2006391)
    - selftests/net: Find nettest in current directory

  * xfs: Preallocated ioend transactions cause deadlock due to log buffer
    exhaustion (LP: #2007219)
    - xfs: drop submit side trans alloc for append ioends

  * CVE-2022-4382
    - USB: gadgetfs: Fix race between mounting and unmounting

  * CVE-2022-2196
    - KVM: VMX: Execute IBPB on emulated VM-exit when guest has IBRS

  * ubuntu_kernel_selftests: net:udpgso_bench.sh failed (LP: #1951447)
    - selftests: net: udpgso_bench: Fix racing bug between the rx/tx programs

  * net:fcnal-test.sh didn't return a non-zero value even with some sub-tests
    failed (LP: #2006692)
    - selftests: net/fcnal-test.sh: add exit code

  * Fix selftests/ftracetests/Meta-selftests in Focal (LP: #2006453)
    - SAUCE: Fix ftrace/Meta-selftests bashism check

  * CVE-2023-23559
    - wifi: rndis_wlan: Prevent buffer overflow in rndis_query_oid

 -- Luke Nowakowski-Krijger <email address hidden> Fri, 17 Mar 2023 11:08:20 -0700

This bug was fixed in the package linux-aws - 5.4.0-1099.107

---------------
linux-aws (5.4.0-1099.107) focal; urgency=medium

  * focal/linux-aws: 5.4.0-1099.107 -proposed tracker (LP: #2012000)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux-aws (5.4.0-1098.106) focal; urgency=medium

  * focal/linux-aws: 5.4.0-1098.106 -proposed tracker (LP: #2008358)

  [ Ubuntu: 5.4.0-145.162 ]

  * focal/linux: 5.4.0-145.162 -proposed tracker (LP: #2008389)
  * [SRU]Update ice driver to support E823 devices (LP: #1986717)
    - ice: Add device ids for E822 devices
    - ice: add support for E823 devices
  * btrfs/154: rename fails with EOVERFLOW when calculating item size during
    item key collision (LP: #2004132)
    - btrfs: correctly calculate item size used when item key collision happens
  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtcpie: Force passing unreliable subtest
  * [UBUNTU 20.04] KVM: s390: pv: don't allow userspace to set the clock under
    PV - kernel part (LP: #1999882)
    - KVM: s390x: fix SCK locking
    - KVM: s390: pv: don't allow userspace to set the clock under PV
  * CVE-2021-3669
    - ipc: replace costly bailout check in sysvipc_find_ipc()
  * net:fcnal-test.sh 'nettest' command not found on F/K (LP: #2006391)
    - selftests/net: Find nettest in current directory
  * xfs: Preallocated ioend transactions cause deadlock due to log buffer
    exhaustion (LP: #2007219)
    - xfs: drop submit side trans alloc for append ioends
  * CVE-2022-4382
    - USB: gadgetfs: Fix race between mounting and unmounting
  * CVE-2022-2196
    - KVM: VMX: Execute IBPB on emulated VM-exit when guest has IBRS
  * ubuntu_kernel_selftests: net:udpgso_bench.sh failed (LP: #1951447)
    - selftests: net: udpgso_bench: Fix racing bug between the rx/tx programs
  * net:fcnal-test.sh didn't return a non-zero value even with some sub-tests
    failed (LP: #2006692)
    - selftests: net/fcnal-test.sh: add exit code
  * Fix selftests/ftracetests/Meta-selftests in Focal (LP: #2006453)
    - SAUCE: Fix ftrace/Meta-selftests bashism check
  * CVE-2023-23559
    - wifi: rndis_wlan: Prevent buffer overflow in rndis_query_oid

 -- Kleber Sacilotto de Souza <email address hidden> Fri, 17 Mar 2023 11:53:44 +0100

This bug was fixed in the package linux - 5.15.0-69.76

---------------
linux (5.15.0-69.76) jammy; urgency=medium

  * jammy/linux: 5.15.0-69.76 -proposed tracker (LP: #2012092)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux (5.15.0-68.75) jammy; urgency=medium

  * jammy/linux: 5.15.0-68.75 -proposed tracker (LP: #2008349)

  * Packaging resync (LP: #1786013)
    - debian/dkms-versions -- update from kernel-versions (main/2023.02.27)

  * Ubuntu 22.04 kernel 5.15.0-46-generic leaks kernel memory in kmalloc-2k
    slabs (LP: #1987430)
    - SAUCE: audit: fix memory leak of audit_log_lsm()

  * [EGS] Backport intel_idle support for Eagle Stream Ubuntu 22.04 release
    (LP: #2003267)
    - intel_idle: add SPR support
    - intel_idle: add 'preferred_cstates' module argument
    - intel_idle: add core C6 optimization for SPR
    - cpuidle: intel_idle: Drop redundant backslash at line end
    - intel_idle: Fix the 'preferred_cstates' module parameter
    - intel_idle: Fix SPR C6 optimization
    - intel_idle: make SPR C1 and C1E be independent

  * Fix speaker mute hotkey doesn't work on Dell G16 series (LP: #2003161)
    - platform/x86: dell-wmi: Add a keymap for KEY_MUTE in type 0x0010 table

  * Fix the ACPI _CPC not found error from kernel dmesg on some dynamic SSDT
    table loaded firmwares (LP: #2006077)
    - ACPI: bus: Avoid using CPPC if not supported by firmware
    - ACPI: bus: Set CPPC _OSC bits for all and when CPPC_LIB is supported
    - ACPI: CPPC: Only probe for _CPC if CPPC v2 is acked

  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtcpie: Force passing unreliable subtest

  * Jammy update: v5.15.87 upstream stable release (LP: #2007441)
    - usb: dwc3: qcom: Fix memory leak in dwc3_qcom_interconnect_init
    - cifs: fix oops during encryption
    - nvme-pci: fix doorbell buffer value endianness
    - nvme-pci: fix mempool alloc size
    - nvme-pci: fix page size checks
    - ACPI: resource: do IRQ override on LENOVO IdeaPad
    - ACPI: resource: do IRQ override on XMG Core 15
    - ACPI: resource: do IRQ override on Lenovo 14ALC7
    - block, bfq: fix uaf for bfqq in bfq_exit_icq_bfqq
    - ata: ahci: Fix PCS quirk application for suspend
    - nvme: fix the NVME_CMD_EFFECTS_CSE_MASK definition
    - nvmet: don't defer passthrough commands with trivial effects to the
      workqueue
    - fs/ntfs3: Validate BOOT record_size
    - fs/ntfs3: Add overflow check for attribute size
    - fs/ntfs3: Validate data run offset
    - fs/ntfs3: Add null pointer check to attr_load_runs_vcn
    - fs/ntfs3: Fix memory leak on ntfs_fill_super() error path
    - fs/ntfs3: Add null pointer check for inode operations
    - fs/ntfs3: Validate attribute name offset
    - fs/ntfs3: Validate buffer length while parsing index
    - fs/ntfs3: Validate resident attribute name
    - fs/ntfs3: Fix slab-out-of-bounds read in run_unpack
    - soundwire: dmi-quirks: add quirk variant for LAPBC710 NUC15
    - fs/ntfs3: Validate index root when initialize NTFS security
    - fs/ntfs3: Use __G...

This bug was fixed in the package linux-aws - 5.15.0-1033.37

---------------
linux-aws (5.15.0-1033.37) jammy; urgency=medium

  * jammy/linux-aws: 5.15.0-1033.37 -proposed tracker (LP: #2011997)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux-aws (5.15.0-1032.36) jammy; urgency=medium

  * jammy/linux-aws: 5.15.0-1032.36 -proposed tracker (LP: #2008308)

  * Packaging resync (LP: #1786013)
    - debian/dkms-versions -- update from kernel-versions (main/2023.02.27)

  [ Ubuntu: 5.15.0-68.75 ]

  * jammy/linux: 5.15.0-68.75 -proposed tracker (LP: #2008349)
  * Packaging resync (LP: #1786013)
    - debian/dkms-versions -- update from kernel-versions (main/2023.02.27)
  * Ubuntu 22.04 kernel 5.15.0-46-generic leaks kernel memory in kmalloc-2k
    slabs (LP: #1987430)
    - SAUCE: audit: fix memory leak of audit_log_lsm()
  * [EGS] Backport intel_idle support for Eagle Stream Ubuntu 22.04 release
    (LP: #2003267)
    - intel_idle: add SPR support
    - intel_idle: add 'preferred_cstates' module argument
    - intel_idle: add core C6 optimization for SPR
    - cpuidle: intel_idle: Drop redundant backslash at line end
    - intel_idle: Fix the 'preferred_cstates' module parameter
    - intel_idle: Fix SPR C6 optimization
    - intel_idle: make SPR C1 and C1E be independent
  * Fix speaker mute hotkey doesn't work on Dell G16 series (LP: #2003161)
    - platform/x86: dell-wmi: Add a keymap for KEY_MUTE in type 0x0010 table
  * Fix the ACPI _CPC not found error from kernel dmesg on some dynamic SSDT
    table loaded firmwares (LP: #2006077)
    - ACPI: bus: Avoid using CPPC if not supported by firmware
    - ACPI: bus: Set CPPC _OSC bits for all and when CPPC_LIB is supported
    - ACPI: CPPC: Only probe for _CPC if CPPC v2 is acked
  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtcpie: Force passing unreliable subtest
  * Jammy update: v5.15.87 upstream stable release (LP: #2007441)
    - usb: dwc3: qcom: Fix memory leak in dwc3_qcom_interconnect_init
    - cifs: fix oops during encryption
    - nvme-pci: fix doorbell buffer value endianness
    - nvme-pci: fix mempool alloc size
    - nvme-pci: fix page size checks
    - ACPI: resource: do IRQ override on LENOVO IdeaPad
    - ACPI: resource: do IRQ override on XMG Core 15
    - ACPI: resource: do IRQ override on Lenovo 14ALC7
    - block, bfq: fix uaf for bfqq in bfq_exit_icq_bfqq
    - ata: ahci: Fix PCS quirk application for suspend
    - nvme: fix the NVME_CMD_EFFECTS_CSE_MASK definition
    - nvmet: don't defer passthrough commands with trivial effects to the
      workqueue
    - fs/ntfs3: Validate BOOT record_size
    - fs/ntfs3: Add overflow check for attribute size
    - fs/ntfs3: Validate data run offset
    - fs/ntfs3: Add null pointer check to attr_load_runs_vcn
    - fs/ntfs3: Fix memory leak on ntfs_fill_super() error path
    - fs/ntfs3: Add null pointer check for inode operations
    - fs/ntfs3: Validate attribute name offset
    - fs/ntfs3: Validate buffer length while parsing index
    - fs/ntfs3: Validate resident ...

Can confirm that 5.15.0-1033 seems to have resolved the NFS bug on Jammy running on AWS.

Is there a timeframe for the same patch to be applied to the 5.19 kernel train?

Confirming the proposed 5.4.0-1099 packages resolve the NFS bug on Bionic running in AWS.

This bug was fixed in the package linux - 4.15.0-208.220

---------------
linux (4.15.0-208.220) bionic; urgency=medium

  * bionic/linux: 4.15.0-208.220 -proposed tracker (LP: #2012097)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux (4.15.0-207.218) bionic; urgency=medium

  * bionic/linux: 4.15.0-207.218 -proposed tracker (LP: #2008419)

  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtctest: Force passing unreliable subtest

  * btrfs/154: rename fails with EOVERFLOW when calculating item size during
    item key collision (LP: #2004132)
    - btrfs: correctly calculate item size used when item key collision happens

  * CVE-2021-3669
    - ipc: replace costly bailout check in sysvipc_find_ipc()

  * Bionic update: upstream stable patchset 2023-02-06 (LP: #2006403)
    - libtraceevent: Fix build with binutils 2.35
    - once: Fix panic when module unload
    - once: add DO_ONCE_SLOW() for sleepable contexts
    - mm/khugepaged: fix GUP-fast interaction by sending IPI
    - mm/khugepaged: invoke MMU notifiers in shmem/file collapse paths
    - block: unhash blkdev part inode when the part is deleted
    - ASoC: ops: Check bounds for second channel in snd_soc_put_volsw_sx()
    - can: sja1000: fix size of OCR_MODE_MASK define
    - can: mcba_usb: Fix termination command argument
    - ASoC: ops: Correct bounds check for second channel on SX controls
    - perf script python: Remove explicit shebang from tests/attr.c
    - udf: Discard preallocation before extending file with a hole
    - udf: Drop unused arguments of udf_delete_aext()
    - udf: Fix preallocation discarding at indirect extent boundary
    - udf: Do not bother looking for prealloc extents if i_lenExtents matches
      i_size
    - udf: Fix extending file within last block
    - usb: gadget: uvc: Prevent buffer overflow in setup handler
    - USB: serial: option: add Quectel EM05-G modem
    - USB: serial: cp210x: add Kamstrup RF sniffer PIDs
    - igb: Initialize mailbox message for VF reset
    - net: loopback: use NET_NAME_PREDICTABLE for name_assign_type
    - usb: musb: remove extra check in musb_gadget_vbus_draw
    - ARM: dts: qcom: apq8064: fix coresight compatible
    - drivers: soc: ti: knav_qmss_queue: Mark knav_acc_firmwares as static
    - arm: dts: spear600: Fix clcd interrupt
    - soc: ti: smartreflex: Fix PM disable depth imbalance in omap_sr_probe
    - arm64: dts: mediatek: mt6797: Fix 26M oscillator unit name
    - ARM: dts: dove: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-370: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-xp: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-375: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-38x: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-39x: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: turris-omnia: Add ethernet aliases
    - ARM: dts: turris-omnia: Add switch port 6 node
    - pstore/ram: Fix error return code in...

This bug was fixed in the package linux-aws - 4.15.0-1153.166

---------------
linux-aws (4.15.0-1153.166) bionic; urgency=medium

  * bionic/linux-aws: 4.15.0-1153.166 -proposed tracker (LP: #2012003)

  * NFS deathlock with last Kernel 5.4.0-144.161 and 5.15.0-67.74 (LP: #2009325)
    - NFS: Correct timing for assigning access cache timestamp

linux-aws (4.15.0-1152.165) bionic; urgency=medium

  * bionic/linux-aws: 4.15.0-1152.165 -proposed tracker (LP: #2008397)

  [ Ubuntu: 4.15.0-207.218 ]

  * bionic/linux: 4.15.0-207.218 -proposed tracker (LP: #2008419)
  * rtcpie in timers from ubuntu_kernel_selftests randomly failing
    (LP: #1814234)
    - SAUCE: selftest: rtctest: Force passing unreliable subtest
  * btrfs/154: rename fails with EOVERFLOW when calculating item size during
    item key collision (LP: #2004132)
    - btrfs: correctly calculate item size used when item key collision happens
  * CVE-2021-3669
    - ipc: replace costly bailout check in sysvipc_find_ipc()
  * Bionic update: upstream stable patchset 2023-02-06 (LP: #2006403)
    - libtraceevent: Fix build with binutils 2.35
    - once: Fix panic when module unload
    - once: add DO_ONCE_SLOW() for sleepable contexts
    - mm/khugepaged: fix GUP-fast interaction by sending IPI
    - mm/khugepaged: invoke MMU notifiers in shmem/file collapse paths
    - block: unhash blkdev part inode when the part is deleted
    - ASoC: ops: Check bounds for second channel in snd_soc_put_volsw_sx()
    - can: sja1000: fix size of OCR_MODE_MASK define
    - can: mcba_usb: Fix termination command argument
    - ASoC: ops: Correct bounds check for second channel on SX controls
    - perf script python: Remove explicit shebang from tests/attr.c
    - udf: Discard preallocation before extending file with a hole
    - udf: Drop unused arguments of udf_delete_aext()
    - udf: Fix preallocation discarding at indirect extent boundary
    - udf: Do not bother looking for prealloc extents if i_lenExtents matches
      i_size
    - udf: Fix extending file within last block
    - usb: gadget: uvc: Prevent buffer overflow in setup handler
    - USB: serial: option: add Quectel EM05-G modem
    - USB: serial: cp210x: add Kamstrup RF sniffer PIDs
    - igb: Initialize mailbox message for VF reset
    - net: loopback: use NET_NAME_PREDICTABLE for name_assign_type
    - usb: musb: remove extra check in musb_gadget_vbus_draw
    - ARM: dts: qcom: apq8064: fix coresight compatible
    - drivers: soc: ti: knav_qmss_queue: Mark knav_acc_firmwares as static
    - arm: dts: spear600: Fix clcd interrupt
    - soc: ti: smartreflex: Fix PM disable depth imbalance in omap_sr_probe
    - arm64: dts: mediatek: mt6797: Fix 26M oscillator unit name
    - ARM: dts: dove: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-370: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-xp: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-375: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-38x: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: armada-39x: Fix assigned-addresses for every PCIe Root Port
    - ARM: dts: turris-omnia...

While the operation rates are drastically improved compared to the broken kernel, even with the the patched kernel 5.15.0-69.76 we're still seeing a large increase in NFS ACCESS operations, up from about 90/s to 1500/s over a couple servers. Additionally, about a 5x increase in LOOKUP operations is observed.

Hi,

I would like to share my thoughts on the increase in LOOKUP operations, which may be attributed to a new mechanism introduced in the Linux 6.2-rc3 NFS client side.
This mechanism clears the access cache as soon as the cache timestamp becomes older than the user's login time.
Its primary objective is to ensure that the NFS client's access cache doesn't become stale due to any changes made to the user's group membership on the server after the user has already logged in on the client.

It's worth noting that POSIX only refreshes the user's supplementary group information upon login.
Upstream has considered that it's not unreasonable for users to expect the access cache to be cleared when they log out and log back in again,
with all behavior returning to normal after the replacement.
You can find all the details logged in the commit 0eb43812c027 (NFS: Clear the file access cache upon login).

If you have any concerns or experience unexpected behavior, please don't hesitate to inform us.
We are always ready to discuss and collaborate with upstream to resolve any issues.
Thank you for taking the time to test and provide us with valuable feedback.

Du ChengEn, thank-you for your additional comments. We are seeing good behaviour after the patch.
However, (and please feel free to call me stupid) when you talk about user's login time, how does this reflect with system processes, such as www-data (apache2), or similar? Is a "login" for such users a systemctl start, restart or reload?

I am asking specifically regarding access cache; our particular issue being apache2.

I think Ben's question could be clarified with:

How does "login" relate to non-login (non-interactive) processes started automatically via system processes?

Our issue is with suexec-like mechanisms, and the 2 orders of magnitude increase ACCESS and 1 order of magnitude for LOOKUP, while significantly better than with the bugs introduced a month ago, are still posing significant problems for applying this kernel upgrade to production systems.

The patch to the upstream release candidate does not seem to have a complete line of reasoning to me, nor is it obvious why this should be backported to stable kernels, but I'm not well versed in interpreting kernel patches.

Hi Du ChengEn,

Thanks for your efforts in making this change for this SRU cycle.

However, I can confirm that we too are facing challenges with orders of magnitude (>10x) increases in ACCESS calls, with the newly released kernels. Still better/differently than 5.15.0-67, but not suitable for production.

Kernel 5.15.0-69 behaves significantly different than test kernel 5.15.0-67.74+test20230307b2h2cbb6062f8eb. I do not think 5.15.0-67.74+test20230308b0h1a13a615ee3 was ever available from your PPA ?. We ran 5.15.0-67.74+test20230307b2h2cbb6062f8eb for 2 weeks, without issues.

What looks like a similar (apache2) environment to Jan and Ben, we have short(er) lived procs with suexec-like setup relying heavily on caching using nocto and increased AC-timeouts options.

While I understand the reasoning for avoiding stale access caches, and clearing it in relation to group-member changes, I do not think it was ever the intention to break default long-time behaviour.

As of now I cannot confirm issues with increased LOOKUPs in our end as our caches are still warming.

I apologize for any confusion that may have arisen.
The test kernel 5.15.0-67.74+test20230307b2h2cbb6062f8eb has completely reverted the new mechanism, resulting in the same behavior as before.
We understand that this can be concerning, and we want to assure you that we are actively working towards finding a possible solution to this issue.

It's important to note that the new mechanism was designed upstream, which means that it may take some time for us to discuss and develop a new approach.
We kindly request your patience and understanding as we work towards a resolution.

We genuinely appreciate your patience and understanding during this time.
Your feedback is important to us, and we will keep you informed of any developments as they arise.

Thanks again, but why is it not possible to roll back the NFS change and roll out only the security relevant content in the meanwhile?

Nearly one month and counting for balancing the need for systems being up vs. having security updates is not quite tenable for most system administrators.

Hi Du ChengEn,

Thanks for the clarification on test kernels. I am sad, I did not get to test your 2nd test kernel, as that would have revealed this issue immediately.

While the changes to fix LP: #2003053, went in there to fix a "bug" - it also created this major regression. But the bug fix just tries to get around the metadata inconsistencies, that has always been an issue with NFS (by design). Not a hard case IHMO, and also "rare" as pointed out in the original commit message.

I follow Jan here, that these changes should be reverted, as they create more problems than they tried to fix.

That does not stop anybody from working on an upstream solution, that would fit both purposes.

For whatever it is worth, this reproducer points out the pain point:

$ cd /nfsdir
$ touch myfiles.{1..1000}
$ md5sum myfiles.{1..1000} > /dev/null
$ sudo -u <myself> md5sum myfiles.{1..1000} > /dev/null

The latter 'md5sum' command produces 1000 GETATTR and 1000 ACCESS calls. The GEATTRs are there to ensure close-to-open consistency - fine. However, ACCESS calls should not be produced in this case, which shows the issue.

This is even more outspoken, if this is mounted using 'nocto' (as we do). The GETATTRs are gone (as they should). The ACCESS calls, however remain. This is where we get that huge increase in numbers.

With regards to potential increase in LOOKUPs - this is still to early. Caches are still not settled (lightly loaded system)

So, where do we go from here - create a new bug ? Give feedback on the linux-nfs mail list ?

I can see you already made a post to the mail list - I will be happy to be of assistance, if you see fit.

This bug is awaiting verification that the linux-aws/5.19.0-1023.24 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-azure/5.19.0-1023.24 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Hi Du ChengEn,

Is there a better place to track the fix to the overall problem with the newer kernel? We needed to revert until we can get a more long term fix in the newer kernel as the current fix does not fully address our problem.

Thanks!

This bug is awaiting verification that the linux-gcp/5.4.0-1103.112 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-aws/5.4.0-1100.108 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-azure/5.4.0-1106.112 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-azure/5.15.0-1036.43 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-azure-4.15/4.15.0-1163.178 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-oracle/5.4.0-1099.108 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-aws/5.15.0-1034.38 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

I would like to begin by expressing my sincere appreciation for your invaluable assistance in identifying the performance issue and presenting us with a clear reproducer for testing.
Your contributions have been extremely helpful.

We have carefully considered the possibility of reverting the NFS-related patches and have consulted with both our kernel team and specialists.
While we acknowledge that the patch does address a functional error as outlined in LP#2003053, we also recognize that the bug responsible for the misbehavior of the new mechanism has been fixed. However, we understand that the current impact on performance resulting from the additional traffic is a significant concern.

Despite the fact that the new mechanism adheres to the principles of POSIX design, we are faced with the challenge of devising an appropriate approach to address this issue.
We understand that the new mechanism in NFS is having an impact on your production environment and we are fully committed to working with upstream to resolve the issue.
As soon as we and upstream reach a consensus, we will open a new bug to address the performance issue.

We sincerely apologize for any inconvenience caused by this issue,
and we appreciate your patience and understanding as we work towards a solution that is both effective and aligned with POSIX design principles.

Hi Du ChengEn,

Thanks for you feedback, and understanding of our issue.

I will be watching the nfs mailing list as well, but kindly post references to the bug here, once the bug is opened.

I support your idea, of a separate mount-option, if it is not possible to address both issues.

Looking at the overall impact of this in our environment, we seem to be trading some GETATTRs for ACCESS calls. While the isolated increase in ACCESS calls is very high, the combined GETATTR+ACCESS is not as high. However, we are still talking about 4-5x here - 3-400% increase.

As our workloads are very meta-data intensive it will push our NFS servers into unacceptable load levels (~50k -> ~200k NFS ops).

Good news with regards to suggested increases in LOOKUPs, though. I cannot confirm any issues here. I have been running 5.15.69, 5.15.60 and your 1st test kernel (5.15.0-67-generic #74+test20230307b2h2cbb6062f8eb) for days. Once caches were settled, they are all hovering at the same levels of LOOKUP calls. So I see no regression on this matter.

Thanks in advance.

This bug is awaiting verification that the linux-dell300x/4.15.0-1063.68 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gcp/5.15.0-1032.40 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-aws/4.15.0-1154.167 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-kvm/5.4.0-1089.95 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-kvm/5.15.0-1031.36 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

we have tested on 5.4.0-1099-aws and this issue is fixed

This bug is awaiting verification that the linux-kvm/5.4.0-1089.95 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gcp/5.15.0-1032.40 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-raspi/5.19.0-1016.23 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

For Jammy

5.19.0-38-generic
and
5.15.0-70-generic

Fixed my problems with nfs client performance.

Thanks!

Should I change the tags the ubuntu-kernel-bot is talking about? verification-needed-jammy to verification-done-jammy?

I'm assuming not but I wouldn't want to see this bug closed and the fix dropped.

This bug was fixed in the package linux - 6.2.0-19.19

---------------
linux (6.2.0-19.19) lunar; urgency=medium

  * lunar/linux: 6.2.0-19.19 -proposed tracker (LP: #2012488)

  * Neuter signing tarballs (LP: #2012776)
    - [Packaging] neuter the signing tarball

  * LSM stacking and AppArmor refresh for 6.2 kernel (LP: #2012136)
    - Revert "UBUNTU: [Config] define CONFIG_SECURITY_APPARMOR_RESTRICT_USERNS"
    - Revert "UBUNTU: SAUCE: apparmor: add user namespace creation mediation"
    - Revert "UBUNTU: SAUCE: apparmor: Add fine grained mediation of posix
      mqueues"
    - Revert "UBUNTU: SAUCE: Revert "apparmor: make __aa_path_perm() static""
    - Revert "UBUNTU: SAUCE: LSM: Specify which LSM to display (using struct cred
      as input)"
    - Revert "UBUNTU: SAUCE: apparmor: Fix build error, make sk parameter const"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in smk_netlbl_mls()"
    - Revert "UBUNTU: SAUCE: LSM: change ima_read_file() to use lsmblob"
    - Revert "UBUNTU: SAUCE: apparmor: rename kzfree() to kfree_sensitive()"
    - Revert "UBUNTU: SAUCE: AppArmor: Remove the exclusive flag"
    - Revert "UBUNTU: SAUCE: LSM: Add /proc attr entry for full LSM context"
    - Revert "UBUNTU: SAUCE: Audit: Fix incorrect static inline function
      declration."
    - Revert "UBUNTU: SAUCE: Audit: Fix for missing NULL check"
    - Revert "UBUNTU: SAUCE: Audit: Add a new record for multiple object LSM
      attributes"
    - Revert "UBUNTU: SAUCE: Audit: Add new record for multiple process LSM
      attributes"
    - Revert "UBUNTU: SAUCE: NET: Store LSM netlabel data in a lsmblob"
    - Revert "UBUNTU: SAUCE: LSM: security_secid_to_secctx in netlink netfilter"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmcontext in security_inode_getsecctx"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmcontext in security_secid_to_secctx"
    - Revert "UBUNTU: SAUCE: LSM: Ensure the correct LSM context releaser"
    - Revert "UBUNTU: SAUCE: LSM: Specify which LSM to display"
    - Revert "UBUNTU: SAUCE: IMA: Change internal interfaces to use lsmblobs"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_cred_getsecid"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_inode_getsecid"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_task_getsecid"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_ipc_getsecid"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_secid_to_secctx"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_secctx_to_secid"
    - Revert "UBUNTU: SAUCE: net: Prepare UDS for security module stacking"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_kernel_act_as"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_audit_rule_match"
    - Revert "UBUNTU: SAUCE: LSM: Create and manage the lsmblob data structure."
    - Revert "UBUNTU: SAUCE: LSM: Infrastructure management of the sock security"
    - Revert "UBUNTU: SAUCE: apparmor: LSM stacking: switch from SK_CTX() to
      aa_sock()"
    - Revert "UBUNTU: SAUCE: apparmor: rename aa_sock() to aa_unix_sk()"
    - Revert "UBUNTU: SAUCE: apparmor: disable showing the mode as part of a secid
      to secctx"
    - Revert "UBUNTU: SAUCE: app...

This bug is awaiting verification that the linux-kvm/4.15.0-1138.143 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-raspi/5.4.0-1083.94 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-raspi/5.15.0-1027.29 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-oracle/5.19.0-1020.23 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gkeop/5.15.0-1018.23 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-ibm/5.4.0-1047.52 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gcp/5.19.0-1020.22 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-oracle/4.15.0-1117.128 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-raspi2/4.15.0-1130.138 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-ibm/5.15.0-1028.31 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-oracle/5.15.0-1033.39 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Hi ChengEn, Du,

Do we have another issue opened to address the NFS mechanism and the performance impacts it has on the kernel? We cannot move to the new kernel currently because of these issues so I just want to make sure we are continuing to push a solution forward. Thank you!

This bug is awaiting verification that the linux-riscv/5.19.0-1016.17 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-starfive/5.19.0-1015.17 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-allwinner/5.19.0-1010.10 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gke/5.15.0-1031.36 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Hi @krglosse,

We have opened a bug to address the performance issue related to NFS.
You can find more details about the bug at this link: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2015827.

We have also sent a proposal to upstream for their consideration, but unfortunately, we have not received a response yet.
As a result, we may need to send a patch directly and await upstream's feedback and judgement.

This bug is awaiting verification that the linux-kvm/5.19.0-1021.22 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Hi @chengendu,

Thanks.

I have posted a comment to the effect of backporting changes in new (and at the time, unreleased and not yet stable) Linux kernels to LTS releases. There is no need to await upstream's feedback and judgment, this change belongs to Linux 6.2, not old LTS kernels.

This bug is awaiting verification that the linux-ibm/5.19.0-1020.22 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-kinetic' to 'verification-done-kinetic'. If the problem still exists, change the tag 'verification-needed-kinetic' to 'verification-failed-kinetic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gkeop/5.4.0-1067.71 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gcp-4.15/4.15.0-1148.164 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-gke/5.4.0-1097.104 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-nvidia-5.19/5.19.0-1009.9 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug was fixed in the package linux-aws - 6.2.0-1003.3

---------------
linux-aws (6.2.0-1003.3) lunar; urgency=medium

  * lunar/linux-aws: 6.2.0-1003.3 -proposed tracker (LP: #2015430)

  * Packaging resync (LP: #1786013)
    - debian/dkms-versions -- update from kernel-versions (main/master)

  * Miscellaneous Ubuntu changes
    - [Config] aws: update annotations after rebase to the latest 6.2

  [ Ubuntu: 6.2.0-20.20 ]

  * lunar/linux: 6.2.0-20.20 -proposed tracker (LP: #2015429)
  * Packaging resync (LP: #1786013)
    - debian/dkms-versions -- update from kernel-versions (main/master)
  * FTBFS with different dkms or when makeflags are set (LP: #2015361)
    - [Packaging] FTBFS with different dkms or when makeflags are set
  * expoline.o is packaged unconditionally for s390x (LP: #2013209)
    - [Packaging] Copy expoline.o only when produced by the build
  * net:l2tp.sh failure with lunar:linux 6.2 (LP: #2013014)
    - SAUCE: l2tp: generate correct module alias strings
  * Miscellaneous Ubuntu changes
    - [Packaging] annotations: prevent duplicate include lines

  [ Ubuntu: 6.2.0-19.19 ]

  * lunar/linux: 6.2.0-19.19 -proposed tracker (LP: #2012488)
  * Neuter signing tarballs (LP: #2012776)
    - [Packaging] neuter the signing tarball
  * LSM stacking and AppArmor refresh for 6.2 kernel (LP: #2012136)
    - Revert "UBUNTU: [Config] define CONFIG_SECURITY_APPARMOR_RESTRICT_USERNS"
    - Revert "UBUNTU: SAUCE: apparmor: add user namespace creation mediation"
    - Revert "UBUNTU: SAUCE: apparmor: Add fine grained mediation of posix
      mqueues"
    - Revert "UBUNTU: SAUCE: Revert "apparmor: make __aa_path_perm() static""
    - Revert "UBUNTU: SAUCE: LSM: Specify which LSM to display (using struct cred
      as input)"
    - Revert "UBUNTU: SAUCE: apparmor: Fix build error, make sk parameter const"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in smk_netlbl_mls()"
    - Revert "UBUNTU: SAUCE: LSM: change ima_read_file() to use lsmblob"
    - Revert "UBUNTU: SAUCE: apparmor: rename kzfree() to kfree_sensitive()"
    - Revert "UBUNTU: SAUCE: AppArmor: Remove the exclusive flag"
    - Revert "UBUNTU: SAUCE: LSM: Add /proc attr entry for full LSM context"
    - Revert "UBUNTU: SAUCE: Audit: Fix incorrect static inline function
      declration."
    - Revert "UBUNTU: SAUCE: Audit: Fix for missing NULL check"
    - Revert "UBUNTU: SAUCE: Audit: Add a new record for multiple object LSM
      attributes"
    - Revert "UBUNTU: SAUCE: Audit: Add new record for multiple process LSM
      attributes"
    - Revert "UBUNTU: SAUCE: NET: Store LSM netlabel data in a lsmblob"
    - Revert "UBUNTU: SAUCE: LSM: security_secid_to_secctx in netlink netfilter"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmcontext in security_inode_getsecctx"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmcontext in security_secid_to_secctx"
    - Revert "UBUNTU: SAUCE: LSM: Ensure the correct LSM context releaser"
    - Revert "UBUNTU: SAUCE: LSM: Specify which LSM to display"
    - Revert "UBUNTU: SAUCE: IMA: Change internal interfaces to use lsmblobs"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in security_cred_getsecid"
    - Revert "UBUNTU: SAUCE: LSM: Use lsmblob in secur...

Judging from previous comments this change was verified.
Verification is requested again because the initial change was pushed in the last cycle and non aws derivatives that were not respun introduced this change in this cycle.
Since this is blocking the release of a lot of kernels for cycle 20230320, I will tag verification-done.

This bug is awaiting verification that the linux-intel-iotg/5.15.0-1028.33 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-bluefield/5.4.0-1061.67 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-riscv-5.15/5.15.0-1031.35~20.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-intel-iotg-5.15/5.15.0-1029.34~20.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-xilinx-zynqmp/5.15.0-1021.25 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Hi,

I can also confirm that kernel 5.4.0-149.166 brings CPU on the 20.04.6 NFS server back from 90-100% to normal, but i observed the following.

After reboot of the NFS server all CPU are going quickly to 100% load, to get load back to normal I had to do a `exportfs -ra' else CPU stays even with the new kernel above 90%

This bug is awaiting verification that the linux-azure/6.2.0-1009.9 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-lunar' to 'verification-done-lunar'. If the problem still exists, change the tag 'verification-needed-lunar' to 'verification-failed-lunar'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-aws-5.15/5.15.0-1046.51~20.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal-linux-aws-5.15' to 'verification-done-focal-linux-aws-5.15'. If the problem still exists, change the tag 'verification-needed-focal-linux-aws-5.15' to 'verification-failed-focal-linux-aws-5.15'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the linux-mtk/5.15.0-1030.34 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-mtk' to 'verification-done-jammy-linux-mtk'. If the problem still exists, change the tag 'verification-needed-jammy-linux-mtk' to 'verification-failed-jammy-linux-mtk'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!