[Ubuntu-24.04] Hugepage memory is not getting released even after destroying the guest!

Bug #2062556 reported by bugproxy
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Fix Released
High
Ubuntu on IBM Power Systems Bug Triage
linux (Ubuntu)
Fix Released
High
Unassigned
Noble
Fix Released
High
Unassigned

Bug Description

---Problem Description---

Hugepages memory is not getting released even after destroying the guest

Machine Type = P10 Denali LPAR

---uname output---
Linux ubuntu2404lp3 6.8.0-22-generic #22-Ubuntu SMP Thu Apr 4 22:47:57 UTC 2024 ppc64le ppc64le ppc64le GNU/Linux

---Steps to Reproduce---
1. Create a guest which is backed by hugepages.
2. Destroy the guest
3. execute "free -h" or "cat /proc/meminfo" to see that Hugepage memory is still getting held.

HugePages_Total: 20480
HugePages_Free: 20419
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 41943040 kB
DirectMap4k: 0 kB
DirectMap64k: 52428800 kB
DirectMap2M: 0 kB
DirectMap1G: 0 kB
root@ubuntu2404lp3:~# virsh list --all
 Id Name State
---------------------------
 - ramlp2g1 shut off
 - ramlp2g2 shut off
 - ramlp2g3 shut off
 - ramlp3g3 shut off

root@ubuntu2404lp3:~# free -h
               total used free shared buff/cache available
Mem: 48Gi 43Gi 4.6Gi 2.6Mi 277Mi 4.6Gi
Swap: 8.0Gi 243Mi 7.8Gi
root@ubuntu2404lp3:~#

====
This is an issue created by commit 1b151e2435fc ("block: Remove special-casing of compound pages") that moved the direct-io hugetlb handling from compound pages to folios.

Following commit has been proposed and merged into 6.9-rc1 which fixes this issue.
38b43539d64b2fa020b3b9a752a986769f87f7a6("block: Fix page refcounts for unaligned buffers in __bio_release_pages()")

So the same needs to be backported to the Ubuntu24.04 kernel as well.

CVE References

bugproxy (bugproxy)
tags: added: architecture-ppc64le bugnameltc-206058 severity-high targetmilestone-inin2404
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → linux (Ubuntu)
Thibf (thibf)
Changed in linux (Ubuntu):
status: New → Confirmed
Revision history for this message
Frank Heimes (fheimes) wrote :

I just had a look at noble's master-next tree:
$ git remote get-url origin
git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/noble
$ git branch
* master-next

and found that commit 19c1ceeca8ed is already applied:
$ git log --oneline | grep -n "block: Fix page refcounts for unaligned buffers in __bio_release_pages"
300:19c1ceeca8ed block: Fix page refcounts for unaligned buffers in __bio_release_pages()

but not yet tagged with (incl. in) any version:
$ git tag --contains 19c1ceeca8ed
$

This is probably because it just came in via upstream stable updates:
$ git show 19c1ceeca8ed | sed '/^diff --git/q'
commit 19c1ceeca8ed31d207859e659482ee83ea1959f5
Author: Tony Battersby <email address hidden>
Date: Thu Feb 29 13:08:09 2024 -0500

    block: Fix page refcounts for unaligned buffers in __bio_release_pages()

    BugLink: https://bugs.launchpad.net/bugs/2060531

    [ Upstream commit 38b43539d64b2fa020b3b9a752a986769f87f7a6 ]

    Fix an incorrect number of pages being released for buffers that do not
    start at the beginning of a page.

    Fixes: 1b151e2435fc ("block: Remove special-casing of compound pages")
    Cc: <email address hidden>
    Signed-off-by: Tony Battersby <email address hidden>
    Tested-by: Greg Edwards <email address hidden>
    Link: https://<email address hidden>
    Signed-off-by: Jens Axboe <email address hidden>
    Signed-off-by: Sasha Levin <email address hidden>
    Signed-off-by: Paolo Pisati <email address hidden>

diff --git a/block/bio.c b/block/bio.c

And since the lastest tagged Ubuntu kernel version in master-next is: Ubuntu-6.8.0-31.31
$ git log --oneline | grep -n $(git describe --tags --abbrev=0)
1170:7fdb45c9bbbc (tag: Ubuntu-6.8.0-31.31) UBUNTU: Ubuntu-6.8.0-31.31

commit 19c1ceeca8ed will be automatically part of the next/upcoming Ubuntu kernel (> Ubuntu-6.8.0-31.31).

With that I'm updating the status of this bug to Fix Committed.

Changed in ubuntu-power-systems:
status: New → Fix Committed
Changed in linux (Ubuntu):
status: Confirmed → Fix Committed
Changed in ubuntu-power-systems:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
Changed in linux (Ubuntu):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → nobody
importance: Undecided → High
Changed in ubuntu-power-systems:
importance: Undecided → High
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2024-05-06 06:50 EDT-------
I have verified this bug on the latest available Ubuntu 24.04 host. The issue has been fixed and is not getting reproduced anymore!

kernel: 6.8.0-31-generic #31-Ubuntu

Before guest creation:

HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

After guest creation:

HugePages_Total: 20480
HugePages_Free: 15360
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

After destroying the Before guest:

HugePages_Total: 20480
HugePages_Free: 20480
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/6.8.0-32.32 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux' to 'verification-done-noble-linux'. If the problem still exists, change the tag 'verification-needed-noble-linux' to 'verification-failed-noble-linux'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-v2 verification-needed-noble-linux
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2024-05-06 14:39 EDT-------
Frank has confirmed in his previous comment that the fix will be part kernel 6.8.0-31.31.

I also verified the bug on the same level of kernel and updated my comments above that the fix is working as expected.

Since it is a generic kernel, I believe that the Fix is already a part of the source code. If not, please get this included in the upcoming release.

*** I have verified the Fix and is working as expected ***

Revision history for this message
Frank Heimes (fheimes) wrote :

Many thanks for your confirmation Kowshik Jois.
With that I'm updating the tags accordingly (and set them to verification done).

tags: added: verification-done-noble-linux
removed: verification-needed-noble-linux
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (74.9 KiB)

This bug was fixed in the package linux - 6.8.0-35.35

---------------
linux (6.8.0-35.35) noble; urgency=medium

  * noble/linux: 6.8.0-35.35 -proposed tracker (LP: #2065886)

  * CVE-2024-21823
    - VFIO: Add the SPR_DSA and SPR_IAX devices to the denylist
    - dmaengine: idxd: add a new security check to deal with a hardware erratum
    - dmaengine: idxd: add a write() method for applications to submit work

linux (6.8.0-34.34) noble; urgency=medium

  * noble/linux: 6.8.0-34.34 -proposed tracker (LP: #2065167)

  * Packaging resync (LP: #1786013)
    - [Packaging] debian.master/dkms-versions -- update from kernel-versions
      (main/2024.04.29)

linux (6.8.0-32.32) noble; urgency=medium

  * noble/linux: 6.8.0-32.32 -proposed tracker (LP: #2064344)

  * Packaging resync (LP: #1786013)
    - [Packaging] drop getabis data
    - [Packaging] update variants
    - [Packaging] update annotations scripts
    - [Packaging] debian.master/dkms-versions -- update from kernel-versions
      (main/2024.04.29)

  * Enable Nezha board (LP: #1975592)
    - [Config] Enable CONFIG_REGULATOR_FIXED_VOLTAGE on riscv64

  * Enable Nezha board (LP: #1975592) // Enable StarFive VisionFive 2 board
    (LP: #2013232)
    - [Config] Enable CONFIG_SERIAL_8250_DW on riscv64

  * RISC-V kernel config is out of sync with other archs (LP: #1981437)
    - [Config] Sync riscv64 config with other architectures

  * obsolete out-of-tree ivsc dkms in favor of in-tree one (LP: #2061747)
    - ACPI: scan: Defer enumeration of devices with a _DEP pointing to IVSC device
    - Revert "mei: vsc: Call wake_up() in the threaded IRQ handler"
    - mei: vsc: Unregister interrupt handler for system suspend
    - media: ipu-bridge: Add ov01a10 in Dell XPS 9315
    - SAUCE: media: ipu-bridge: Support more sensors

  * Fix after-suspend-mediacard/sdhc-insert test failed (LP: #2042500)
    - PCI/ASPM: Move pci_configure_ltr() to aspm.c
    - PCI/ASPM: Always build aspm.c
    - PCI/ASPM: Move pci_save_ltr_state() to aspm.c
    - PCI/ASPM: Save L1 PM Substates Capability for suspend/resume
    - PCI/ASPM: Call pci_save_ltr_state() from pci_save_pcie_state()
    - PCI/ASPM: Disable L1 before configuring L1 Substates
    - PCI/ASPM: Update save_state when configuration changes

  * RTL8852BE fw security fail then lost WIFI function during suspend/resume
    cycle (LP: #2063096)
    - wifi: rtw89: download firmware with five times retry

  * intel_rapl_common: Add support for ARL and LNL (LP: #2061953)
    - powercap: intel_rapl: Add support for Lunar Lake-M paltform
    - powercap: intel_rapl: Add support for Arrow Lake

  * Kernel panic during checkbox stress_ng_test on Grace running noble 6.8
    (arm64+largemem) kernel (LP: #2058557)
    - aio: Fix null ptr deref in aio_complete() wakeup

  * Avoid creating non-working backlight sysfs knob from ASUS board
    (LP: #2060422)
    - platform/x86: asus-wmi: Consider device is absent when the read is ~0

  * Include cifs.ko in linux-modules package (LP: #2042546)
    - [Packaging] Replace fs/cifs with fs/smb/client in inclusion list

  * Add Real-time Linux Analysis tool (rtla) to linux-tools (LP: #2059080)
    - SAUCE: rtla: fix deb build
 ...

Changed in linux (Ubuntu Noble):
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-azure/6.8.0-1008.8 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-azure' to 'verification-done-noble-linux-azure'. If the problem still exists, change the tag 'verification-needed-noble-linux-azure' to 'verification-failed-noble-linux-azure'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-azure-v2 verification-needed-noble-linux-azure
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-gke/6.8.0-1004.7 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-gke' to 'verification-done-noble-linux-gke'. If the problem still exists, change the tag 'verification-needed-noble-linux-gke' to 'verification-failed-noble-linux-gke'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-gke-v2 verification-needed-noble-linux-gke
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-gcp/6.8.0-1008.9 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-gcp' to 'verification-done-noble-linux-gcp'. If the problem still exists, change the tag 'verification-needed-noble-linux-gcp' to 'verification-failed-noble-linux-gcp'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-gcp-v2 verification-needed-noble-linux-gcp
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-aws/6.8.0-1009.9 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-aws' to 'verification-done-noble-linux-aws'. If the problem still exists, change the tag 'verification-needed-noble-linux-aws' to 'verification-failed-noble-linux-aws'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-aws-v2 verification-needed-noble-linux-aws
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-ibm/6.8.0-1006.6 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-ibm' to 'verification-done-noble-linux-ibm'. If the problem still exists, change the tag 'verification-needed-noble-linux-ibm' to 'verification-failed-noble-linux-ibm'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-ibm-v2 verification-needed-noble-linux-ibm
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-intel/6.8.0-1005.12 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-intel' to 'verification-done-noble-linux-intel'. If the problem still exists, change the tag 'verification-needed-noble-linux-intel' to 'verification-failed-noble-linux-intel'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-intel-v2 verification-needed-noble-linux-intel
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-oem-6.8/6.8.0-1006.6 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-oem-6.8' to 'verification-done-noble-linux-oem-6.8'. If the problem still exists, change the tag 'verification-needed-noble-linux-oem-6.8' to 'verification-failed-noble-linux-oem-6.8'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-oem-6.8-v2 verification-needed-noble-linux-oem-6.8
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-nvidia/6.8.0-1007.7 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-nvidia' to 'verification-done-noble-linux-nvidia'. If the problem still exists, change the tag 'verification-needed-noble-linux-nvidia' to 'verification-failed-noble-linux-nvidia'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-nvidia-v2 verification-needed-noble-linux-nvidia
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-riscv/6.8.0-35.35.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-riscv' to 'verification-done-noble-linux-riscv'. If the problem still exists, change the tag 'verification-needed-noble-linux-riscv' to 'verification-failed-noble-linux-riscv'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-riscv-v2 verification-needed-noble-linux-riscv
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-raspi-realtime/6.8.0-2004.4 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-raspi-realtime' to 'verification-done-noble-linux-raspi-realtime'. If the problem still exists, change the tag 'verification-needed-noble-linux-raspi-realtime' to 'verification-failed-noble-linux-raspi-realtime'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-raspi-realtime-v2 verification-needed-noble-linux-raspi-realtime
Frank Heimes (fheimes)
Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Changed in ubuntu-power-systems:
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-lowlatency/6.8.0-35.35.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux-lowlatency' to 'verification-done-noble-linux-lowlatency'. If the problem still exists, change the tag 'verification-needed-noble-linux-lowlatency' to 'verification-failed-noble-linux-lowlatency'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-lowlatency-v2 verification-needed-noble-linux-lowlatency
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.