[UBUNTU 23.04] Kernel config option missing for s390x PCI passthrough

Bug #2042853 reported by bugproxy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Skipper Bug Screeners
linux (Ubuntu)
Status tracked in Noble
Lunar
Fix Released
Medium
Canonical Kernel Team
Mantic
Fix Released
Medium
Canonical Kernel Team
Noble
Fix Released
High
Canonical Kernel Team

Bug Description

SRU Justification:

[Impact]

 * Today no s390x-specific vfio-pci devices (zPCI) can be passed
   from a KVM host to a KVM guest (incl. secure execution guests
   in the context of confidential computing).

 * s390x PCI passthrough needs various changes in the s390x kernel zPCI
   code (incl. the new s390x-specific Kernel config option
   'CONFIG_VFIO_PCI_ZDEV_KVM') that were introduced with kernel 6.0
   and got backported to 22.04/jammy as part of LP: #1853306.

 * Lunar an newer Ubuntu releases have the code already included from
   upstream (incl. the Kernel option 'CONFIG_VFIO_PCI_ZDEV_KVM'), but the
   config option is not set, hence zPCI pass-through is still not possible.

[Fix]

 * To be able to make use of VFIO zPCI pass-through on s390x running newer
   Ubuntu releases (especially needed in the context of secure execution)
   the (s390x-specific) Kernel config option 'CONFIG_VFIO_PCI_ZDEV_KVM' needs
   to be enabled and set to 'y'.

[Test Case]

 * Hardware used: z14 or greater LPAR, PCI-attached devices
   (RoCE VFs, ISM devices, NVMe drive)

 * Setup: Both the kernel and QEMU features are needed for the feature
   to function (an upstream QEMU can be used to verify the kernel early),
   and the facility is only available on z14 or newer.
   When any of those pieces is missing,
   the interpretation facility will not be used.
   When both the kernel and QEMU features are included in their respective
   packages, and running in an LPAR on a z14 or newer machine,
   this feature will be enabled automatically.
   Existing supported devices should behave as before with no changes
   required by an end-user (e.g. no changes to libvirt domain definitions)
   -- but will now make use of the interpretation facility.
   Additionally, ISM devices will now be eligible for vfio-pci passthrough
   (where before QEMU would exit on error if attempting to provide an ISM
   device for vfio-pci passthrough, preventing the guest from starting)

 * Testing will include the following scenarios, repeated each for RoCE,
   ISM and NVMe:

   1) Testing of basic device passthrough (create a VM with a vfio-pci
      device as part of the libvirt domain definition, passing through
      a RoCE VF, an ISM device, or an NVMe drive. Verify that the device
      is available in the guest and functioning)
   2) Testing of device hotplug/unplug (create a VM with a vfio-pci device,
      virsh detach-device to remove the device from the running guest,
      verify the device is removed from the guest, then virsh attach-device
      to hotplug the device to the guest again, verify the device functions
      in the guest)
   3) Host power off testing: Power off the device from the host, verify
      that the device is unplugged from the guest as part of the poweroff
   4) Guest power off testing: Power off the device from within the guest,
      verify that the device is unusable in the guest,
      power the device back on within the guest and verify that the device
      is once again usable.
   5) Guest reboot testing: (create a VM with a vfio-pci device,
      verify the device is in working condition, reboot the guest,
      verify that the device is still usable after reboot)

[Regression Potential]

 * The regression potential is moderate, since the code is upstream
   for quite a while and already enabled in jammy.

 * The general way on using passthrough has not changed, with this
   change (config option) it's now just possible to passthrough
   zPCI on top.

 * CCW devices are not affected.

 * And this is s390x-specific anyway, so no other architectures are affected.

[Other]

 * The enabling of the kernel config option is exactly the same for L, M
   and U/N, but I submitted separate patches due to slightly different context
   and offsets.
__________

=== Description by <email address hidden> ===

LP#1853306 / IBM bug 182254 backported the necessary kernel pieces to enable enhanced interpretation of PCI passthrough on s390. It also included a kernel config update for CONFIG_VFIO_PCI_ZDEV_KVM=y which is necessary to activate this kernel feature.

For lunar and mantic, the kernel code did not require backporting due to the base kernel version already containing it, but the kernel config option still needs to be enabled. Comparison from git.launchpad.net:

Jammy:
cat debian.master/config/annotations | grep CONFIG_VFIO_PCI_ZDEV_KVM
CONFIG_VFIO_PCI_ZDEV_KVM policy<{'s390x': 'y'}>
CONFIG_VFIO_PCI_ZDEV_KVM note<'LP#1853306 Enable VFIO zPCI pass-through for s390x'>

Lunar:
cat debian.master/config/annotations | grep CONFIG_VFIO_PCI_ZDEV_KVM
CONFIG_VFIO_PCI_ZDEV_KVM policy<{'s390x': 'n'}>

Mantic:
cat debian.master/config/annotations | grep CONFIG_VFIO_PCI_ZDEV_KVM
CONFIG_VFIO_PCI_ZDEV_KVM policy<{'s390x': 'n'}>

This setting is supposed to default y when S390 && KVM via Kconfig. Can this be enabled for lunar, mantic, and future releases?

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-203971 severity-medium targetmilestone-inin---
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
Changed in linux (Ubuntu Noble):
assignee: Skipper Bug Screeners (skipper-screen-team) → nobody
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → Triaged
Changed in linux (Ubuntu Lunar):
status: New → Triaged
Changed in linux (Ubuntu Mantic):
status: New → Triaged
Changed in linux (Ubuntu Noble):
status: New → Triaged
Frank Heimes (fheimes)
summary: - [UBUNTU 23.04] Kernel config option missing for s390 PCI passthrough
+ [UBUNTU 23.04] Kernel config option missing for s390x PCI passthrough
Frank Heimes (fheimes)
description: updated
Revision history for this message
Frank Heimes (fheimes) wrote :

SRU request submitted to the Ubuntu kernel team mailing list for unstable/noble, mantic and lunar.
https://lists.ubuntu.com/archives/kernel-team/2023-November/thread.html#146982

Changing status to 'In Progress' for noble, mantic and lunar.

Changed in ubuntu-z-systems:
importance: Undecided → High
Changed in linux (Ubuntu Lunar):
importance: Undecided → High
Changed in linux (Ubuntu Mantic):
importance: Undecided → High
Changed in linux (Ubuntu Noble):
importance: Undecided → High
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Changed in linux (Ubuntu Lunar):
status: Triaged → In Progress
Changed in linux (Ubuntu Mantic):
status: Triaged → In Progress
Changed in linux (Ubuntu Noble):
status: Triaged → In Progress
Changed in linux (Ubuntu Lunar):
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Changed in linux (Ubuntu Mantic):
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Changed in linux (Ubuntu Noble):
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Revision history for this message
Frank Heimes (fheimes) wrote :
Stefan Bader (smb)
Changed in linux (Ubuntu Mantic):
status: In Progress → Fix Committed
importance: High → Medium
Changed in linux (Ubuntu Lunar):
importance: High → Medium
status: In Progress → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/6.2.0-41.42 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-lunar-linux' to 'verification-done-lunar-linux'. If the problem still exists, change the tag 'verification-needed-lunar-linux' to 'verification-failed-lunar-linux'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-lunar-linux-v2 verification-needed-lunar-linux
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/6.5.0-16.16 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-mantic-linux' to 'verification-done-mantic-linux'. If the problem still exists, change the tag 'verification-needed-mantic-linux' to 'verification-failed-mantic-linux'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-mantic-linux-v2 verification-needed-mantic-linux
Changed in linux (Ubuntu Noble):
status: In Progress → Fix Committed
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2024-01-12 13:06 EDT-------
Verified with the kernels in -proposed for both mantic and lunar that zPCI interpretation is now enabled and usable. Thanks!

tags: removed: verification-needed-lunar-linux verification-needed-mantic-linux
Revision history for this message
Frank Heimes (fheimes) wrote :

Thanks for the verification @mjrosato !

tags: added: verification-done-lunar-linux verification-done-mantic-linux
bugproxy (bugproxy)
tags: added: targetmilestone-inin2304
removed: targetmilestone-inin--- verification-done-lunar-linux verification-done-mantic-linux
bugproxy (bugproxy)
tags: added: verification-done
Boris Barth (borisbarth)
tags: added: verification-done-lunar-linux verification-done-mantic-linux
Revision history for this message
Boris Barth (borisbarth) wrote :

I re-added the tags "verification-done-lunar-linux verification-done-mantic-linux" manually as they were deleted by the bridge (bugproxy) when syncing the target milestone.
This problem of disappearing tags has been seen lately also in other bugs / LP items. IBM is currently investigating what's causing this.

Revision history for this message
Frank Heimes (fheimes) wrote :

This is already Fix Released for L and M.
It's not set in noble's 6.6.0-14.14, but in 6.8.0-2.2.
Hence leaving it as Fix Committed for noble until we are on 6.8.

Changed in linux (Ubuntu Lunar):
status: Fix Committed → Fix Released
Changed in linux (Ubuntu Mantic):
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-gcp-6.5/6.5.0-1013.13~22.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-gcp-6.5' to 'verification-done-jammy-linux-gcp-6.5'. If the problem still exists, change the tag 'verification-needed-jammy-linux-gcp-6.5' to 'verification-failed-jammy-linux-gcp-6.5'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-gcp-6.5-v2 verification-needed-jammy-linux-gcp-6.5
removed: verification-done
Revision history for this message
Frank Heimes (fheimes) wrote :

This is s390x-specific only and does not affect gcp.
Hence I'm adjusting the tag to potentially unblock the process.

tags: added: verification-done
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-azure/6.5.0-1013.13 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-mantic-linux-azure' to 'verification-done-mantic-linux-azure'. If the problem still exists, change the tag 'verification-needed-mantic-linux-azure' to 'verification-failed-mantic-linux-azure'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-mantic-linux-azure-v2 verification-needed-mantic-linux-azure
removed: verification-done
Revision history for this message
Frank Heimes (fheimes) wrote :

This issue is s390-specific only, and does not affect azure at all.
Hence adding tag verification-done again to potentially unblock the process.

tags: added: verification-done
Revision history for this message
Kleber Sacilotto de Souza (kleber-souza) wrote :

Confirmed the fix has been applied to 24.04 Noble linux 6.8 repository:

https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/noble/commit/?h=master-next&id=3842f676a88b7c4b2063987beaae37f80505d841

As soon as the kernel is promoted to the noble-release pocket the bug report will be updated automatically.

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-aws-6.5/6.5.0-1013.13~22.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-aws-6.5' to 'verification-done-jammy-linux-aws-6.5'. If the problem still exists, change the tag 'verification-needed-jammy-linux-aws-6.5' to 'verification-failed-jammy-linux-aws-6.5'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-aws-6.5-v2 verification-needed-jammy-linux-aws-6.5
removed: verification-done
Revision history for this message
Frank Heimes (fheimes) wrote :

This issue is s390-specific only, and does not affect aws at all.
Hence adding tag verification-done again to potentially unblock the process.

tags: added: verification-done verification-done-jammy-linux-aws-6.5 verification-done-jammy-linux-gcp-6.5 verification-done-mantic-linux-azure
removed: verification-needed-jammy-linux-aws-6.5 verification-needed-jammy-linux-gcp-6.5 verification-needed-mantic-linux-azure
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 6.8.0-11.11

---------------
linux (6.8.0-11.11) noble; urgency=medium

  * noble/linux: 6.8.0-11.11 -proposed tracker (LP: #2053094)

  * Miscellaneous Ubuntu changes
    - [Packaging] riscv64: disable building unnecessary binary debs

 -- Paolo Pisati <email address hidden> Wed, 14 Feb 2024 00:04:31 +0100

Changed in linux (Ubuntu Noble):
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-nvidia-6.5/6.5.0-1014.14 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy-linux-nvidia-6.5' to 'verification-done-jammy-linux-nvidia-6.5'. If the problem still exists, change the tag 'verification-needed-jammy-linux-nvidia-6.5' to 'verification-failed-jammy-linux-nvidia-6.5'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-jammy-linux-nvidia-6.5-v2 verification-needed-jammy-linux-nvidia-6.5
removed: verification-done
Revision history for this message
Frank Heimes (fheimes) wrote :

This LP bug was entirely s390x specific, hence not affecting any other platform.
So setting verification again to done, to potentially unblock any further processes.

tags: added: verification-done verification-done-jammy-linux-nvidia-6.5
removed: verification-needed-jammy-linux-nvidia-6.5
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.