[18.04 FEAT] zKVM: Add hardware CPU Model - kernel part

------- Comment From <email address hidden> 2019-07-11 03:30 EDT-------
To simplify the backport to 4.15 you should add these 2 commits:

11ba5961a2156 KVM: s390: add debug logging for cpu model subfunctions
346fa2f891c71 KVM: s390: implement subfunction processor calls

I have verified that on bionic/master the following works without a problem:

git cherry-pick 346fa2f891c71 11ba5961a2156 7832e91cd33f d5cb6ab1e3d4 13209ad0395c d668139718a9 173aec2d5a9f 4f45b90e1c03 8ec2fa52eac5

Kernel SRU request submitted:
https://lists.ubuntu.com/archives/kernel-team/2019-July/thread.html#102228

FYI - the related qemu changes have a PPA at https://launchpad.net/~paelzer/+archive/ubuntu/bug-1836154-s390x-hwmodels/+packages

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

------- Comment From <email address hidden> 2019-07-26 03:12 EDT-------
I can confirm that the KVM part (this bugzilla) works fine with
a. this kernel from proposed
b. qemu from https://launchpad.net/~paelzer/+archive/ubuntu/bug-1836154-s390x-hwmodels/+packages

Now. The kernel in proposed has a bug which prohibits this kernel to be released.

dmesg is flooded with
[ 417.945769] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.951747] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.955339] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.960318] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.961260] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 422.248440] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 430.012698] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 434.051283] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 526.865193] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 526.873399] BUG: non-zero pgtables_bytes on freeing mm: -16384

messages. Looks like you picked an upstream patch and missed several followup patches in that area.
Thjis is around commit e12e4044aede97974f2222eb7f0ed726a5179a32 ("s390/mm: fix mis-accounting of pgtable_bytes") and later patches on top of this.

(This is independent from the KVM changes which are working perfectly fine).

@cborntra many thanks for the verification.

Commit 9967c2a is indeed not (yet) in this 4.15 bionic kernel.
It got upstream accepted with 4.20 and is with that in disco, hence that should all work on disco:

$ git log --oneline | grep -m 1 "s390/mm: fix mis-accounting of pgtable_bytes"
e12e404 s390/mm: fix mis-accounting of pgtable_bytes
$ git tag --contains e12e404
Ubuntu-5.0.0-7.8
...
Ubuntu-5.0.0-22.23
v5.0
$ rmadison --arch=s390x linux-generic | grep disco-proposed
 linux-generic | 5.0.0.22.23 | disco-proposed | s390x

Regarding bionic, the patch you pointed out will be picked up with the upstream stable release update process that is regularly done.
The next update is addressed by ticket LP 1836968 and covers patchset 2019-07-17, that includes "s390/mm: fix mis-accounting of pgtable_bytes"
(Ctrl+f on page https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1836968 will show)
Hence there is no need to open a new LP for this particular patch or to submit it in a separate kernel SRU. Because it will probably come with the next bionic kernel update (since LP 1836968 is already marked as Fix Committed).

I think with strictly looking at the KVM part we can consider this verification as successfully done.

I will btw. mark the 'linux (Ubuntu)' entry that represents eoan and Fix Released, because with it's 5.2 kernel it has already everything need.

With "[SRU][Bionic][PULL] Bionic upstream stable patchset 2019-07-30+"
commit "s390/mm: fix mis-accounting of pgtable_bytes" landed in between in bionic master-next.

~/ubuntu-bionic-master-next/ubuntu-bionic$ git log --oneline | grep -m 1 "s390/mm: fix mis-accounting of pgtable_bytes"
76d93ab s390/mm: fix mis-accounting of pgtable_bytes
~/ubuntu-bionic-master-next/ubuntu-bionic$ git describe --tags --contains 76d93ab
Ubuntu-4.15.0-56.62~514

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

------- Comment From <email address hidden> 2019-08-07 08:50 EDT-------
Due to vacation reason, the verification is delayed by 3 weeks.

Verification was already done on disco and bionic, just xenial will be delayed.

This bug was fixed in the package linux - 5.0.0-25.26

---------------
linux (5.0.0-25.26) disco; urgency=medium

  * CVE-2019-1125
    - x86/cpufeatures: Carve out CQM features retrieval
    - x86/cpufeatures: Combine word 11 and 12 into a new scattered features word
    - x86/speculation: Prepare entry code for Spectre v1 swapgs mitigations
    - x86/speculation: Enable Spectre v1 swapgs mitigations
    - x86/entry/64: Use JMP instead of JMPQ
    - x86/speculation/swapgs: Exclude ATOMs from speculation through SWAPGS

 -- Kleber Sacilotto de Souza <email address hidden> Thu, 01 Aug 2019 12:04:35 +0200

This bug was fixed in the package linux - 4.15.0-58.64

---------------
linux (4.15.0-58.64) bionic; urgency=medium

  * unable to handle kernel NULL pointer dereference at 000000000000002c (IP:
    iget5_locked+0x9e/0x1f0) (LP: #1838982)
    - Revert "ovl: set I_CREATING on inode being created"
    - Revert "new primitive: discard_new_inode()"

linux (4.15.0-57.63) bionic; urgency=medium

  * CVE-2019-1125
    - x86/cpufeatures: Carve out CQM features retrieval
    - x86/cpufeatures: Combine word 11 and 12 into a new scattered features word
    - x86/speculation: Prepare entry code for Spectre v1 swapgs mitigations
    - x86/speculation: Enable Spectre v1 swapgs mitigations
    - x86/entry/64: Use JMP instead of JMPQ
    - x86/speculation/swapgs: Exclude ATOMs from speculation through SWAPGS

  * Packaging resync (LP: #1786013)
    - update dkms package versions

linux (4.15.0-56.62) bionic; urgency=medium

  * bionic/linux: 4.15.0-56.62 -proposed tracker (LP: #1837626)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log
    - [Packaging] update helper scripts

  * CVE-2019-2101
    - media: uvcvideo: Fix 'type' check leading to overflow

  * hibmc-drm Causes Unreadable Display for Huawei amd64 Servers (LP: #1762940)
    - [Config] Set CONFIG_DRM_HISI_HIBMC to arm64 only
    - SAUCE: Make CONFIG_DRM_HISI_HIBMC depend on ARM64

  * Bionic: support for Solarflare X2542 network adapter (sfc driver)
    (LP: #1836635)
    - sfc: make mem_bar a function rather than a constant
    - sfc: support VI strides other than 8k
    - sfc: add Medford2 (SFC9250) PCI Device IDs
    - sfc: improve PTP error reporting
    - sfc: update EF10 register definitions
    - sfc: populate the timer reload field
    - sfc: update MCDI protocol headers
    - sfc: support variable number of MAC stats
    - sfc: expose FEC stats on Medford2
    - sfc: expose CTPIO stats on NICs that support them
    - sfc: basic MCDI mapping of 25/50/100G link speeds
    - sfc: support the ethtool ksettings API properly so that 25/50/100G works
    - sfc: add bits for 25/50/100G supported/advertised speeds
    - sfc: remove tx and MCDI handling from NAPI budget consideration
    - sfc: handle TX timestamps in the normal data path
    - sfc: add function to determine which TX timestamping method to use
    - sfc: use main datapath for HW timestamps if available
    - sfc: only enable TX timestamping if the adapter is licensed for it
    - sfc: MAC TX timestamp handling on the 8000 series
    - sfc: on 8000 series use TX queues for TX timestamps
    - sfc: only advertise TX timestamping if we have the license for it
    - sfc: simplify RX datapath timestamping
    - sfc: support separate PTP and general timestamping
    - sfc: support second + quarter ns time format for receive datapath
    - sfc: support Medford2 frequency adjustment format
    - sfc: add suffix to large constant in ptp
    - sfc: mark some unexported symbols as static
    - sfc: update MCDI protocol headers
    - sfc: support FEC configuration through ethtool
    - sfc: remove ctpio_dmabuf_start from stats
    - sfc: stop the TX queue before pushing new buffers

  * [18.04 FEAT] zKVM: Add hardwar...

------- Comment From <email address hidden> 2019-08-15 08:45 EDT-------
IBM Bugzilla status -> closed, Released by all requested Distros

------- Comment From <email address hidden> 2019-09-03 14:58 EDT-------
This still has the verification-needed-xenial tag. Will this be backported or can we remove that tag? xenial-proposed does not seem to contain a kernel update and a backport to xenial was not requested. (It would be possible though).

Please ignore the xenial verification request, as it was generated in error. I can confirm this issue is resolved in bionic and up and calls for no further actions. I apologise for any inconvenience caused.