lockdown on power

Marking as "incomplete", while awaiting IBM's test results.

------- Comment From <email address hidden> 2019-12-15 12:11 EDT-------
Daniel Axtens would be performing the testing and update the results.

------- Comment From <email address hidden> 2020-01-09 00:14 EDT-------
Hi,

Apologies for the delay.

I installed the most recent kernel, modules and extra modules I could find from that PPA on a p8 kvm guest.

dja@dja-guest:~$ uname -a
Linux dja-guest 5.4.0-9-generic #12-Ubuntu SMP Mon Dec 16 22:32:07 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux

The kernel boots fine with no additional command line options, but if booted with lockdown=confidentiality, it has a lot of issues.

Firstly, there are a flood of lines like

[ 0.265197] Could not create tracefs 'set_ftrace_pid' entry
[ 0.265247] Lockdown: swapper/0: use of tracefs is restricted; see man kernel_lockdown.7

for various tracefs entries

Then there are 2 splats:

[ 0.265868] Could not create tracefs 'set_graph_function' entry
[ 0.265931] Lockdown: swapper/0: use of tracefs is restricted; see man kernel_lockdown.7
[ 0.266005] Could not create tracefs 'set_graph_notrace' entry
[ 0.266070] Lockdown: swapper/0: use of tracefs is restricted; see man kernel_lockdown.7
[ 0.266145] ------------[ cut here ]------------
[ 0.266195] Could not register function stat for cpu 0
[ 0.266255] WARNING: CPU: 5 PID: 1 at kernel/trace/ftrace.c:987 ftrace_init_tracefs_toplevel+0x1e8/0x264
[ 0.266342] Modules linked in:
[ 0.266384] CPU: 5 PID: 1 Comm: swapper/0 Not tainted 5.4.0-9-generic #12-Ubuntu
[ 0.266458] NIP: c000000001363430 LR: c00000000136342c CTR: c00000003fff8a00
[ 0.266532] REGS: c0000003fa67f890 TRAP: 0700 Not tainted (5.4.0-9-generic)
[ 0.266612] MSR: 8000000000029033 <SF,EE,ME,IR,DR,RI,LE> CR: 28000244 XER: 20000000
[ 0.266694] CFAR: c00000000013aa5c IRQMASK: 0
GPR00: c00000000136342c c0000003fa67fb20 c000000001a4bb00 000000000000002a
GPR04: 0000000000000001 0000000000000000 00000000000002ca 300d0a7374617420
GPR08: 00000003fbeb0000 c0000000018e3248 c0000000018e3248 c0000000011a3048
GPR12: 0000000000000000 c00000003fff8a00 c0000000000106c0 0000000000000000
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR24: c0000000011afd48 c000000001a821e0 c0000000019e6f90 c000000001464300
GPR28: c0000003f78fe910 0000000000000000 0000000000000000 c0000003fce14300
[ 0.267313] NIP [c000000001363430] ftrace_init_tracefs_toplevel+0x1e8/0x264
[ 0.267376] LR [c00000000136342c] ftrace_init_tracefs_toplevel+0x1e4/0x264
[ 0.267439] Call Trace:
[ 0.267465] [c0000003fa67fb20] [c00000000136342c] ftrace_init_tracefs_toplevel+0x1e4/0x264 (unreliable)
[ 0.267554] [c0000003fa67fbc0] [c00000000136408c] tracer_init_tracefs+0x100/0x270
[ 0.267632] [c0000003fa67fc10] [c000000000010144] do_one_initcall+0x64/0x2b0
[ 0.267707] [c0000003fa67fce0] [c000000001334694] kernel_init_freeable+0x29c/0x3a0
[ 0.267783] [c0000003fa67fdb0] [c0000000000106dc] kernel_init+0x24/0x148
[ 0.267846] [c0000003fa67fe20] [c00000000000b648] ret_from_kernel_thread+0x5c/0x74
[ 0.267921] Instruction dump:
[ 0.267961] f95f0048 f93f0050 fb830000 4af4eced 60000000 2c230000 4182ff30 3c62ff76
[ 0.268038] 7fa4eb78 38634288 4add75cd 60000000 <0fe00000> 7f83e378 4b0c08f1 600000...

------- Comment (attachment only) From <email address hidden> 2020-01-09 00:15 EDT-------

Changing back to Triaged - after test done and info provided my Daniel.

Btw. in between kernel 5.4 landed in the focal (20.04) release pocket as well:
linux-generic | 5.4.0.9.11 | focal | ppc64el
So any lock-down tests can now be done based on the normal kernel from focal's release pocket.

After discussing the the kernel team this seems to be the correct behavior and output.
This is obviously okay:
"Kernel is locked down from command line; see man kernel_lockdown.7"
but the further msgs like "Lockdown: swapper/0: use of tracefs..." seem to be right.

Just waiting for another quick test based on the very latest kernel 5.4 from the archives (the one above was from the PPA) and if the output is similar, we can consider this as done.

Changing to Fix Committed.

------- Comment From <email address hidden> 2020-02-10 01:28 EDT-------
I tried this out with the latest kernel in proposed. It looks like the -14 kernel picked up commit a356646a56857c2e5ad875beec734d7145ecd49a and that got rid of the warns. I tired access to /dev/mem and got correct results.

The commit "a356646a56857c2e5ad875beec734d7145ecd49a" is upstream with 5.5 and named "tracing: Do not create directories if lockdown is in affect".
Looking this up in focal master-next tells me that it was indeed picked-up, but under commit "ce5fac3cf42b": tracing: Do not create directories if lockdown is in affect.
It's actually in since 5.4.0-13.
Since we still have 5.4.0.12 in the focal release pocket, I can't set it to Fix Released, yet -
need to wait until 5.4.0.14 migrates from proposed to release.

------- Comment From <email address hidden> 2020-02-16 22:34 EDT-------
Hi,

I'm going to ask you to hold this open for a little bit - we're investigating internally another ppcism that may need additional lockdown support.

In the mean time I will test the kernel in -proposed.

Kind regards,
Daniel

------- Comment From <email address hidden> 2020-02-17 00:27 EDT-------
Hi,

I'm sorry, I thought I had already mentioned this but it was a case of me getting projects and teams mixed up.

Please could you pick up (in addition to the issue still pending) commit
69393cb03ccd ("powerpc/xmon: Restrict when kernel is locked down").

From the pull-request that included it, the commit does the following:

- A change to xmon (our crash handler / pseudo-debugger) to restrict
it to read-only mode when the kernel is lockdown'ed, otherwise it's
trivial to drop into xmon and modify kernel data, such as the
lockdown state.

To exploit this you'd need to boot with command line including 'xmon=rw', as xmon isn't read-write by default on the Focal kernel, but that's not exactly a challenge. I have used this to drop down from lockdown=confidentiality to lockdown=none on 5.4.0-14-generic #17-Ubuntu

Regards,
Daniel

Hi Daniel,
I found that that commit
69393cb03ccd "powerpc/xmon: Restrict when kernel is locked down"
landed upstream with v5.5-rc1.

I created a separate LP bug / ticket to get it into focal's kernel 5.4 (hoping that it's a simple cherry pick):
LP 1863562 - "Restrict ppc64el xmon to read-only-mode if kernel is locked down"
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1863562

You may consider mirroring LP 1863562 to bugzilla for completeness reasons.

------- Comment From <email address hidden> 2020-03-27 10:03 EDT-------
I would like to understand that with new lockdown patches upstreamed now,

* Is Ubuntu still going to carry a patch linking secureboot with lockdown ? If yes, would you be doing same for powerpc ?
* Is Ubuntu going to enable lockdown during build time ? If yes, then in which mode - INTEGRITY or CONFIDENTIALITY. ?

Thanks & Regards,
- Nayna

Well, prior to 20.04 the secure-boot lockdown in Ubuntu was largely based on Matthew Garrett patch set. With the upstream acceptance of secure boot in 5.4 we moved over to the upstream code, and 20.04 contains kernel 5.4 anyway.
In a different LP bug IBM got generally asked for checking lock-down on two architectures, and this is the spin-off ticket of this ticket for Power.
With Daniel Axtens comments on this tickets and the request to pick up some commits we thought that now all lock-down patches are in - secureboot is in from upstream - incl. any linkages.
I suggest to have a chat with Daniel on that, too.

------- Comment From <email address hidden> 2020-03-27 11:17 EDT-------
Hi, Thanks for the quick response. I have one question based on your statement - "prior to 20.04 the secure-boot lockdown in Ubuntu was largely based on Matthew Garrett patch set."

Q. Is the lockdown enabled during build ? And if yes, in which mode INTEGRITY/CONFIDENTIALITY ?

Thanks & Regards,
- Nayna

Looking up the options I see that on ppc64el there is (on focal/20.04):
CONFIG_LOCK_DOWN_KERNEL_FORCE_INTEGRITY=n
CONFIG_LOCK_DOWN_KERNEL_FORCE_CONFIDENTIALITY=n
CONFIG_LOCK_DOWN_KERNEL_FORCE_NONE=y
but
CONFIG_LSM="lockdown,yama,integrity,apparmor"

Lockdown is enabled in focal, and the default mode when booted without any secure boot scheme is NONE.

When booted under a secure boot scheme, we had previously forced the CONFIDENTIALITY mode for lockdown. But we have now scaled that back, and the kernel in focal-proposed sets the mode to INTEGRITY under secure boot. This kernel should be released early next week.

Also I'll add, you can use this ppa to test the -proposed kernels without enabling all of -proposed.

https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/proposed/

These are kernels copied from the -proposed pocked, after we've built signed packages in -proposed.

------- Comment on attachment From <email address hidden> 2020-04-02 08:35 EDT-------

Hi,

Thanks Nayna for the reminder to look at this again.

AFAICT, Canonical's Focal kernel sets up its non-upstreamed secure-boot-enforces-lockdown support in the following set of commits:
(edited down from the list of all commits with UBUNTU: and lockdown in the title.)

40fc208c8aae UBUNTU: SAUCE: (lockdown) security: lockdown: expose a hook to lock the kernel down
8309e3e2a4c2 UBUNTU: SAUCE: (lockdown) efi: Add an EFI_SECURE_BOOT flag to indicate secure boot mode
f8d21cba9d0e UBUNTU: SAUCE: (lockdown) efi: Lock down the kernel if booted in secure boot mode
36ca37871ad2 UBUNTU: SAUCE: (lockdown) arm64: Allow locking down the kernel under EFI secure boot
7bfea7ace0ff UBUNTU: SAUCE: (lockdown) s390/ipl: lockdown kernel when booted secure
d0b71cb9b8a2 UBUNTU: [Config] Enable lockdown under secure boot
ef7c6600bb3e UBUNTU: SAUCE: (lockdown) Reduce lockdown level to INTEGRITY for secure boot

This shows a secure-boot-enforces-lockdown patch for x86, arm64 and s390. I think we also need a powerpc one.

I've written a short 2 patch series and attached it. I also needed to cherry-pick from upstream:
commit 1a8916ee3ac2 ("powerpc: Detect the secure boot mode of the system")
commit 2702809a4a1a ("powerpc: Detect the trusted boot state of the system")

I've only been able to build-test as I only have an unsecured system. Nayna, could you try signing and booting the kernel on system with secure boot, and see if it comes up in lockdown=integrity mode? I'll send you the kernel via internal channels.

Unfortunately they're against focal/master not focal/master-next because I had trouble with the zfs stuff in master-next, but it only affects the config patch and I'm not sure I did that right anyway...

Kind regards,
Daniel

------- Comment (attachment only) From <email address hidden> 2020-04-02 08:36 EDT-------

Patch one is included on the test build for bug 1866909 in https://launchpad.net/~sforshee/+archive/ubuntu/lp1866909/+packages. I incorporated the config changes in with those requested for that bug.

This is noted on the other bug, but I'll also note it here. This kernel is *not* signed with the archive key. The public half of the key pair used to sign this build can be found in this tarball:

http://ppa.launchpad.net/sforshee/lp1866909/ubuntu/dists/focal/main/signed/linux-ppc64el/current/signed.tar.gz

------- Comment on attachment From <email address hidden> 2020-04-03 03:05 EDT-------

Hi Seth,

Thanks, that was extremely helpful.

Nayna noticed that I was overly keen to lock things down - I should only lock down in Secure mode: if a system is in Trusted mode only I shouldn't lock it down. This now matches the UEFI behaviour: (AFAICT) measurements are made unconditionally but lockdown only occurs in Secure Boot mode.

I have updated patch 1/2.

Kind regards,
Daniel

The revised patch looks indeed less strict - we are considering that one ...

New test build with the updated patch in the same ppa.

------- Comment From <email address hidden> 2020-04-06 09:26 EDT-------
Hi,

This works as expected on a machine with secure boot disabled in hardware:

dja@talos2:~$ uname -a
Linux talos2 5.4.0-21-generic #25+lp1866909v202004031128-Ubuntu SMP Fri Apr 3 18:38:30 UTC 202 ppc64le ppc64le ppc64le GNU/Linux
dja@talos2:~$ sudo cat /sys/kernel/security/lockdown
[none] integrity confidentiality

I don't have access to a system with end-to-end secure boot enabled - Nayna will post test results for that.

Kind regards,
Daniel

------- Comment From <email address hidden> 2020-04-06 11:28 EDT-------
I tested the ppa kernel patch which links secureboot with lockdown.

When secureboot is disabled:
ubuntu@ltc-wspoon13:~$ sudo cat /sys/kernel/security/lockdown
[none] integrity confidentiality

When secureboot is enabled:
ubuntu@ltc-wspoon13:~$ sudo cat /sys/kernel/security/lockdown
none [integrity] confidentiality

It does move to integrity lockdown mode.

Daniel helped with testing the lockdown functionality itself in secureboot enabled state.

Here are his test results:
xmon is in read-only mode.

54:mon> ls is_ppc_secureboot_enabled
is_ppc_secureboot_enabled: c000000000085430
54:mon> b c000000000085430
Operation disabled: xmon in read-only mode
54:mon>

/dev/mem is blocked:
root@ltc-wspoon13:/boot# cat /dev/mem
cat: /dev/mem: Operation not permitted
root@ltc-wspoon13:/boot# dmesg|tail
...
[ 991.917345] Lockdown: cat: /dev/mem,kmem,port is restricted; see man kernel_lockdown.7

He also ensured that kexec load is disabled and can boot successfully to a signed kernel if the key is present in the keyring.

Thank Daniel for the linking patch between secureboot and lockdown. And also for the quick testing of lockdown itself.
Thanks to Canonical team for respining the kernel with the updated patch from Daniel.

Thanks to Michael for his support throughout this work.

Thanks & Regards,
- Nayna

Thanks for testing. I've applied the patches to focal/master-next.

This bug was fixed in the package linux - 5.4.0-24.28

---------------
linux (5.4.0-24.28) focal; urgency=medium

  * focal/linux: 5.4.0-24.28 -proposed tracker (LP: #1871939)

  * getitimer returns it_value=0 erroneously (LP: #1349028)
    - [Config] CONTEXT_TRACKING_FORCE policy should be unset

  * 12d1:1038 Dual-Role OTG device on non-HNP port - unable to enumerate USB
    device on port 1 (LP: #1047527)
    - [Config] USB_OTG_FSM policy not needed

  * Add DCPD backlight support for HP CML system (LP: #1871589)
    - SAUCE: drm/i915: Force DPCD backlight mode for HP CML 2020 system

  * Backlight brightness cannot be adjusted using keys (LP: #1860303)
    - SAUCE drm/i915: Force DPCD backlight mode for HP Spectre x360 Convertible
      13t-aw100

  * CVE-2020-11494
    - slcan: Don't transmit uninitialized stack data in padding

  * Ubuntu Kernel Support for OpenPOWER NV Secure & Trusted Boot (LP: #1866909)
    - powerpc: Detect the secure boot mode of the system
    - powerpc/ima: Add support to initialize ima policy rules
    - powerpc: Detect the trusted boot state of the system
    - powerpc/ima: Define trusted boot policy
    - ima: Make process_buffer_measurement() generic
    - certs: Add wrapper function to check blacklisted binary hash
    - ima: Check against blacklisted hashes for files with modsig
    - powerpc/ima: Update ima arch policy to check for blacklist
    - powerpc/ima: Indicate kernel modules appended signatures are enforced
    - powerpc/powernv: Add OPAL API interface to access secure variable
    - powerpc: expose secure variables to userspace via sysfs
    - x86/efi: move common keyring handler functions to new file
    - powerpc: Load firmware trusted keys/hashes into kernel keyring
    - x86/efi: remove unused variables

  * [roce-0227]sync mainline kernel 5.6rc3 roce patchset into ubuntu HWE kernel
    branch (LP: #1864950)
    - RDMA/hns: Cleanups of magic numbers
    - RDMA/hns: Optimize eqe buffer allocation flow
    - RDMA/hns: Add the workqueue framework for flush cqe handler
    - RDMA/hns: Delayed flush cqe process with workqueue
    - RDMA/hns: fix spelling mistake: "attatch" -> "attach"
    - RDMA/hns: Initialize all fields of doorbells to zero
    - RDMA/hns: Treat revision HIP08_A as a special case
    - RDMA/hns: Use flush framework for the case in aeq
    - RDMA/hns: Stop doorbell update while qp state error
    - RDMA/hns: Optimize qp destroy flow
    - RDMA/hns: Optimize qp context create and destroy flow
    - RDMA/hns: Optimize qp number assign flow
    - RDMA/hns: Optimize qp buffer allocation flow
    - RDMA/hns: Optimize qp param setup flow
    - RDMA/hns: Optimize kernel qp wrid allocation flow
    - RDMA/hns: Optimize qp doorbell allocation flow
    - RDMA/hns: Check if depth of qp is 0 before configure

  * [hns3-0316]sync mainline kernel 5.6rc4 hns3 patchset into ubuntu HWE kernel
    branch (LP: #1867586)
    - net: hns3: modify an unsuitable print when setting unknown duplex to fibre
    - net: hns3: add enabled TC numbers and DWRR weight info in debugfs
    - net: hns3: add support for dump MAC ID and loopback status in debugfs
    - net: hns3: add missing help info for QS shaper...