unprivileged tests in test_verifier from ubuntu_bpf failed with "Failed to load prog 'Operation not permitted'" on B-4.15

Bug #1980648 reported by Po-Hsu Lin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-kernel-tests
Fix Released
Undecided
Po-Hsu Lin
linux (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Po-Hsu Lin

Bug Description

[Impact]
We have kernel.unprivileged_bpf_disabled enabled for Bionic kernel:
$ sysctl kernel.unprivileged_bpf_disabled
kernel.unprivileged_bpf_disabled = 2

This causes all unprivileged tests in test_verifier of bpf selftests
to fail like:
  #0/u add+sub+mul FAIL
  Failed to load prog 'Operation not permitted'!

Because it permanently disables unprivileged BPF access for
the currently running kernel.

[Fix]
* d0a0e4956f ("selftests/bpf: Count tests skipped by unpriv")
* 0a67487403 ("selftests/bpf: Only run tests if !bpf_disabled")

These two patches can be cherry-picked into our Bionic kernel.

Note that there is a follow-up fix for 0a67487403, which is commit
deea81228b ("selftests/bpf: check return value of fopen in
test_verifier.c"), but this is intended for older kernels (< 4.4) thus
I will leave it alone.

[Test]
Patch tested with Bionic 4.15.0-188, and these unprivileged won't fail
with "Failed to load prog 'Operation not permitted'!" anymore, they
will be marked as skipped tests.

Overall test result improves from:
  Summary: 551 PASSED, 286 FAILED
To:
  Summary: 551 PASSED, 278 SKIPPED, 8 FAILED

[Where problems could occur]
Change limited to the bpf selftest code, no actual changes to kernel
function. If this fix is wrong, we might get incorrect test results.

[Original Bug Report]
Issue found on Bionic 4.15 cloud variants (as we don't run this test on bare-metals)

 #0/u add+sub+mul FAIL
 Failed to load prog 'Operation not permitted'!
--
 #1/u DIV32 by 0, zero check 1 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #2/u DIV32 by 0, zero check 2 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #3/u DIV64 by 0, zero check FAIL
 Failed to load prog 'Operation not permitted'!
--
 #4/u MOD32 by 0, zero check 1 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #5/u MOD32 by 0, zero check 2 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #6/u MOD64 by 0, zero check FAIL
 Failed to load prog 'Operation not permitted'!
--
 #36/u test6 ld_imm64 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #37/u test7 ld_imm64 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #46/u arsh64 on imm FAIL
 Failed to load prog 'Operation not permitted'!
--
 #47/u arsh64 on reg FAIL
 Failed to load prog 'Operation not permitted'!
--
 #60/u uninitialized stack1 Failed to create hash map 'Operation not permitted'!
--
 #63/u non-invalid fp arithmetic FAIL
 Failed to load prog 'Operation not permitted'!
--
 #67/u check valid spill/fill, skb mark FAIL
 Failed to load prog 'Operation not permitted'!
--
 #81/u don't check return value before access Failed to create hash map 'Operation not permitted'!
--
 #82/u access memory with incorrect alignment Failed to create hash map 'Operation not permitted'!
--
 #83/u sometimes access memory with incorrect alignment Failed to create hash map 'Operation not permitted'!
--
 #86/u jump test 3 Failed to create hash map 'Operation not permitted'!
--
 #89/u access skb fields ok FAIL
 Failed to load prog 'Operation not permitted'!
--
 #91/u access skb fields bad2 Failed to create hash map 'Operation not permitted'!
--
 #92/u access skb fields bad3 Failed to create hash map 'Operation not permitted'!
--
 #93/u access skb fields bad4 Failed to create hash map 'Operation not permitted'!
--
 #118/u check cb access: byte FAIL
 Failed to load prog 'Operation not permitted'!
--
 #121/u check skb->hash byte load permitted FAIL
 Failed to load prog 'Operation not permitted'!
--
 #126/u check cb access: half FAIL
 Failed to load prog 'Operation not permitted'!
--
 #130/u check skb->hash half load permitted FAIL
 Failed to load prog 'Operation not permitted'!
--
 #133/u check cb access: word FAIL
 Failed to load prog 'Operation not permitted'!
--
 #138/u check cb access: double FAIL
 Failed to load prog 'Operation not permitted'!
--
 #149/u PTR_TO_STACK store/load FAIL
 Failed to load prog 'Operation not permitted'!
--
 #155/u unpriv: add const to pointer FAIL
 Failed to load prog 'Operation not permitted'!
--
 #161/u unpriv: pass pointer to helper function Failed to create hash map 'Operation not permitted'!
--
 #162/u unpriv: indirectly pass pointer on stack to helper function Failed to create hash map 'Operation not permitted'!
--
 #167/u unpriv: spill/fill of ctx FAIL
 Failed to load prog 'Operation not permitted'!
--
 #173/u unpriv: write pointer into map elem value Failed to create hash map 'Operation not permitted'!
--
 #174/u alu32: mov u32 const FAIL
 Failed to load prog 'Operation not permitted'!
--
 #176/u unpriv: pass pointer to tail_call Failed to create prog array 'Operation not permitted'!
--
 #177/u unpriv: cmp map pointer with zero Failed to create hash map 'Operation not permitted'!
--
 #184/u runtime/jit: pass negative index to tail_call Failed to create prog array 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #185/u runtime/jit: pass > 32bit index to tail_call Failed to create prog array 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #186/u PTR_TO_STACK check high 1 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #187/u PTR_TO_STACK check high 2 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #193/u PTR_TO_STACK check low 1 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #200/u PTR_TO_STACK mixed reg/k, 1 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #201/u PTR_TO_STACK mixed reg/k, 2 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #202/u PTR_TO_STACK mixed reg/k, 3 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #203/u PTR_TO_STACK reg FAIL
 Failed to load prog 'Operation not permitted'!
--
 #204/u stack pointer arithmetic FAIL
 Failed to load prog 'Operation not permitted'!
--
 #273/u valid map access into an array with a constant Failed to create hash map 'Operation not permitted'!
--
 #274/u valid map access into an array with a register Failed to create hash map 'Operation not permitted'!
--
 #275/u valid map access into an array with a variable Failed to create hash map 'Operation not permitted'!
--
 #276/u valid map access into an array with a signed variable Failed to create hash map 'Operation not permitted'!
--
 #277/u invalid map access into an array with a constant Failed to create hash map 'Operation not permitted'!
--
 #278/u invalid map access into an array with a register Failed to create hash map 'Operation not permitted'!
--
 #279/u invalid map access into an array with a variable Failed to create hash map 'Operation not permitted'!
--
 #280/u invalid map access into an array with no floor check Failed to create hash map 'Operation not permitted'!
--
 #281/u invalid map access into an array with a invalid max check Failed to create hash map 'Operation not permitted'!
--
 #282/u invalid map access into an array with a invalid max check Failed to create hash map 'Operation not permitted'!
--
 #289/u invalid map access from else condition Failed to create hash map 'Operation not permitted'!
--
 #305/u leak pointer into ctx 1 Failed to create hash map 'Operation not permitted'!
--
 #307/u leak pointer into ctx 3 Failed to create hash map 'Operation not permitted'!
--
 #308/u leak pointer into map val Failed to create hash map 'Operation not permitted'!
--
 #341/u map element value is preserved across register spilling Failed to create hash map 'Operation not permitted'!
--
 #342/u map element value or null is marked on register spilling Failed to create hash map 'Operation not permitted'!
--
 #343/u map element value store of cleared call register Failed to create hash map 'Operation not permitted'!
--
 #344/u map element value with unaligned store Failed to create hash map 'Operation not permitted'!
--
 #345/u map element value with unaligned load Failed to create hash map 'Operation not permitted'!
--
 #346/u map element value illegal alu op, 1 Failed to create hash map 'Operation not permitted'!
--
 #347/u map element value illegal alu op, 2 Failed to create hash map 'Operation not permitted'!
--
 #348/u map element value illegal alu op, 3 Failed to create hash map 'Operation not permitted'!
--
 #349/u map element value illegal alu op, 4 Failed to create hash map 'Operation not permitted'!
--
 #350/u map element value illegal alu op, 5 Failed to create hash map 'Operation not permitted'!
--
 #351/u map element value is preserved across register spilling Failed to create hash map 'Operation not permitted'!
--
 #381/u invalid and of negative number Failed to create hash map 'Operation not permitted'!
--
 #382/u invalid range check Failed to create hash map 'Operation not permitted'!
--
 #383/u map in map access Failed to create array 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #384/u invalid inner map pointer Failed to create array 'Operation not permitted'!
--
 #385/u forgot null checking on the inner map pointer Failed to create array 'Operation not permitted'!
--
 #391/u ld_abs: check calling conv, r7 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #398/u ld_ind: check calling conv, r7 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #405/u bounds checks mixing signed and unsigned, positive bounds Failed to create hash map 'Operation not permitted'!
--
 #406/u bounds checks mixing signed and unsigned Failed to create hash map 'Operation not permitted'!
--
 #407/u bounds checks mixing signed and unsigned, variant 2 Failed to create hash map 'Operation not permitted'!
--
 #408/u bounds checks mixing signed and unsigned, variant 3 Failed to create hash map 'Operation not permitted'!
--
 #409/u bounds checks mixing signed and unsigned, variant 4 Failed to create hash map 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #410/u bounds checks mixing signed and unsigned, variant 5 Failed to create hash map 'Operation not permitted'!
--
 #412/u bounds checks mixing signed and unsigned, variant 7 Failed to create hash map 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #413/u bounds checks mixing signed and unsigned, variant 8 Failed to create hash map 'Operation not permitted'!
--
 #414/u bounds checks mixing signed and unsigned, variant 9 Failed to create hash map 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #415/u bounds checks mixing signed and unsigned, variant 10 Failed to create hash map 'Operation not permitted'!
--
 #416/u bounds checks mixing signed and unsigned, variant 11 Failed to create hash map 'Operation not permitted'!
--
 #417/u bounds checks mixing signed and unsigned, variant 12 Failed to create hash map 'Operation not permitted'!
--
 #418/u bounds checks mixing signed and unsigned, variant 13 Failed to create hash map 'Operation not permitted'!
--
 #419/u bounds checks mixing signed and unsigned, variant 14 Failed to create hash map 'Operation not permitted'!
--
 #420/u bounds checks mixing signed and unsigned, variant 15 Failed to create hash map 'Operation not permitted'!
--
 #421/u subtraction bounds (map value) variant 1 Failed to create hash map 'Operation not permitted'!
--
 #422/u subtraction bounds (map value) variant 2 Failed to create hash map 'Operation not permitted'!
--
 #423/u check subtraction on pointers for unpriv Failed to create hash map 'Operation not permitted'!
--
 #424/u bounds check based on zero-extended MOV Failed to create hash map 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #425/u bounds check based on sign-extended MOV. test1 Failed to create hash map 'Operation not permitted'!
--
 #426/u bounds check based on sign-extended MOV. test2 Failed to create hash map 'Operation not permitted'!
--
 #429/u bounds check after truncation of non-boundary-crossing range Failed to create hash map 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #430/u bounds check after truncation of boundary-crossing range (1) Failed to create hash map 'Operation not permitted'!
--
 #431/u bounds check after truncation of boundary-crossing range (2) Failed to create hash map 'Operation not permitted'!
--
 #432/u bounds check after wrapping 32-bit addition Failed to create hash map 'Operation not permitted'!
 FAIL
 Failed to load prog 'Operation not permitted'!
 #433/u bounds check after shift with oversized count operand Failed to create hash map 'Operation not permitted'!
--
 #434/u bounds check after right shift of maybe-negative number Failed to create hash map 'Operation not permitted'!
--
 #435/u bounds check after 32-bit right shift with 64-bit input Failed to create hash map 'Operation not permitted'!
--
 #436/u bounds check map access with off+size signed 32bit overflow. test1 Failed to create hash map 'Operation not permitted'!
--
 #437/u bounds check map access with off+size signed 32bit overflow. test2 Failed to create hash map 'Operation not permitted'!
--
 #438/u bounds check map access with off+size signed 32bit overflow. test3 Failed to create hash map 'Operation not permitted'!
--
 #439/u bounds check map access with off+size signed 32bit overflow. test4 Failed to create hash map 'Operation not permitted'!
--
 #440/u pointer/scalar confusion in state equality check (way 1) Failed to create hash map 'Operation not permitted'!
--
 #441/u pointer/scalar confusion in state equality check (way 2) Failed to create hash map 'Operation not permitted'!
--
 #449/u varlen_map_value_access pruning Failed to create hash map 'Operation not permitted'!
--
 #539/u masking, test out of bounds 1 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #540/u masking, test out of bounds 2 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #541/u masking, test out of bounds 3 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #542/u masking, test out of bounds 4 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #543/u masking, test out of bounds 5 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #544/u masking, test out of bounds 6 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #545/u masking, test out of bounds 7 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #546/u masking, test out of bounds 8 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #547/u masking, test out of bounds 9 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #548/u masking, test out of bounds 10 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #549/u masking, test out of bounds 11 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #550/u masking, test out of bounds 12 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #551/u masking, test in bounds 1 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #552/u masking, test in bounds 2 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #553/u masking, test in bounds 3 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #554/u masking, test in bounds 4 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #555/u masking, test in bounds 5 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #556/u masking, test in bounds 6 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #557/u masking, test in bounds 7 FAIL
 Failed to load prog 'Operation not permitted'!
--
 #558/u masking, test in bounds 8 FAIL
 Failed to load prog 'Operation not permitted'!
....
 Summary: 551 PASSED, 286 FAILED

It looks like these tests are "unprivileged tests".

Bisect shows this happens between 4.15.0-169-generic and 4.15.0-171-generic

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Please find attachment for the complete test log of this test-verifier test.

tags: added: 4.15 bionic oracle sru-20220620
tags: added: ubuntu-bpf
Po-Hsu Lin (cypressyew)
description: updated
Po-Hsu Lin (cypressyew)
description: updated
Changed in linux (Ubuntu):
status: New → Fix Released
summary: - Some tests in test_verifier from ubuntu_bpf failed with "Failed to load
- prog 'Operation not permitted'" on B-4.15
+ unprivileged tests in test_verifier from ubuntu_bpf failed with "Failed
+ to load prog 'Operation not permitted'" on B-4.15
Changed in linux (Ubuntu Bionic):
status: New → In Progress
assignee: nobody → Po-Hsu Lin (cypressyew)
Changed in ubuntu-kernel-tests:
assignee: nobody → Po-Hsu Lin (cypressyew)
status: New → In Progress
Po-Hsu Lin (cypressyew)
description: updated
description: updated
description: updated
Changed in linux (Ubuntu Bionic):
status: In Progress → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/4.15.0-190.201 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Verified with B-AWS-4.15.0-1138.149

The test will be skipped now:

 Running './test_verifier'
 #0/u add+sub+mul SKIP
 #0/p add+sub+mul OK
 #1/u DIV32 by 0, zero check 1 SKIP
 #1/p DIV32 by 0, zero check 1 OK
 #2/u DIV32 by 0, zero check 2 SKIP
 #2/p DIV32 by 0, zero check 2 OK
 #3/u DIV64 by 0, zero check SKIP
 #3/p DIV64 by 0, zero check OK
 #4/u MOD32 by 0, zero check 1 SKIP

tags: added: verification-done-bionic
removed: verification-needed-bionic
Changed in ubuntu-kernel-tests:
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (12.7 KiB)

This bug was fixed in the package linux - 4.15.0-191.202

---------------
linux (4.15.0-191.202) bionic; urgency=medium

  * CVE-2022-2586
    - SAUCE: netfilter: nf_tables: do not allow SET_ID to refer to another table
    - SAUCE: netfilter: nf_tables: do not allow RULE_ID to refer to another chain

  * CVE-2022-2588
    - SAUCE: net_sched: cls_route: remove from list when handle is 0

  * CVE-2022-34918
    - netfilter: nf_tables: stricter validation of element data

  * BUG: kernel NULL pointer dereference, address: 0000000000000008
    (LP: #1981658)
    - tcp: make sure treq->af_specific is initialized

linux (4.15.0-190.201) bionic; urgency=medium

  * bionic/linux: 4.15.0-190.201 -proposed tracker (LP: #1981321)

  * CVE-2022-1679
    - SAUCE: ath9k: fix use-after-free in ath9k_hif_usb_rx_cb

  * Bionic update: upstream stable patchset 2022-07-06 (LP: #1980879)
    - MIPS: Use address-of operator on section symbols
    - block: drbd: drbd_nl: Make conversion to 'enum drbd_ret_code' explicit
    - can: grcan: grcan_probe(): fix broken system id check for errata workaround
      needs
    - can: grcan: only use the NAPI poll budget for RX
    - Bluetooth: Fix the creation of hdev->name
    - mmc: rtsx: add 74 Clocks in power on flow
    - mm: hugetlb: fix missing cache flush in copy_huge_page_from_user()
    - mm: userfaultfd: fix missing cache flush in mcopy_atomic_pte() and
      __mcopy_atomic()
    - ALSA: pcm: Fix races among concurrent hw_params and hw_free calls
    - ALSA: pcm: Fix races among concurrent read/write and buffer changes
    - ALSA: pcm: Fix races among concurrent prepare and hw_params/hw_free calls
    - ALSA: pcm: Fix races among concurrent prealloc proc writes
    - ALSA: pcm: Fix potential AB/BA lock with buffer_mutex and mmap_lock
    - VFS: Fix memory leak caused by concurrently mounting fs with subtype
    - batman-adv: Don't skb_split skbuffs with frag_list
    - net: Fix features skip in for_each_netdev_feature()
    - ipv4: drop dst in multicast routing path
    - netlink: do not reset transport header in netlink_recvmsg()
    - mac80211_hwsim: call ieee80211_tx_prepare_skb under RCU protection
    - hwmon: (ltq-cputemp) restrict it to SOC_XWAY
    - s390/ctcm: fix variable dereferenced before check
    - s390/ctcm: fix potential memory leak
    - s390/lcs: fix variable dereferenced before check
    - net/smc: non blocking recvmsg() return -EAGAIN when no data and
      signal_pending
    - net: sfc: ef10: fix memory leak in efx_ef10_mtd_probe()
    - hwmon: (f71882fg) Fix negative temperature
    - ASoC: max98090: Reject invalid values in custom control put()
    - ASoC: max98090: Generate notifications on changes for custom control
    - ASoC: ops: Validate input values in snd_soc_put_volsw_range()
    - tcp: resalt the secret every 10 seconds
    - usb: cdc-wdm: fix reading stuck on device close
    - USB: serial: pl2303: add device id for HP LM930 Display
    - USB: serial: qcserial: add support for Sierra Wireless EM7590
    - USB: serial: option: add Fibocom L610 modem
    - USB: serial: option: add Fibocom MA510 modem
    - cgroup/cpuset: Remove cpus_allowed/mems_allowed setup in cpuset_init_smp()
 ...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.