[SRU] riscv64 images fail to boot with libvirt and u-boot-qemu in 22.10

Steps to reproduce:

1. Download the RISC-V server install image for 22.10: https://cdimage.ubuntu.com/releases/22.10/release/ubuntu-22.10-live-server-riscv64.img.gz

2. Decompress the file and save it to /var/lib/libvirt/images/ubuntu-22.10-live-server-riscv64.img

3. Create a VM in libvirt with the following XML:

cat << EOF > riscv-guest.xml
<?xml version="1.0"?>
<domain type="qemu">
  <name>ubuntu22.10-riscv64</name>
  <os>
    <type arch="riscv64" machine="virt">hvm</type>
    <kernel>/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf</kernel>
  </os>
  <memory unit="GiB">2</memory>
  <vcpu placement="static">4</vcpu>
  <devices>
    <emulator>usr/bin/qemu-system-riscv64</emulator>
    <disk type="file" device="disk" cache="none">
      <driver name="qemu" type="raw"/>
      <source file="/var/lib/libvirt/images/ubuntu-22.10-live-server-riscv64.img"/>
      <target dev="sda" bus="scsi"/>
    </disk>
    <controller type="scsi" model="virtio-scsi"/>
    <interface type="network">
      <source network="default"/>
      <target dev="vnet0"/>
    </interface>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
  </devices>
</domain>
EOF

4. Define the VM with "virsh define riscv-guest.xml"

5. Start the VM with "virsh start ubuntu22.10-riscv64"

6. Open the console with "virsh console ubuntu22.10-riscv64"

7. You can now see how the VM constantly reboots with the error message reported in the bug. This also occurs with the preinstalled image from https://cdimage.ubuntu.com/releases/22.10/release/ubuntu-22.10-preinstalled-server-riscv64+unmatched.img.xz

The following command works and the VM boots successfully:

sudo qemu-system-riscv64 \
        -machine virt -nographic -m 2048 -smp 4 \
        -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf \
        -drive file=/var/lib/libvirt/images/ubuntu-22.10-preinstalled-server-riscv64+unmatched.img,format=raw,if=virtio

To debug the issue, we can add '-gdb tcp::1234' to the command line.

virt-xml ubuntu22.10-riscv64 --edit --confirm --qemu-commandline '-gdb tcp::1234'

On the U-Boot command line execute bdinfo to retrieve the relocation address and the debug with

gdb-multiarch u-boot -ex 'target remote localhost:1234'
add-symbol-file u-boot <relocaddr>

The command 'virtio scan' which is part of distroboot leads to a illegal memory access. The following upstream bug fixes the crash:

04dd7c8e5624 ("virtio: pci: fix bug of virtio_pci_map_capability")

After adding the patch the crash is gone but the virtio drive is not detected.

Upstream U-Boot (2023.01-rc1-00041-g0cbeed4f66) also cannot detect the drive provided by libvirt.

The qemu command used by libvirt is:

usr/bin/qemu-system-riscv64 -name guest=ubuntu22.10-riscv64,debug-threads=on -S -object {"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-9-ubuntu22.10-riscv64/master-key.aes"} -machine virt,usb=off,dump-guest-core=off,memory-backend=riscv_virt_board.ram -accel tcg -m 2048 -object {"qom-type":"memory-backend-ram","id":"riscv_virt_board.ram","size":2147483648} -overcommit mem-lock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 12ec3afe-8af9-46c5-a194-492a79735de1 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=32,server=on,wait=off -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf -device {"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"} -device {"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"} -device {"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"} -device {"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"} -device {"driver":"virtio-scsi-pci","id":"scsi0","bus":"pci.2","addr":"0x0"} -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/ubuntu-22.10-live-server-riscv64.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"} -device {"driver":"scsi-hd","bus":"scsi0.0","channel":0,"scsi-id":0,"lun":0,"device_id":"drive-scsi0-0-0-0","drive":"libvirt-1-format","id":"scsi0-0-0-0","bootindex":1} -netdev tap,fd=33,id=hostnet0 -device {"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:3a:31:4b","bus":"pci.1","addr":"0x0"} -chardev pty,id=charserial0 -serial chardev:charserial0 -audiodev {"id":"audio1","driver":"none"} -device {"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.3","addr":"0x0"} -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on

Status changed to 'Confirmed' because the bug affects multiple users.

@itrue: Please, define the disk as follows:

    <disk type="file" device="disk" cache="none">
      <driver name="qemu" type="raw"/>
      <source file="/var/lib/libvirt/images/ubuntu-22.10-live-server-riscv64.img"/>
      <target dev='vda' bus='virtio'/>
    </disk>

Thanks @xypron - everything's working properly now. I can boot the VM using the above libvirt XML snippet (and your virtio bus change) and the patched U-Boot you provided.

This bug was fixed in the package u-boot - 2022.07+dfsg-1ubuntu6

---------------
u-boot (2022.07+dfsg-1ubuntu6) lunar; urgency=medium

  * Fix probing of virtio devices (LP: #1996285)
    d/p/virtio-pci-fix-bug-of-virtio_pci_map_capability.patch

 -- Heinrich Schuchardt <email address hidden> Fri, 11 Nov 2022 16:14:13 +0100

sru information missing from the description

Hello Isaac, or anyone else affected,

Accepted u-boot into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/u-boot/2022.07+dfsg-1ubuntu4.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

SRU to Kinetic tested successfully.

Launching with the following definition get's us into U-Boot. The command 'virtio scan' succeeds. But U-Boot does not have a virtio-scsi driver. So the image does not boot.

<?xml version="1.0"?>
<domain type="qemu">
  <name>ubuntu22.10-riscv64</name>
  <os>
    <type arch="riscv64" machine="virt">hvm</type>
    <kernel>/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf</kernel>
  </os>
  <memory unit="GiB">2</memory>
  <vcpu placement="static">4</vcpu>
  <devices>
    <emulator>usr/bin/qemu-system-riscv64</emulator>
    <disk type="file" device="disk" cache="none">
      <driver name="qemu" type="raw"/>
      <source file="/tmp/ubuntu-22.10-live-server-riscv64.img"/>
      <target dev="sda" bus="scsi"/>
    </disk>
    <controller type="scsi" model="virtio-scsi"/>
    <interface type="network">
      <source network="default"/>
      <target dev="vnet0"/>
    </interface>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
  </devices>
</domain>

With a change to the <target /> definition U-Boot's virtio-blk driver is used and booting succeds.

<?xml version="1.0"?>
<domain type="qemu">
  <name>ubuntu22.10-riscv64</name>
  <os>
    <type arch="riscv64" machine="virt">hvm</type>
    <kernel>/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf</kernel>
  </os>
  <memory unit="GiB">2</memory>
  <vcpu placement="static">4</vcpu>
  <devices>
    <emulator>usr/bin/qemu-system-riscv64</emulator>
    <disk type="file" device="disk" cache="none">
      <driver name="qemu" type="raw"/>
      <source file="/tmp/ubuntu-22.10-live-server-riscv64.img"/>
      <!-- <target dev="sda" bus="scsi"/> -->
      <target dev="vda" />
    </disk>
    <controller type="scsi" model="virtio-scsi"/>
    <interface type="network">
      <source network="default"/>
      <target dev="vnet0"/>
    </interface>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
  </devices>
</domain>

To actually use the installer we have to add a second drive:

<?xml version="1.0"?>
<domain type="qemu">
  <name>ubuntu22.10-riscv64</name>
  <os>
    <type arch="riscv64" machine="virt">hvm</type>
    <kernel>/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf</kernel>
  </os>
  <memory unit="GiB">2</memory>
  <vcpu placement="static">4</vcpu>
  <devices>
    <emulator>usr/bin/qemu-system-riscv64</emulator>
    <disk type="file" device="disk" cache="none">
      <driver name="qemu" type="raw"/>
      <source file="/tmp/ubuntu-22.10-live-server-riscv64.img"/>
      <target dev="vda" />
    </disk>
    <disk type="file" device="disk" cache="none">
      <driver name="qemu" type="raw"/>
      <source file="/tmp/disk.img"/>
      <!-- <target dev="sda" bus="scsi"/> -->
      <target dev="vdb" />
    </disk>
    <controller type="scsi" model="virtio-scsi"/>
    <interface type="network">
      <source network="default"/>
      <target dev="vnet0"/>
    </interface>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
  </devices>
</domain>

Tested OpenSBI 1.1-0ubuntu0.22.04.1 successfully on Jammy

I installed u-boot-nezha_2022.04+git20220405.7446a472-0ubuntu0.2_all.deb which is based on OpenSBI 1.1-0ubuntu0.22.04.1 both on a "Sipeed Lichee RV Dock" and a "Allwinner D1 Nezha".
Both boards boot successfully via GRUB into Ubuntu Jammy.
On both boards the command 'clang -v' executes successfully.
On both boards the command 'reboot' executes successfully.

I installed a U-Boot based on OpenSBI 1.1-0ubuntu0.22.04.1 on an Unmatched board.
The board boot successfully via GRUB into Ubuntu.
The command 'clang -v' executes successfully.
The command 'reboot' executes successfully.

I started a preinstalled Jammy image on QEMU using OpenSBI 1.1-0ubuntu0.22.04.1
The image boots successfully via GRUB into Ubuntu Jammy.
The command 'clang -v' executes successfully.
The command 'reboot' executes successfully.

This bug was fixed in the package u-boot - 2022.07+dfsg-1ubuntu4.1

---------------
u-boot (2022.07+dfsg-1ubuntu4.1) kinetic; urgency=medium

  * Rebuild against updated OpenSBI
    Fixes emulation of fence.tso (LP: #1995860)
  * Fix probing of virtio devices (LP: #1996285)
    d/p/virtio-pci-fix-bug-of-virtio_pci_map_capability.patch
  * Enable reset via SBI on PolarFire Icicle Kit (LP: #1995932)
    d/p/riscv64/enable-reset-via-SBI-on-PolarFire-Icicle-Kit.patch

 -- Heinrich Schuchardt <email address hidden> Fri, 18 Nov 2022 12:51:39 +0100

The verification of the Stable Release Update for u-boot has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.