Ubuntu
qemu package

Bug #2012763
Comment #55

Comment 55 for bug 2012763

Revision history for this message

Sergio Durigan Junior (sergiodj) wrote on 2024-02-01:

#55

Jeff et al,

After exchanging some emails with a QEMU developer (David Woodhouse; kudos to him), I tried a different incantation that seemed to work here. Unfortunately, I don't have a bare-metal Jammy machine ready to test this, so I used a Mantic one. Here is the command line I used:

$ sudo qemu-system-x86_64 -M pc-q35-mantic-maxcpus,accel=kvm,kernel-irqchip=split -device intel-iommu,intremap=on -smp cpus=300,maxcpus=300 -enable-kvm -net none -m 4096M -nographic -kernel /boot/vmlinuz -initrd /boot/initrd.img -chardev stdio,mux=on,id=char0 -mon chardev=char0,mode=readline -serial chardev:char0 -append "console=ttyS0"

The trick here is the use of the following options:

-M pc-q35-mantic-maxcpus,accel=kvm,kernel-irqchip=split -device intel-iommu,intremap=on

This will effectively remap the host's IOMMU into the guest, which (as David explained) is a required step to make QEMU work well with more than 288 vCPUs. It's also necessary to provide more memory to the VM (hence the "-m 4096M" option), because the kernel needs it in order to proper allocate enough data structures to represent the 300 vCPUs we're asking (otherwise, you will see a kernel panic complaining that there's not enough memory).

I would like to ask if you guys can give this a try on a *Mantic* system that *actually* has more than 288 CPUs. My test system "only" has 12 CPUs...

The PPA where you can find this new QEMU build is the same as before:

https://launchpad.net/~sergiodj/+archive/ubuntu/qemu

The QEMU package version is 1:8.0.4+dfsg-1ubuntu3.23.10.3~ppa2.

When launching the VM, you can use one of the following machine types:

pc-i440fx-mantic-maxcpus Ubuntu 23.10 PC (i440FX + PIIX, maxcpus=1024, 1996)
pc-q35-mantic-maxcpus Ubuntu 23.10 PC (Q35 + ICH9, maxcpus=1024, 2009)

Please let me know how it goes. If you can, please also test it using a real disk image.

I believe this is good news; it seems that we won't need to patch Mantic/Noble QEMUs. I still need to check Jammy's.

Thanks.

Jeff et al,

After exchanging some emails with a QEMU developer (David Woodhouse; kudos to him), I tried a different incantation that seemed to work here.  Unfortunately, I don't have a bare-metal Jammy machine ready to test this, so I used a Mantic one.  Here is the command line I used:

The trick here is the use of the following options:

-M pc-q35-mantic-maxcpus,accel=kvm,kernel-irqchip=split -device intel-iommu,intremap=on

This will effectively remap the host's IOMMU into the guest, which (as David explained) is a required step to make QEMU work well with more than 288 vCPUs.  It's also necessary to provide more memory to the VM (hence the "-m 4096M" option), because the kernel needs it in order to proper allocate enough data structures to represent the 300 vCPUs we're asking (otherwise, you will see a kernel panic complaining that there's not enough memory).

I would like to ask if you guys can give this a try on a *Mantic* system that *actually* has more than 288 CPUs.  My test system "only" has 12 CPUs...