grub2 2.12~rc1-4ubuntu1: 10_linux_zfs needs update for new sorting

I'm affected too. apt full-upgrade installs grub2 2.12 and I get:
Generating grub configuration file ...
/etc/grub.d/10_linux_zfs: 425: version_find_latest: not found
and then a lot of subsequent errors.
I've changed line 425 with
linux=$(echo $list_basename | tr ' ' '\n' | sort -V | head -1 | cat)
and the installation went smoot but now I have to reboot, cross my fingers and hope for the best (I'm not an expert in bash programming and grub)
Comparing i.e. /etc/grub.d/20_linux_xen between 2.06-2ubuntu18 and 2.12~rc1-4ubuntu1 I've seen that the code has been refactored around this (now removed) function, so 10_linux_zfs needs it too

Status changed to 'Confirmed' because the bug affects multiple users.

A first solution is available here [1]. It's not a patch for packaging grub2, but for the /etc/grub.d/10_linux_zfs file itself, distributed with grub-common.
Those changes are required due to the removal of the version_find_latest function in [2], after the introduction of changes in [3].
The solution itself is based on what has been done in [3].

Have fun :)

[1] https://gist.github.com/marinelli/3795a93f833d394cb18a9edff53dd780
[2] https://git.savannah.gnu.org/cgit/grub.git/commit/?id=a79c567f6b5820a8795c273a9eaabd06a1f92b29
[3] https://git.savannah.gnu.org/cgit/grub.git/commit/?id=99e05ab555f013b5ce45a3fd04f8ccd5f4e5bf95

Let's attach the patch I shared above.

The attachment "10_linux_zfs_grub_2.12~rc1.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

Description: grub is borked

Expecting: "sudo apt update" just works

Actual:

$ dpkg -l | grep -Ev '^ii|^rc|^ic'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=============================================-=======================================-============-================================================================================
iU grub-efi-amd64-signed 1.194+2.12~rc1-4ubuntu1 amd64 GRand Unified Bootloader, version 2 (EFI-AMD64 version, signed)
iF grub-pc 2.12~rc1-4ubuntu1 amd64 GRand Unified Bootloader, version 2 (PC/BIOS version)
iF linux-image-6.3.0-7-generic 6.3.0-7.7+1 amd64 Signed kernel image generic

$ sudo grub-mkconfig |& tail
set menu_color_highlight=black/light-gray
### END /etc/grub.d/05_debian_theme ###

### BEGIN /etc/grub.d/10_linux ###
### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/10_linux_zfs ###
/etc/grub.d/10_linux_zfs: 425: version_find_latest: not found
umount: /tmp/zfsmnt.XMN7oz: target is busy.
rmdir: failed to remove '/tmp/zfsmnt.XMN7oz': Device or resource busy

Platform:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Mantic Minotaur (development branch)
Release: 23.10
Codename: mantic

The patch "10_linux_zfs_grub_2.12~rc1.patch" worked for me in that I could "sudo apt upgrade" the system without the issues I posted in my previous comment (comment 6).

$ sudo apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
#
# You can verify the status of security fixes using the `pro fix` command.
# E.g., a recent Ruby vulnerability can be checked with: `pro fix USN-6219-1`
# For more detail see: https://ubuntu.com/security/notices/USN-6219-1
#
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
3 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
Setting up linux-image-6.3.0-7-generic (6.3.0-7.7+1) ...
Setting up grub-pc (2.12~rc1-4ubuntu1) ...
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: vmlinuz-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found linux image: vmlinuz-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found memtest86+x64 image: /BOOT/ubuntu_via8pv@/memtest86+x64.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
Setting up grub-efi-amd64-signed (1.194+2.12~rc1-4ubuntu1) ...
Installing grub to /boot/efi.
Installing for x86_64-efi platform.
grub-install: warning: EFI variables cannot be set on this system.
grub-install: warning: You will have to complete the GRUB setup manually.
Installation finished. No error reported.
Processing triggers for linux-image-6.3.0-7-generic (6.3.0-7.7+1) ...
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-6.3.0-7-generic
I: The initramfs will attempt to resume from /dev/vda3
I: (UUID=f29902c8-253f-4318-bf33-026715c7f5d4)
I: Set the RESUME variable to override this.
/etc/kernel/postinst.d/zz-update-grub:
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: vmlinuz-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found linux image: vmlinuz-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found memtest86+x64 image: /BOOT/ubuntu_via8pv@/memtest86+x64.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done

$ dpkg -l | grep -Ev '^ii|^rc|^ic'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ N...

This bug was fixed in the package grub2 - 2.12~rc1-4ubuntu2

---------------
grub2 (2.12~rc1-4ubuntu2) mantic; urgency=medium

  * ubuntu-zfs-enhance-support.patch: Adjustments for 2.12 library
    (LP: #2029260)
  * zfs: on_exit: Unmount ${MNTDIR}/boot before ${MNTDIR} (LP: #2031042)
  * Temporarily rmmod peimage for os-prober chainloader entries (LP: #2030810)

 -- Julian Andres Klode <email address hidden> Mon, 21 Aug 2023 14:26:07 +0200

As soon as 2.12~rc1-4ubuntu2 appeared in 23.10 kubuntu repo (yesterday) I upgraded. At reboot the system went directly, immediately to memtest. I was unable to fix the problem (i.e. supergrub does not support zfs), I had to reinstall from scratch in a new ssd (this time with ext4) and copy all my data from rpool of the old ssd. Don't know for sure if is the "fixed" grub that made this mess but...

I have the same issue with booting straight into memtest.

I wish I knew why "set -u" was added to /etc/grub.d/10_linux_zfs as it's easy to see that it was added; it is not easy to see why it was added.

"set -u" appears to be a significant factor in why zfs grub is failing.

sudo grub-mkconfig

[...skip some output...]

### BEGIN /etc/grub.d/10_linux_zfs ###
/etc/grub.d/10_linux_zfs: 229: version_sort_sort_has_v: parameter not set
Warning: didn't find any valid initrd or kernel.
### END /etc/grub.d/10_linux_zfs ###

grub-mkconfig works if I comment out the 'set -u' like this:

$ diff -u ~/10_linux_zfs-grub-4ubuntu1 10_linux_zfs
--- /home/user/10_linux_zfs-grub-4ubuntu1 2023-08-29 08:53:42.056383448 -0400
+++ 10_linux_zfs 2023-08-29 08:53:57.896516515 -0400
@@ -30,7 +30,7 @@
 export TEXTDOMAIN=grub
 export TEXTDOMAINDIR="${datarootdir}/locale"

-set -u
+# set -u

 ## Skip early if zfs utils isn't installed (instead of failing on first zpool list)
 if ! `which zfs >/dev/null 2>&1`; then

$ sudo grub-mkconfig > /dev/null
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
jFound linux image: vmlinuz-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found linux image: vmlinuz-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found memtest86+x64 image: /BOOT/ubuntu_via8pv@/memtest86+x64.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done

$ sudo dpkg-reconfigure grub-pc
grub-pc: Running grub-install ...
Installing for i386-pc platform.
Installation finished. No error reported.
  grub-install success for /dev/vda
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: vmlinuz-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.3.0-7-generic in rpool/ROOT/ubuntu_via8pv
Found linux image: vmlinuz-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found initrd image: initrd.img-6.2.0-21-generic in rpool/ROOT/ubuntu_via8pv
Found memtest86+x64 image: /BOOT/ubuntu_via8pv@/memtest86+x64.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done

Once 'set -u' is commented out, the system boots from zfs and displays the login screen instead of memtest.

Versions I tested with:

$ dpkg -l | grep grub
ii grub-common 2.12~rc1-4ubuntu2 amd64 GRand Unified Bootloader (common files)
ii grub-efi-amd64-bin 2.12~rc1-4ubuntu1 amd64 GRand Unified Bootloader, version 2 (EFI-AMD64 modules)
ii grub-efi-amd64-signed 1.194+2.12~rc1-4ubuntu1 amd64 GRand Unified Bootloader, version 2 (EFI-AMD64 version, signed)
ii ...

To recover, I used this guide:

https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/Ubuntu%2020.04%20Root%20on%20ZFS.html#rescuing-using-a-live-cd

for the step which goes "Do whatever you need to do to fix your system." I used:

Comment out 'set -u' for example:
vi /mnt/etc/grub.d/10_linux_zfs

update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: vmlinuz-6.3.0-7-generic in rpool/ROOT/ubuntu_jyphyz
Found initrd image: initrd.img-6.3.0-7-generic in rpool/ROOT/ubuntu_jyphyz
Found linux image: vmlinuz-6.2.0-23-generic in rpool/ROOT/ubuntu_jyphyz
Found initrd image: initrd.img-6.2.0-23-generic in rpool/ROOT/ubuntu_jyphyz
Found memtest86+x64 image: /BOOT/ubuntu_jyphyz@/memtest86+x64.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done

Earlier when I wrote that I wish I knew why 'set -u' was there, it was because I wanted some clues as to what the impact, if any, would be if I removed that 'set -u'.

The missing set -u hank was tracked in Bug #2033256 and just fixed in 4ubuntu3

This issue appears to be resolved from my POV.

I tried an OpenZFS install on the current live iso (2023-09-01) and
installation went well.
It is solved now from my POV too.

Thanks all! 👍

Le ven. 1 sept. 2023 à 20:25, Acceptable Name <email address hidden>
a écrit :

> This issue appears to be resolved from my POV.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (2032962).
> https://bugs.launchpad.net/bugs/2029260
>
> Title:
> grub2 2.12~rc1-4ubuntu1: 10_linux_zfs needs update for new sorting
>
> Status in grub2 package in Ubuntu:
> Fix Released
>
> Bug description:
> 10_linux_zfs.in still uses the old version sorting function which no
> longer exists
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/2029260/+subscriptions
>
>

--
---------------------------------------------
Bernard Tremblay
<email address hidden>
R: (418) 658-1411
C: (581) 988-1411
---------------------------------------------
Le but de Linux est de gérer vos ressources et faire le travail,
le but des OS propriétaire est de vous vendre d'autres licences...