Grub EFI amd64 no longer start EFI/Microsoft/Boot/bootmgfw.efi

Bug #1845289 reported by Joseph Maillardet
78
This bug affects 21 people
Affects Status Importance Assigned to Milestone
grub2 (Ubuntu)
Fix Released
High
Unassigned
Eoan
Fix Released
High
Unassigned

Bug Description

[Impact]
Dual-boot users who have somehow had shim uninstalled from their system / Secure Boot disabled.

[Test case]
(system with dual-boot setup for Ubuntu and Windows)
1) uninstall shim/shim-signed from the system
2) Run 'sudo grub-install -v'; ensure grubx64.efi is installed as the ubuntu bootentry (check 'sudo efibootmgr -v')
3) Reboot.
4) In system firmware, ensure Secure Boot is disabled.
5) Attempt to start Windows from the Grub menu.

Verify that Windows is chainloaded from Grub successfully; without returning to the GRUB menu or restarting the whole system.

[Regression potential]
Minimal; this is returning the state of the code paths for linuxefi validation back to how they were in GRUB 2.02; the extra check for the Secure Boot state is unnecessary at that point since it will be run again later when validating the images. This affects all validation code paths on UEFI but the net effect of reverting that change should be nil due to the duplication of code that already exists.

---

On a dual boot computer with Windows 10 Pro and Ubuntu 19.10, since a month, Grub was unable to boot Windows 10. Hitting the Windows entry line give me a 0.1s black screen and get me back to Grub Menu.

sudo update-grub give me :

Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.3.0-10-generic
Found initrd image: /boot/initrd.img-5.3.0-10-generic
Found Windows Boot Manager on /dev/sda1@/EFI/Microsoft/Boot/bootmgfw.efi
Adding boot menu entry for EFI firmware configuration
done

ProblemType: Bug
DistroRelease: Ubuntu 19.10
Package: grub-efi-amd64 2.04-1ubuntu5
ProcVersionSignature: Ubuntu 5.3.0-10.11-generic 5.3.0-rc8
Uname: Linux 5.3.0-10-generic x86_64
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Wed Sep 25 08:01:05 2019
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=fr_FR.UTF-8
 SHELL=/bin/bash
SourcePackage: grub2
UpgradeStatus: Upgraded to eoan on 2018-11-14 (314 days ago)

Revision history for this message
Joseph Maillardet (jokx) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in grub2 (Ubuntu):
status: New → Confirmed
Revision history for this message
Jay Tuckey (jay-tuckey) wrote :

I upgraded to the eoan beta yesterday, and I have the same issue.

Running update-grub gives me:
~> sudo update-grub
[sudo] password for jtuckey:
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.3.0-13-generic
Found initrd image: /boot/initrd.img-5.3.0-13-generic
Found linux image: /boot/vmlinuz-5.3.0-12-generic
Found initrd image: /boot/initrd.img-5.3.0-12-generic
Found Windows Boot Manager on /dev/sda1@/EFI/Microsoft/Boot/bootmgfw.efi
Adding boot menu entry for EFI firmware configuration
done
~>

The symptom I have is the same, I can see the windows boot option, but selecting it just reloads grub in about 0.1s.

I have attached my grub.cfg file, let me know if you want anything else.

Revision history for this message
Brian Murray (brian-murray) wrote :

@Jay - What version of grub2 do you have installed?

Changed in grub2 (Ubuntu):
importance: Undecided → High
tags: added: rls-ee-tracking
tags: removed: rls-ee-tracking
Revision history for this message
Jay Tuckey (jay-tuckey) wrote :

@brian-murray I have 2.04-1ubuntu9 installed, as well as these extras:

~> sudo apt list --installed '*grub*'
Listing... Done
grub-common/eoan,now 2.04-1ubuntu9 amd64 [installed]
grub-efi-amd64-bin/eoan,now 2.04-1ubuntu9 amd64 [installed,automatic]
grub-efi-amd64-signed/eoan,now 1.125+2.04-1ubuntu9 amd64 [installed]
grub-gfxpayload-lists/eoan,now 0.7 amd64 [installed]
grub-pc-bin/eoan,now 2.04-1ubuntu9 amd64 [installed]
grub-pc/eoan,now 2.04-1ubuntu9 amd64 [installed]
grub2-common/eoan,now 2.04-1ubuntu9 amd64 [installed]

I've included some extra info below, please let me know if you need any further info or want me to test anything.

Here is my machine info:
~> inxi -F
System: Host: jay-alien-ubu Kernel: 5.3.0-13-generic x86_64 bits: 64 Desktop: KDE Plasma 5.16.5
           Distro: Ubuntu 19.10 (Eoan Ermine)
Machine: Type: Desktop System: Alienware product: ASM100 v: N/A serial: <root required>
           Mobo: Dell model: 03V3TG v: A00 serial: <root required> UEFI: Alienware v: A07 date: 06/12/2018
CPU: Topology: Dual Core model: Intel Core i3-4130T bits: 64 type: MT MCP L2 cache: 3072 KiB
           Speed: 1097 MHz min/max: 800/2900 MHz Core speeds (MHz): 1: 1097 2: 1097 3: 1098 4: 1098
Graphics: Device-1: NVIDIA GM107M [GeForce GTX 860M] driver: nvidia v: 435.21
           Display: x11 server: X.Org 1.20.5 driver: nvidia unloaded: fbdev,modesetting,nouveau,vesa
           resolution: 1680x1050~60Hz
           OpenGL: renderer: GeForce GPU/PCIe/SSE2 v: 4.6.0 NVIDIA 435.21
Audio: Device-1: Intel 8 Series/C220 Series High Definition Audio driver: snd_hda_intel
           Device-2: NVIDIA driver: snd_hda_intel
           Device-3: Kingston type: USB driver: hid-generic,snd-usb-audio,usbhid
           Sound Server: ALSA v: k5.3.0-13-generic
Network: Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169
           IF: enp3s0 state: up speed: 1000 Mbps duplex: full mac: 98:90:96:a3:d9:75
           Device-2: Intel Wireless 3160 driver: iwlwifi
           IF: wlp4s0 state: down mac: 34:de:1a:37:86:24
Drives: Local Storage: total: 931.51 GiB used: 391.01 GiB (42.0%)
           ID-1: /dev/sda vendor: Samsung model: SSD 860 EVO 1TB size: 931.51 GiB
Partition: ID-1: / size: 468.35 GiB used: 390.97 GiB (83.5%) fs: ext4 dev: /dev/sda2
Sensors: System Temperatures: cpu: 44.0 C mobo: N/A gpu: nvidia temp: 36 C
           Fan Speeds (RPM): N/A
Info: Processes: 274 Uptime: 3m Memory: 11.65 GiB used: 1.90 GiB (16.3%) Shell: fish inxi: 3.0.36

Running update-grub gives me:
~> sudo update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.3.0-13-generic
Found initrd image: /boot/initrd.img-5.3.0-13-generic
Found linux image: /boot/vmlinuz-5.3.0-12-generic
Found initrd image: /boot/initrd.img-5.3.0-12-generic
Found Windows Boot Manager on /dev/sda1@/EFI/Microsoft/Boot/bootmgfw.efi
Adding boot menu entry for EFI firmware configuration
done

tags: added: id-5d978799ba24715c5ddcabc5
Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

If you use F8 or F12 (or whatever hotkey to get the firmware's boot menu); can you then run "Windows Boot Manager" and successfully start Windows?

How was the installation done? Could you please describe how the disk was partitioned for dual-booting, and the steps taked in the installer?

At this moment, I am unable to reproduce the failure. Here, dual-booting Windows works fine.

Changed in grub2 (Ubuntu Eoan):
status: Confirmed → Incomplete
Revision history for this message
Joseph Maillardet (jokx) wrote :

Yes, by using the F12 key, I can display the EFI start menu and start Windows with the "Windows Boot Manager" line.

This is a test machine that follows the development of "Ubuntu next". Initially installed with Ubuntu 17.10 beta, it has followed the updates of all Ubuntu development versions (18.04, 18.10, ...) until today. Now in 19.10, it will soon switch to Ubuntu Focal 20.04).

The Windows part has been re-installed from scratch with an Windows 10 18.03 USB flash drive installer and regularly updated. Now running W10-19.03.

After installing Windows, Grub was re-installed by starting the machine with Ubuntu-Live 18.04 and then entering the following commands:
$ sudo mount /dev/sda2 /mnt
$ for i in dev sys proc dev/pts; do sudo mount -o bind /$i /mnt/$i; done
$ sudo chroot /mnt
# apt purge grub-common grub-efi grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed grub2-common
# apt install grub-common grub-efi grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed grub2-common
# update-grub
(... reboot → Ubuntu ...)
$ sudo update-grub

Everything was working well until more or less half-time of 19.10 developpement.

Here my partitioning :

$ sudo parted /dev/sda print
Model: ATA M4-CT512M4SSD2 (scsi)
Disk /dev/sda: 512GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
 1 1049kB 1075MB 1074MB fat32 esp boot, esp
 2 1075MB 207GB 206GB ext4 Ubuntu
 3 207GB 345GB 137GB ntfs data msftdata
 4 345GB 345GB 16.8MB Microsoft reserved partition msftres
 5 345GB 512GB 167GB ntfs Basic data partition msftdata
 6 512GB 512GB 520MB ntfs hidden, diag

Revision history for this message
Joseph Maillardet (jokx) wrote :

Here the details of /boot file (with EFI partition mounted)

Revision history for this message
Joseph Maillardet (jokx) wrote :
Changed in grub2 (Ubuntu Eoan):
status: Incomplete → Confirmed
Revision history for this message
Jay Tuckey (jay-tuckey) wrote :

Hi @cyphermox, I have tested with F12 on my Alienware Alpha:
* F12
* Select Windows Boot Manager
* Goes straight to grub

So no difference regardless of which boot option I select.

How was the installation done?
I installed Ubuntu 19.04 the upgraded to 19.10.

Here's my disk layout:
~> sudo parted /dev/sda print
Model: ATA Samsung SSD 860 (scsi)
Disk /dev/sda: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
 1 1049kB 512MB 511MB fat32 boot, esp
 2 512MB 513GB 512GB ext4
 3 513GB 513GB 16.8MB Microsoft reserved partition msftres
 4 513GB 1000GB 488GB ntfs Basic data partition msftdata

Could you please describe how the disk was partitioned for dual-booting, and the steps taked in the installer?
I can't remember exactly what I did, as it was quite a while ago. However, the basic steps were:
* Install Ubuntu using 512GB
* Install Win10 into the remaining space
* Boot the Ubuntu usb and chroot into the installed ubuntu and re-install grub there.

Is there any extra info you want me to collect about my PC?

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

This is probably duplicate of https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1839317.

I have the same issue.

I can boot Windows 10 from BIOS/EFI boot menu, but can't boot it from grub.

Setting debug=chain shows that grub tries to use SecureBoot, but it's disabled.

Revision history for this message
ottergauze (ottergauze) wrote :

Same issue here. GRUB is my only way of accessing Windows which is what I use as a secondary for games and Windows-only software.

Hoping for a fix soon!

Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

This bug is *not* a duplicate of 1839317; since we're not looking at ARM-based systems here.

Could you please update your grub-efi-amd64 and grub-efi-amd64-signed packages (well, make sure your system is fully up to date), then run:

sudo grub-install -v

And include the output here.

Not all the failures here are for the same causes:

Joseph Maillardet (jokx), it looks like the packages are not all installed correctly, though I am not sure why (grub-install will tell); so you are booting GRUB directly, not through shim, so Secure Boot is being skipped in fun ways. This means chainloading Windows from GRUB is unlikely to work.

Ivan, it sounds like your Windows no longer has a bootloader -- otherwise you should still see "Windows Boot Manager" work to boot Windows. If you see GRUB again, then it means something went missing on disk, in which case, per my comment #6, we would need to see what is on disk under /boot/efi.

Changed in grub2 (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
JJC (jjcervera05) wrote :

Hi there

Same problem after update from 19.04 to 19.10.
I can't load Windows 10 from Grub: Grub EFI amd64 no longer start EFI/Microsoft/Boot/bootmgfw.efi

Please fix it as soon as possible.

Thank you.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

Bug 1839317 looks too similar to ignore the possibility of the same underlying issue. There's other person there besides me who reports the same behavior on amd64.

My Windows bootloader is fine.
I can boot Windows using my motherboard's EFI boot menu selector.
I can boot Windows if I downgrade GRUB to the version from Ubuntu 19.04 (2.02+dfsg1-12ubuntu2).

My setup is EFI + GPT. Secure Boot is disabled.

Partitions:

Device Start End Sectors Size Type
/dev/sda1 2048 821247 819200 400M EFI System
/dev/sda2 821248 1083391 262144 128M Microsoft reserved
/dev/sda3 1083392 210209396 209126005 99.7G Microsoft basic data (WINDOWS 10 SYSTEM IS HERE)
/dev/sda4 210210816 211257343 1046528 511M Windows recovery environment
/dev/sda5 211257344 420513791 209256448 99.8G Microsoft basic data
/dev/sda6 420513792 445679615 25165824 12G Linux filesystem (UBUNTU ROOT, EXT4)
/dev/sda7 445679616 468860927 23181312 11.1G Linux filesystem (UBUNTU HOME, XFS)

Mounts:

/dev/sda1 on /boot/efi
/dev/sda6 on /
/dev/sda7 on /home

/etc/default/grub custom options:

GRUB_TIMEOUT=5
GRUB_CMDLINE_LINUX_DEFAULT="quiet systemd.show_status=1"
GRUB_GFXMODE=2560x1440
GRUB_DISABLE_RECOVERY="true"
GRUB_DISABLE_SUBMENU=y
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_FONT=/boot/grub/terminus-32b.pf2

/etc/grub.d/30_os-prober renamed to /etc/grub.d/07_os-prober

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

For the sake of testing I've updated to the latest grub from 19.10 (2.04-1ubuntu12) and I can't boot Windows anymore. My exact steps were:

0. Ubuntu 19.10 with grub from 19.04. Everything works
1. Move /etc/grub.d/07_os-prober back to /etc/grub.d/30_os-prober to rule this change out
2. sudo apt update && sudo apt dist-upgrade
3. sudo grub-install /dev/sda
4. sudo update-grub
5. Reboot
6. Can't boot Windows anymore

I've compared grub config from 2.02 to 2.04. They look almost exactly the same. The only differences are:

- set vt_handoff=vt.handoff=1
+ set vt_handoff=vt.handoff=7

and

+### BEGIN /etc/grub.d/10_linux_zfs ###
+### END /etc/grub.d/10_linux_zfs ###

Windows part config:

menuentry 'Windows Boot Manager (on /dev/sda1)' --class windows --class os $menuentry_id_option 'osprober-efi-CAC2-FF0C' {
 insmod part_gpt
 insmod fat
 set root='hd0,gpt1'
 if [ x$feature_platform_search_hint = xy ]; then
   search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1 CAC2-FF0C
 else
   search --no-floppy --fs-uuid --set=root CAC2-FF0C
 fi
 chainloader /EFI/Microsoft/Boot/bootmgfw.efi
}

I've recorded video of boot process with "set debug=all" grub config. Send me a message through https://launchpad.net/~xeron-oskom/+contactuser if you want to see it.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

Downgraded to 2.02 and everything works again.

Major difference in debug output on 2.02 is the line which says:

2.02:

shim bla bla not available
shim_lock bla bla
linuxefi_secure_validate: 0

2.04:

SecureBoot: 0
SetupMode: 1
secure boot not enabled, not validating bla bla linuxefi_secure_validate: 1

If you didn't notice the difference - 2.02 detects that Secure Boot is disabled and sets:

linuxefi_secure_validate: 0

2.04 also detects that Secure Boot is disabled, however sets:

linuxefi_secure_validate: 1.

This issue looks almost exactly the same as:

https://github.com/aarch64-laptops/build/issues/18
and as Bug 1839317

Please confirm and link with 1839317.

Revision history for this message
Joseph Maillardet (jokx) wrote :

@cyphermox

I have executed the commands as indicated. I reinstalled grub-efi-amd64 and grub-efi-amd64-signed and ran grub-install -v. You will find the results of the commands in the attached file.

Windows still refuses to boot from Grub.

Revision history for this message
Joseph Maillardet (jokx) wrote :

I noticed in the attached document of comment #18 that some files were missing:
grub-install: info: cannot open `/usr/lib/shim/shim/fbx64.efi': No such file or directory.
grub-install: info: cannot open `/usr/lib/shim/mmx64.efi': No such file or directory.

Because these files come from the "shim" package, I installed it. Then, I restarted "grub-install" but it didn't have any effect, Windows still refuses to boot from Grub.

Is it normal that the shim package is not a dependency of grub-efi-amd64-signed?

Changed in grub2 (Ubuntu):
status: Incomplete → Confirmed
Revision history for this message
Dane Powell (danepowell) wrote :

I was also experiencing this problem and fixed it by using the shim bootloader instead of Grub. I'm not sure if using Grub directly _should_ work, maybe there's still a bug here, and I'm also not sure what changed during the Ubuntu upgrade (I assume it was using Grub directly before as well, but never thought to check and it's too late now).

Revision history for this message
WinEunuchs2Unix (ricklee518) wrote :

For some this problem is answered by: [Windows option just reloads grub bootloader after updating to Ubuntu 19.10](https://askubuntu.com/questions/1183528/windows-option-just-reloads-grub-bootloader-after-updating-to-ubuntu-19-10/1183623#1183623).

Before following steps in answer use `boot-repair` to get diagnosis to ensure same problem.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

Take a look at this diff between disco and eoan:

--- grub2-2.02+dfsg1/grub-core/loader/efi/linux.c 2019-10-29 00:23:53.000000000 +0000
+++ grub2-2.04/grub-core/loader/efi/linux.c 2019-10-29 00:10:44.000000000 +0000
@@ -23,6 +23,7 @@
 #include <grub/efi/efi.h>
 #include <grub/efi/pe32.h>
 #include <grub/efi/linux.h>
+#include <grub/efi/sb.h>

 #define SHIM_LOCK_GUID \
  { 0x605dab50, 0xe046, 0x4300, {0xab, 0xb6, 0x3d, 0xd8, 0x10, 0xdd, 0x8b, 0x23} }
@@ -40,6 +41,13 @@
   grub_efi_shim_lock_t *shim_lock;
   int status;

+ if (! grub_efi_secure_boot())
+ {
+ grub_dprintf ("linuxefi", "secure boot not enabled, not validating");
+ return 1;
+ }
+
+ grub_dprintf ("linuxefi", "Locating shim protocol\n");
   shim_lock = grub_efi_locate_protocol(&guid, NULL);
   grub_dprintf ("secureboot", "shim_lock: %p\n", shim_lock);
   if (!shim_lock)

if grub_efi_secure_boot() returns 0 then grub_linuxefi_secure_validate() prints "secure boot not enabled" but returns 1? Doesn't sound right. Should return 0 I guess?

And if grub_linuxefi_secure_validate() returns 1 then this chainloader code switches to secureboot:

  rc = grub_linuxefi_secure_validate((void *)((grub_addr_t) address), fsize);
  grub_dprintf ("chain", "linuxefi_secure_validate: %d\n", rc);
  if (rc > 0)
    {
      grub_file_close (file);
      grub_loader_set (grub_secureboot_chainloader_boot,
         grub_secureboot_chainloader_unload, 0);
      return 0;
    }
  else if (rc == 0)
    {
      grub_load_and_start_image(boot_image);
      grub_file_close (file);
      grub_loader_set (grub_chainloader_boot, grub_chainloader_unload, 0);

      return 0;
    }

resulting in trying to secure boot windows efi image instead of just calling grub_load_and_start_image().

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

Well may be grub_linuxefi_secure_validate() should still return 1 as intended by this code to skip validation in some cases but then chainloader should not depend on this value.

I hope grub2 package maintainers will find some time to take a look at this.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

Possible fix is to change chainloader to check `grub_efi_secure_boot()` and not `grub_linuxefi_secure_validate()`.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

Possible patch. Untested, just an idea.

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Patch to keep shim protocol for chainload but fix boot when SB disabled" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

PPA with this patch – https://launchpad.net/~xeron-oskom/+archive/ubuntu/grub2

Reproduced this big in VirtualBox and tested this PPA and it fixes the issue.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

For anyone trying to reproduce:

VirtualBox with EFI enabled with Win 10 Pro x86_64 installed first and Ubuntu 19.10 installed after Windows to the latest additional partition of /dev/sda. On fresh install with all updates:

1. Make sure you can boot both Windows and Ubuntu
2. Purge all shim* and grub* packages
3. sudo apt autoremove --purge
4. sudo rm -rf /boot/efi/EFI/ubuntu /boot/efi/EFI/grub /boot/grub
5. sudo apt install grub-efi-amd64-bin
6. sudo grub-install /dev/sda
7. sudo update-grub
8. Try to boot windows. Grub should blink and return you back to the menu.

You should have only the following packages installed:

* grub-efi-amd64-bin
* grub-efi-amd64
* grub-common
* grub2-common

There should be no shim packages.

Then try:

sudo add-apt-repository ppa:xeron-oskom/grub2
sudo apt update
sudo apt dist-upgrade
sudo grub-install /dev/sda
sudo update-grub

You should be able to boot Windows again.

Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

No; this patch is wrong.

For starters, you really ought to have shim installed, and that is the primary cause for such failures -- somehow your install is incomplete, and shim was removed either because the package was removed, or through the use of boot-repair (IIRC it replaces shim with using grub directly, which is not supported).

Then, the code for the chainloader already is supposed to handle Secure Boot being off -- if it's disabled, the linuxefi_secure_validate() function sees that and returns 1 (image is deemed valid, since SB is disabled). Then we run grub_secureboot_chainloader_boot(), which will execute handle_image(), which should immediately check read_header() and find out, again, whether Secure Boot is enabled and/or if shim is available; returning with whatever status was retrieved reading the header for the binary that is to be chainloaded.

I'd expect that if Secure Boot is not enabled, you would again have a return code of 0 there, and move on to using the normal chainload behavior of load_image and start_image.

Please, if you're running into this issue, try stopping at the grub menu, and edit the chainload entry to add:

set debug="chain"

At the top, taking a screenshot or video of the boot process that you can then attach here. We really need to know if this is caused by one unusually built binary, or what precisely is going on before making any patches to GRUB.

Changed in grub2 (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

This patch may be wrong but it fixes the issue for me. I already tested it on my actual system (not just VirtualBox).

Now to your comments:

1. shim is not dependency of grub2. My 18.04, 18.10 and 19.04 installations had only these packages and no shim:

grub-common, grub2-common, grub-efi-amd64, grub-efi-amd64-bin

Shim is not required on non-SB systems. Ubuntu supported direct usage of grub without shim since forever and I see no reason to drop such support. If you're sure shim is required then make it dependency of the grub.

I didn't use boot-repair. I simply updated from 19.04 (without shim) to 19.10 (still without shim) which broke chainload.

2. You're right. While I don't exactly understand what grub_loader_set() does:

grub_loader_set (grub_secureboot_chainloader_boot, grub_secureboot_chainloader_unload, 0);

but I took a look at grub_secureboot_chainloader_boot() and it indeed gets 0 from handle_image()/read_header() and expected to call grub_load_and_start_image(). However on my system the latest log record comes from:

grub_dprintf ("chain", "Secure Boot is not enabled\n");

and then I guess from grub_load_and_start_image() it immediately goes back to the menu.

This is the last page of logs with "debug=all" – https://i.imgur.com/PFRSqg8.png

This screenshot has been taken using the latest grub2 from eoan without any custom patches. If you want to see full video of boot process send me a message using https://launchpad.net/~xeron-oskom/+contactuser.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

Given that my patch fixes the issue for me means:

This works:

      grub_load_and_start_image(boot_image);
      grub_file_close (file);
      grub_loader_set (grub_chainloader_boot, grub_chainloader_unload, 0);

      return 0;

This doesn't:

      grub_file_close (file);
      grub_loader_set (grub_secureboot_chainloader_boot,
         grub_secureboot_chainloader_unload, 0);
      return 0;

I'm not that familiar with C but may be it has something to do with calling grub_file_close() before grub_loader_set()? I mean, hd0 is closed, will it work to call grub_load_and_start_image() later?

Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

It has nothing to do with whether the file handle is closed or not; this is just confusing code because of how it's built.

Shim is supposed to be installed by the installer when you set up the system in UEFI mode; this is done for you by the installer, and isn't a dependency of grub because grub does not require shim to work; but things *will* work better with shim installed. So, I strongly recommend anyone running into such issues make sure it is installed on their system, as it is supposed to be.

Now, after tracing more of the code, it looks like the issue is that grub_load_and_start_image() is insufficient to complete the boot process in handle_image()... You get the image loaded, but StartImage is never called, which would explain why you're seeing things go back to the menu.

Now, I don't think it's worth digging more into this code path, since if SB is disabled then shim_verify should be letting images work, unvalidated but accepted; so it would otherwise not get run. I'll go ahead and drop the extra check for SB state in linuxefi_secure_validate().

description: updated
Changed in grub2 (Ubuntu):
status: Incomplete → In Progress
Changed in grub2 (Ubuntu Eoan):
status: Confirmed → Triaged
Revision history for this message
Dmitry-a-durnev (dmitry-a-durnev) wrote :

For me installing shim fixed nothing: still the same issue - return to grub menu when trying to boot(chain load) windows. Maybe I also have to re-install grub after shim is installed?

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package grub2 - 2.04-1ubuntu13

---------------
grub2 (2.04-1ubuntu13) focal; urgency=medium

  * debian/patches/ubuntu-tpm-unknown-error-non-fatal.patch: treat "unknown"
    TPM errors as non-fatal, but still write up the details as debug messages
    so we can further track what happens with the systems throwing those up.
    (LP: #1848892)
  * debian/patches/ubuntu-linuxefi.patch: Drop extra check for Secure Boot
    status in linuxefi_secure_validate(); it's unnecessary and blocking boot
    in chainload (like chainloading Windows) when SB is disabled.
    (LP: #1845289)

 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 31 Oct 2019 17:58:47 -0400

Changed in grub2 (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Jay Tuckey (jay-tuckey) wrote :

Is there any way to test 2.04-1ubuntu13 from focal on 19.10?

Revision history for this message
Dmitry-a-durnev (dmitry-a-durnev) wrote :

Update: after some reboots (PC on/off, etc) and win 10 update (insider preview last update), now bootmgfw.efi boots from grub(2.04-1ubuntu12) successfully. Shim is currently installed.

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Joseph, or anyone else affected,

Accepted grub2 into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2/2.04-1ubuntu12.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in grub2 (Ubuntu Eoan):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-eoan
Revision history for this message
Adam Conrad (adconrad) wrote :

Hello Joseph, or anyone else affected,

Accepted grub2-signed into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/grub2-signed/1.128.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Joseph Maillardet (jokx) wrote :

Hello Adam,

My test machine has been migrated to Focal. I don't know if this can help, however, the bug has disappeared with the Focal versions of grub/shim.

Ask me for command outputs if necessary.

Revision history for this message
Jay Tuckey (jay-tuckey) wrote :

I have enabled the eoan-proposed apt repo, and installed the latest grub2 packages from there.

After a reboot, booting windows works nicely! :D

Thanks heaps, @adconrad and all the others who have helped diagnose and fix this issue.

Let me know if you want me to run any further testing or get any info from my PC.

Revision history for this message
Ivan Larionov (xeron-oskom) wrote :

I can confirm grub2 from eoan-proposed fixes the issue and I'm able to boot both Windows and Linux on a system without SB and without shim.

Revision history for this message
ianfas (ianfas) wrote :

#40 works, would be nice if you release this!

Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

There are multiple reports of chainload working again with the fix uploaded to eoan-proposed; marking as verification-done for Eoan with 2.04-1ubuntu12.1

tags: added: id-5d978799ba24715c5ddcabc verification-done-eoan
removed: id-5d978799ba24715c5ddcabc5 verification-needed verification-needed-eoan
tags: added: id-5d978799ba24715c5ddcabc5
removed: id-5d978799ba24715c5ddcabc
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package grub2 - 2.04-1ubuntu12.1

---------------
grub2 (2.04-1ubuntu12.1) eoan; urgency=medium

  * debian/patches/ubuntu-tpm-unknown-error-non-fatal.patch: treat "unknown"
    TPM errors as non-fatal, but still write up the details as debug messages
    so we can further track what happens with the systems throwing those up.
    (LP: #1848892)
  * debian/patches/ubuntu-linuxefi.patch: Drop extra check for Secure Boot
    status in linuxefi_secure_validate(); it's unnecessary and blocking boot
    in chainload (like chainloading Windows) when SB is disabled.
    (LP: #1845289)

 -- Mathieu Trudel-Lapierre <email address hidden> Fri, 01 Nov 2019 15:16:43 -0400

Changed in grub2 (Ubuntu Eoan):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for grub2 has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
dimes (john-fallenbit) wrote :

I seem to have this exact issue:

   "Initial install 19.04, dual boot with Win10, everything fine; Upgrade to 19.10, Ubuntu boots fine, Win10 does not"

On a fully up to date 19.10 install, however, I seem to have one difference:

When I select in the grub menu to boot windows it throws an error similar to:

   "Alloc magic is broken at XXXXXX Press any key"

It seems like the same issue, but from what I can tell, I have the updated grub2 packages and still have this issue. Am I barking up the wrong tree?

grub-common 2.04-1ubuntu12.1
grub-efi-amd64 2.04-1ubuntu12.1
grub-efi-amd64-bin 2.04-1ubuntu12.1
grub-efi-amd64-signed 1.128.1+2.04-1ubuntu12.1
grub2-common 2.04-1ubuntu12.1

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.