#2116 Issue `closed`: Leap15.0/UEFI fails to boot post-recovery for not calling shim-install¶

Labels: enhancement, bug, fixed / solved / done

petroniusniger opened issue at 2019-04-11 16:05:¶

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.4 / 2018-06-21
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):

NAME="openSUSE Leap"
VERSION="15.0"
ID="opensuse-leap"
ID_LIKE="suse opensuse"
VERSION_ID="15.0"
PRETTY_NAME="openSUSE Leap 15.0"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:15.0"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL="nfs://XXXXXXX.XXXXX.XXX/Stations_bkup/rear/"
### Include all kernel modules
MODULES=( 'all_modules' )

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
Virtual machine (VMware 6.5)
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86_64
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
UEFI (secure boot NOT enabled) + GRUB2-EFI
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
Local virtual disk.
Description of the issue (ideally so that others can reproduce it):
"rear recovery" ran fine, all the partitions, logical volumes and filesystems were recreated. Then, 630_run_efibootmgr.sh was called (successfully). But once the recovery process was over, the virtual machine failed to boot (got stuck at GRUB) with the following error message:

error: disk 'lvmid/CLwYPh-8mlg-4knU-TdNb-DJse-YmZV-sQiZWg/3vjNwK-yySJ-irSt-VY0p-v2YQ-NGw4-ktRQ7f' not found

After investigation, it turned out that 2 (identical) EFI files on the system still contained the UUID for the disks of the old system:

./boot/grub2/x86_64-efi/core.efi
./boot/efi/EFI/opensuse/grubx64.efi

To recreate (or adapt) these files, you should call a script called "shim-install" (that takes your grub.cfg as input).

Workaround, if any:
I've created a new SUSE-specific script (finalize/SUSE_LINUX/i386/640_install_shims.sh) to call shim-install (inside a chroot on the recovered system). With it, the recovered system boots fine. I'll do a pull request to contribute it.

jsmeix commented at 2019-04-12 07:28:¶

@petroniusniger
thank you for your explanatory description of the problem and
in particular thank you for your analysis what the root cause is.

I also experienced booting issues on a SLES12-SP4 system with UEFI
(a KVM/QEMU virtual machine with OVMF/TianoCore UEFI firmware for QEMU)
where rebooting a recreated system failed.
But I could always work around it by going into the OVMF/TianoCore firmware
boot menue and there directly selecting my harddisk to boot from
(instead of the default entry wherefrom booting failed).

I look forward to your pull request!

jsmeix commented at 2019-04-12 07:33:¶

@gozora
because I know not much about UEFI and even less about secure boot
I would appreciate it if you could have a look here.

Perhaps calling shim-install (in chroot on TARGET_FS_ROOT)
is not only needed on (open)SUSE systems but in general
also on other Linux distributions?

jsmeix commented at 2019-04-12 07:39:¶

@gdha
I dared to add this to the ReaR v2.5 milestone
because the issue looks severe enough to be
really useful when it is fixed in the next ReaR release.

But if you object I won't mind to postpone it to ReaR 2.6
because we do not have any other issue reports except this one,
in particular nothing about SLES12 or SLES15 up to now.
I guess everybody tries to avoid secure boot ( like me ;-)

jsmeix commented at 2019-04-12 07:48:¶

FYI regarding shim-install:

On my openSUSE Leap 15.0 system I have

# type -a shim-install 
shim-install is /usr/sbin/shim-install

# rpm -qf /usr/sbin/shim-install
shim-14-lp150.8.5.1.x86_64

# man shim-install
No manual entry for shim-install

# /usr/sbin/shim-install --help
Usage: shim-install [OPTION] [INSTALL_DEVICE]
Install Secure Boot Loaders on your drive.
--directory=DIR use images from DIR.
--grub-probe=FILE use FILE as grub-probe.
--removable the installation device is removable.
--no-nvram don't update the NVRAM variable.
--bootloader-id=ID the ID of bootloader.
--efi-directory=DIR use DIR as the EFI System Partition root.
--config-file=FILE use FILE as config file, default is /boot/grub2/grub.cfg.
--clean remove all installed files and configs.
--suse-enable-tpm install grub.efi with TPM support.
INSTALL_DEVICE must be system device filename.

and how it is called during installation by YaST
(long line shown wrapped here):

# grep shim-install /var/log/YaST2/*
/var/log/YaST2/y2log-1:
 2018-09-13 09:52:55 <1> linux-4la7(3837) [Ruby] lib/cheetah.rb:158
 Executing "/usr/sbin/shim-install --config-file\=/boot/grub2/grub.cfg".

On my SLES12-SP4 system it is the same (except the shim RPM version):

# type -a shim-install 
shim-install is /usr/sbin/shim-install

# rpm -qf /usr/sbin/shim-install
shim-0.9-23.14.x86_64

# man shim-install
No manual entry for shim-install

# shim-install --help
Usage: shim-install [OPTION] [INSTALL_DEVICE]
Install Secure Boot Loaders on your drive.
--directory=DIR use images from DIR.
--grub-probe=FILE use FILE as grub-probe.
--removable the installation device is removable.
--no-nvram don't update the NVRAM variable.
--bootloader-id=ID the ID of bootloader.
--efi-directory=DIR use DIR as the EFI System Partition root.
--config-file=FILE use FILE as config file, default is /boot/grub2/grub.cfg.
--clean remove all installed files and configs.
--suse-enable-tpm install grub.efi with TPM support.
INSTALL_DEVICE must be system device filename.

and how it is called during installation by YaST
(long line shown wrapped here):

# grep shim-install /var/log/YaST2/*
/var/log/YaST2/y2log-1:
 2019-02-21 15:20:00 <1> linux-cvfz(3349) [Ruby] lib/cheetah.rb:158
 Executing "/usr/sbin/shim-install --config-file\=/boot/grub2/grub.cfg".

jsmeix commented at 2019-04-12 10:06:¶

My main concern is how to determine whether or not it is needed
to call shim-install (in chroot on TARGET_FS_ROOT).

In other words:
My main concern is how to determine whether or not
secure boot was actually used on the original system?

I fear when we call shim-install in the recreated system
but secure boot was not used on the original system
we may introduce secure boot on the recreated system.

We have various variables that deal with UEFI in default.conf:

BOOTLOADER
USING_UEFI_BOOTLOADER
UEFI_BOOTLOADER
SECURE_BOOT_BOOTLOADER
EFI_STUB
OUTPUT_EFISTUB_SYSTEMD_BOOTLOADER
EFI_STUB_EFIBOOTMGR_ARGS

but it is not clear to me how each one actually is meant to be used, cf.
https://github.com/rear/rear/issues/1942#issuecomment-438220832
and
https://github.com/rear/rear/issues/1942#issuecomment-438215291

On first glance SECURE_BOOT_BOOTLOADER looks right
to determine whether or not it is needed to call shim-install
but according to
https://github.com/rear/rear/issues/1942#issuecomment-438220832
SECURE_BOOT_BOOTLOADER currently belongs only
to the ReaR recovery system - or I misunderstand something...

petroniusniger commented at 2019-04-12 11:29:¶

Quick reply to some of the points raised by @jsmeix above:

in the case of my test system, secure boot was not used before, and I have no reason to believe that calling "shim-install" has enabled it
I learned about the need to call "shim-install" by reading the YaST2 logs after I used it to re-install the bootloader "manually" in order to recover my then unbootable VM
I've not made any test (yet) with secure boot enabled

jsmeix commented at 2019-04-12 12:37:¶

@petroniusniger
no need to worry about details right now - just submit a pull request
so that we can have a first look at what helped in your case.

As said in my
https://github.com/rear/rear/issues/2116#issuecomment-482470708
I don't know much about UEFI and even less about secure boot
so I cannot tell if shim-install is perhaps even always needed
on (open)SUSE systems with UEFI firmware.
I only know from shim-install --help that its usage is to

Install Secure Boot Loaders on your drive

so that I concluded it is only needed when secure boot is used.
Via Googling I found up to now
https://en.opensuse.org/openSUSE:UEFI
that mentiones shim only in relation to secure boot and also in
https://doc.opensuse.org/documentation/leap/reference/html/book.opensuse.reference/cha.grub2.html

If you are using UEFI Secure Boot and your system
is not reaching GRUB 2 correctly anymore, you may need
to additionally reinstall Shim and regenerate the UEFI boot table.
To do so, use:
  shim-install --config-file=/boot/grub2/grub.cfg

indicates that shim-install is only needed when secure boot is used.

petroniusniger commented at 2019-04-12 16:08:¶

Pull request submitted: https://github.com/petroniusniger/rear/pull/new/issue-2116

jsmeix commented at 2019-04-15 09:38:¶

In contrast to what I wrote above in
https://github.com/rear/rear/issues/2116#issuecomment-482472370
I changed it now to the ReaR v2.6 milestone because
I need more time to sufficiently understand what goes on behind.

jsmeix commented at 2019-05-24 11:25:¶

With https://github.com/rear/rear/pull/2117 merged
this issue should be fixed.

[Export of Github issue for rear/rear.]

#2116 Issue closed: Leap15.0/UEFI fails to boot post-recovery for not calling shim-install¶