#2883 Issue closed: bootloader grub for output usb is not working for some uefi cases

Labels: enhancement, fixed / solved / done

DEvil0000 opened issue at 2022-10-26 14:32:

  • ReaR version ("/usr/sbin/rear -V"):
    2.7 and also master (f21a6d6940896c1fbdd51721560737df982b1a8d)

  • OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
    ubuntu 20.04

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

BACKUP=BORG
BORGBACKUP_ENC_TYPE="repokey"
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export TMPDIR="/wsp_var/tmp/"
USING_UEFI_BOOTLOADER=1
USB_BOOTLOADER=grub
OUTPUT=USB
OUTPUT_URL=usb:///dev/disk/by-label/REAR-EFI
USB_DEVICE=/dev/disk/by-label/REAR-000
USB_DEVICE_FILESYSTEM_LABEL=REAR-000
USB_DEVICE_PARTED_LABEL=gpt
USB_DEVICE_FILESYSTEM=ext4
USB_UEFI_PART_SIZE="2048"
BORGBACKUP_REPO="/borg"
BORGBACKUP_UMASK="0002"

  • Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
    HP gen9 380

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
    amd64

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
    grub using uefi

  • Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):

/dev/sda                       8:0    0   10G  0 disk 
|-/dev/sda1                    8:1    0  538M  0 part /boot/efi
`-/dev/sda2                    8:2    0  9.5G  0 part /boot
/dev/sdb                       8:16   0  1.6T  0 disk 
`-/dev/sdb1                    8:17   0  1.6T  0 part 
  |-/dev/mapper/system   253:4    0   46G  0 lvm  /
/dev/sdc                       8:32   1 57.3G  0 disk
  • Description of the issue (ideally so that others can reproduce it):
    rear format -- --efi # (same for all variations of rear format on a system booted with efi)
    rear mkrescue fails due to grub-install failing.
    the executed grub-install command from /usr/share/rear/output/USB/Linux-i386/300_create_grub.sh is:
    grub-install --boot-directory=/test --recheck /dev/sdc
    the error from grub-install is:
  Installing for x86_64-efi platform.
  grub-install: error: cannot find EFI directory.

  • first analysis:
    this is due to actually multiple issues with the grub install.

  • target not set explicitly..
    the issue is that grub-install defaults to --target x86_64-efi if the system is booted with efi. So you will not be able to install a legacy-bios (or 32bit) grub without setting the --target parameter. the documentation tells us default=i386-pc which is not exactly correct - this is more like a fallback default. So setting the target in the command should fix this one.

  • in case of a hybrid bootable device (uefi and legacy-bios)
    a) both grub packages (grub-efi-amd64-bin and grub-pc-bin) need to be installed. there is no check for that
    b) grub-install must be called for each case individually with correct parameters

-3) for a efi install depending on your grub version it may be neccecary to set more parameters-
-a) cannot find EFI directory. so the --efi-directory may need to get set similar to the --boot-directory-
-b) the efi partition needs to be mounted at the --efi-directory path wich may need to be efi or EFI directory within the boot path.-

  • What is actually working:
    looks like grub in output usb mode is only working on a legacy-boot system and only for --bios.
    looks like the newer grub version is more strict about those things and the parameters.

  • Workaround, if any:
    do things manually or use another bootloader

DEvil0000 commented at 2022-10-26 14:34:

since this is quite its own chapter i did not mix it with the older ticket #2648

DEvil0000 commented at 2022-11-03 11:14:

to my current understanding 100_create_efiboot.sh handles uefi installations no matter if grub or non grub. it handles grub in a bit strange manual way but should be fine. while the file 300_create_grub.sh only handles grub legacy boot - which may be used in addition to uefi grub but this is not always the case.
so point 3 of my first analysis is not correct since this is handled differently. and point 2 is only partially true due to this file split

DEvil0000 commented at 2022-11-03 12:07:

this patch solves the main issue for me. creating a usb backup device on a system booted with efi and format options (both, efi and bios). This however does not fix the efi only case - see todo 1 and is not well suted for all target variants see todo 2.

there are however two TODOs left for which I am not sure what would be the correct way to implement those in rear.
open todos:

  • only call grub-install if legacy boot install is requested. maybe introduce something like USING_BIOS_BOOTLOADER or provide the format variable.
  • set the target as correct as possible. maybe use a switch case based on the target and possibly other info? or make this a config option?
diff --git a/usr/share/rear/output/USB/Linux-i386/300_create_grub.sh b/usr/share/rear/output/USB/Linux-i386/300_create_grub.sh
index 9b6e3dc6..aba29d62 100644
--- a/usr/share/rear/output/USB/Linux-i386/300_create_grub.sh
+++ b/usr/share/rear/output/USB/Linux-i386/300_create_grub.sh
@@ -51,7 +51,20 @@ if [ ! -d "$usb_boot_dir" ] ; then
     mkdir -p $v "$usb_boot_dir" || Error "Failed to create USB boot dir '$usb_boot_dir'"
 fi
 DebugPrint "Installing GRUB2 as USB bootloader on $RAW_USB_DEVICE"
-$grub_install_binary --boot-directory=$usb_boot_dir --recheck $RAW_USB_DEVICE || Error "Failed to install GRUB2 on $RAW_USB_DEVICE"
+
+#grub-install
+#--target available targets: arm-coreboot, arm-efi, arm-uboot, arm64-efi, i386-coreboot, i386-efi, i386-ieee1275, i386-multiboot, i386-pc,
+#            i386-qemu, i386-xen, i386-xen_pvh, ia64-efi, mips-arc, mips-qemu_mips, mipsel-arc, mipsel-loongson, mipsel-qemu_mips, powerpc-ieee1275, riscv32-efi,  riscv64-efi,  sparc64-ieee1275,
+#            x86_64-efi, x86_64-xen
+if is_true $USING_UEFI_BOOTLOADER ; then
+    #TODO only call grub-install if legacy boot install is requested
+    #TODO use a switch case based on the target and possibly other info? or make this a config option?
+    #TARGET=`uname -m`
+    # force to legacy BIOS installation since efi was handled in 100_create_efiboot.sh
+    $grub_install_binary --target=i386-pc --boot-directory=$usb_boot_dir --recheck $RAW_USB_DEVICE || Error "Failed to install GRUB2 on $RAW_USB_DEVICE"
+else
+    $grub_install_binary --boot-directory=$usb_boot_dir --recheck $RAW_USB_DEVICE || Error "Failed to install GRUB2 on $RAW_USB_DEVICE"
+fi
 # grub[2]-install creates the $BUILD_DIR/outputfs/boot/grub[2] sub-directory that is needed
 # to create the GRUB2 config $BUILD_DIR/outputfs/boot/grub[2].cfg in the next step:
 DebugPrint "Creating GRUB2 config for legacy BIOS boot as USB bootloader"

jsmeix commented at 2022-11-03 12:10:

@DEvil0000
many thanks for your continuous analysis of
ReaR recovery system UEFI booting issues!

For me with my single simple use case test
BIOS and UEFI (without Secure Boot) booting
the ReaR recovery system on a USB disk worked
with openSUSE Leap 15.3, see
https://github.com/rear/rear/pull/2829
and it also worked with RHEL 8, see
https://github.com/rear/rear/pull/2829#issuecomment-1165570472

Next week I will not have time for ReaR
but the week after the next week I have (hopefully)
time for ReaR to have a closer look at this issue.

But I am not a Ubuntu user so when things are
different on Ubuntu there is not much what I can do
in practice (I won't find time to install Ubuntu
and try out how things behave there).

jsmeix commented at 2022-11-03 12:12:

@DEvil0000
WOW!
Thank you for your proposed patch!
It helps so much to see where things go wrong and how.

DEvil0000 commented at 2022-11-03 12:18:

This issue may be introduced by a newer grub version used in ubuntu or some other ubuntu specific thing. Looks like at least for some cases setting the target (and efi_directory in case efi install would be done with the command - which is not currently) is a must now.
Thank you for your work on this project ;)

github-actions commented at 2023-01-03 02:25:

Stale issue message

jsmeix commented at 2023-01-11 14:30:

With https://github.com/rear/rear/pull/2905 merged
this issue should be fixed at least for now, see
https://github.com/rear/rear/pull/2905#issuecomment-1377282602

[Export of Github issue for rear/rear.]