#1942 Issue closed: Support EFISTUB booting (e.g. on Arch Linux)

Labels: enhancement, cleanup, fixed / solved / done

johnny-cash opened issue at 2018-10-27 15:39:

rear/usr/share/rear/finalize/Linux-i386/630_run_efibootmgr.sh

This script fails because arch linux loads uefi from /boot and not from /boot/efi

# check if $TARGET_FS_ROOT/boot/efi is mounted
[[ -d "$TARGET_FS_ROOT/boot/efi" ]]
StopIfError "Could not find directory $TARGET_FS_ROOT/boot/efi"

BootEfiDev="$( mount | grep "boot/efi" | awk '{print $1}' )"
Dev=$( get_device_name $BootEfiDev )    # /dev/sda1 or /dev/mapper/vol34_part2 or /dev/mapper/mpath99p4
ParNr=$( get_partition_number $Dev )  # 1 (must anyway be a low nr <9)
Disk=$( echo ${Dev%$ParNr} ) # /dev/sda or /dev/mapper/vol34_part or /dev/mapper/mpath99p

jsmeix commented at 2018-10-29 09:33:

@gozora
could you have a look here how to enhance it because
it seems Arch Linux uses /boot and not /boot/efi

gozora commented at 2018-10-29 09:50:

@jsmeix I'll take a look as a time permits ...

@johnny-cash there is several ways ho Arch can deal with UEFI, can you please describe yours in more detail ?
It would also help if you'd not ignore our issue template ...

V.

jsmeix commented at 2018-10-29 16:46:

@gdha
I also assigned you here because

git log -p --follow usr/share/rear/finalize/Linux-i386/630_run_efibootmgr.sh

shows that 630_run_efibootmgr.sh was intitially from you
and
I assigned me because I think 630_run_efibootmgr.sh needs to be
generally overhauled like I did it for 620_install_grub2.sh
in particular I think 'finalize' stage scripts should usually not error out,
see my comment in 620_install_grub2.sh

jsmeix commented at 2018-10-29 16:52:

At a very first glance I think I can use the UEFI_BOOTLOADER value
to get the right directory where the ESP is mounted
cf. the code in rescue/default/850_save_sysfs_uefi_vars.sh
... more tomorrow ...

jsmeix commented at 2018-10-29 17:00:

@johnny-cash
in your ReaR log file (and on your terminal when you run ReaR in verbose mode)
you got a message like

Using '/boot/loader.efi' as UEFI bootloader file

What exactly did you get?

Best provide us a whole rear debug log file,
see what we need at
https://github.com/rear/rear/blob/master/.github/ISSUE_TEMPLATE.md
and the section
"Debugging issues with Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery

gozora commented at 2018-10-29 17:20:

I currently see two problems here:

  1. Arch gives you freedom during installation to set up your UEFI partition (esp) wherever you like. As an example I've managed to setup my Arch as follows:
/dev/sda2 on / type ext4 (rw,relatime)
/dev/sda1 on /boot/hello type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)

Hence heaving esp in /boot/hello ...

This is good for Arch users but can quite easily become nightmare for us, since we can't rely on "standard" filesystem layout when determining which EFI files to copy.

  1. You can setup Arch not to use GRUB2 (or any other boot loader) and nicely tell UEFI "Hey UEFI, don't bother with grubx64.efi and friends but rather boot vmliuz directly!". (a.k.a. EFISTUB) Such setup which is recommended directly by Arch wiki, and which I guess is reason of @johnny-cash trouble, will most probably avoid ReaR to create recovery system as you are not bound to have any boot loader helper utilities (GRUB, elilo, ...) installed.
    And even if you'd install them, and you'd convince ReaR to ignore all the hard coded boot/efi references, ReaR would incorrectly install GRUB on finalize stage, instead using EFISTUB ...

In upcoming days (if time permits) I'll try to find some reasonable way to implement EFISTUB into ReaR.

V.

jsmeix commented at 2018-10-30 09:25:

@gozora
give me a bit of time - I would like to do a pull request soon - then you can
have a look if you like it...

gozora commented at 2018-10-30 10:02:

@jsmeix you have all the time of the world, I'm currently exploring new feeling of booting without Grub ;-) (strange feeling BTW), but not actually writing any ReaR code ...

V.

gozora commented at 2018-10-30 10:10:

@rear/contributors I'm doing small research.
Can you please look for string "EFI" in kernels of your testing systems and paste results to this thread together with kernel version and distro? It doesn't really matter if your OS is setup for LEGACY or UEFI boot ...

Something like:

arch-efi:(/root)(root)# cat /etc/os-release; uname -r && strings /boot/vmlinuz-linux | grep EFI
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
ID_LIKE=archlinux
ANSI_COLOR="0;36"
HOME_URL="https://www.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
4.18.16-arch1-1-ARCH
EFI stub: ERROR: Failed to alloc mem for file handle list
EFI stub: ERROR: Failed to alloc highmem for files
EFI stub: ERROR: We've run out of free low memory
EFI stub: ERROR: Failed to read file
EFI stub: ERROR: Failed to allocate usable memory for kernel.
EFI stub: UEFI Secure Boot is enabled.
EFI stub: ERROR: Could not determine UEFI Secure Boot status.

Thanks

V.

jsmeix commented at 2018-10-30 10:12:

@gozora @gdha
have a look at my blind attempt to clean up 630_run_efibootmgr.sh
('blind' because I did not test anything I only looked at the code)
in https://github.com/rear/rear/pull/1945

gdha commented at 2018-10-30 10:18:

My laptop is capable of UEFI booting, but I turned it off.

$ sudo cat /etc/os-release; uname -r && strings /boot/vmlinuz-3.10.0-862.14.4.el7.x86_64 | grep EFI
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

3.10.0-862.14.4.el7.x86_64
EFI stub: ERROR: Failed to alloc mem for initrds
EFI stub: ERROR: Failed to alloc highmem for initrds
EFI stub: ERROR: We've run out of free low memory
EFI stub: ERROR: Failed to read initrd
EFI stub: ERROR: Failed to alloc mem for kernel

jsmeix commented at 2018-10-30 10:20:

On my worksation where I use UEFI:

# cat /etc/os-release
NAME="openSUSE Leap"
VERSION="15.0"
ID="opensuse-leap"
ID_LIKE="suse opensuse"
VERSION_ID="15.0"
PRETTY_NAME="openSUSE Leap 15.0"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:15.0"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"

# ls -lhtr /boot/vmlinu*
-rw-r--r-- 1 root root 7.7M Oct  5 10:22 /boot/vmlinux-4.12.14-lp150.12.19-default.gz
-rw-r--r-- 1 root root 6.8M Oct  5 10:47 /boot/vmlinuz-4.12.14-lp150.12.19-default
-rw-r--r-- 1 root root 7.7M Oct 13 16:55 /boot/vmlinux-4.12.14-lp150.12.22-default.gz
-rw-r--r-- 1 root root 6.8M Oct 13 17:24 /boot/vmlinuz-4.12.14-lp150.12.22-default
lrwxrwxrwx 1 root root   35 Oct 17 08:47 /boot/vmlinuz -> vmlinuz-4.12.14-lp150.12.22-default

# uname -r
4.12.14-lp150.12.22-default

# strings /boot/vmlinuz-4.12.14-lp150.12.22-default | grep EFI
EFI_SUCCESS
EFI_LOAD_ERROR
EFI_INVALID_PARAMETER
EFI_UNSUPPORTED
EFI_BAD_BUFFER_SIZE
EFI_BUFFER_TOO_SMALL
EFI_NOT_READY
EFI_DEVICE_ERROR
EFI_WRITE_PROTECTED
EFI_OUT_OF_RESOURCES
EFI_NOT_FOUND
EFI_ABORTED
EFI_SECURITY_VIOLATION
EFI stub: ERROR: Failed to alloc mem for file handle list
EFI stub: ERROR: Failed to alloc highmem for files
EFI stub: ERROR: We've run out of free low memory
EFI stub: ERROR: Failed to read file
EFI stub: ERROR: Failed to allocate usable memory for kernel.
EFI stub: UEFI Secure Boot is enabled.
EFI stub: ERROR: Could not determine UEFI Secure Boot status.

On my old SLES11 32-bit system with BIOS:

 # cat /etc/os-release 
NAME="SLED"
VERSION="11.4"
VERSION_ID="11.4"
PRETTY_NAME="SUSE Linux Enterprise Desktop 11 SP4"
ID="sled"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sled:11:4"

# ls -lhtr /boot/vmlinuz*
-rw-r--r-- 1 root root 3.8M Dec 31  2017 /boot/vmlinuz-3.0.101-108.21-pae
lrwxrwxrwx 1 root root   26 Jan 15  2018 /boot/vmlinuz -> vmlinuz-3.0.101-108.21-pae

# uname -r
3.0.101-108.21-pae

# strings /boot/vmlinuz-3.0.101-108.21-pae | grep EFI
[no output]

gozora commented at 2018-10-30 14:48:

To my big surprise I just successfully recovered Arch Linux with EFISTUB without changing a bit in actual ReaR code.(OK, OK, I've used @jsmeix https://github.com/rear/rear/pull/1945, BTW I'd like to once write such code unblinded ;-)).
When using EFISTUB, ReaR incorrectly sets USING_UEFI_BOOTLOADER=0, by :

+ source /usr/share/rear/prep/default/320_include_uefi_env.sh
...
+++ find /boot -maxdepth 1 -iname efi -type d
++ test ''
++ return 0

and continues ... As a side effect of such wrong evaluation, ReaR recovery ISO can't automatically boot, however recovery system can be booted manually using UEFI shell (with EFISTUB we have kernel that can be booted directly without helper tools or boot loaders).
After rear recover finishes, I've got

For this system
Arch/ on Linux-i386 (based on /i386)
there is no code to install a boot loader on the recovered system
or the code that we have failed to install the boot loader correctly.
Please contribute appropriate code to the Relax-and-Recover project,
see http://relax-and-recover.org/development/
Take a look at the scripts in /usr/share/rear/finalize,
for example see the scripts
/usr/share/rear/finalize/Linux-i386/210_install_grub.sh
/usr/share/rear/finalize/Linux-i386/220_install_grub2.sh

---------------------------------------------------
|  IF YOU DO NOT INSTALL A BOOT LOADER MANUALLY,  |
|  THEN YOUR SYSTEM WILL NOT BE ABLE TO BOOT.     |
---------------------------------------------------

You can use 'chroot /mnt/local bash --login'
to change into the recovered system.
You should at least mount /proc in the recovered system
e.g. via 'mount -t proc none /mnt/local/proc'
before you change into the recovered system
and manually install a boot loader therein.

Finished recovering your system. You can explore it under '/mnt/local'.
Exiting rear recover (PID 434) and its descendant processes
Running exit tasks

message, but again, since system boots with EFISTUB manual boot in EFI shell did the trick, and system is up and running ...

When I've did another test with USING_UEFI_BOOTLOADER=y enforced in /etc/rear/local.conf,
rear mkrescue failed with:

ERROR: Cannot autodetect what to use as UEFI_BOOTLOADER, you have to manually specify it in /etc/rear/local.conf
Some latest log messages since the last called script 850_save_sysfs_uefi_vars.sh:
  2018-10-30 15:32:24.854058742 Including rescue/default/850_save_sysfs_uefi_vars.sh
  2018-10-30 15:32:24.855709222 Trying to find what to use as UEFI bootloader...
  2018-10-30 15:32:24.871059830 Trying to find a 'well known file' to be used as UEFI bootloader...
  find: '/boot/efi': No such file or directory
  find: '/boot/efi': No such file or directory
  find: '/boot/EFI': No such file or directory
  2018-10-30 15:32:24.889992856 Trying to autodetect from EFI variables what to use as UEFI bootloader file...
  grep: Trailing backslash
Aborting due to an error, check /var/log/rear/rear-arch-efi.log for details
Terminated

The problem here simply is that, there is nothing that could be used as UEFI_BOOTLOADER, bacause EFISTUB ...

Now I'm not sure if I want to fix something that is actually working :-)

V.

gozora commented at 2018-10-31 09:56:

I'll give systemd-boot a try to implement EFISTUB ...

V.

jsmeix commented at 2018-10-31 10:46:

@gozora
I think it should be possible (with reasonable effort) to enhance
the existing USING_UEFI_BOOTLOADER and UEFI_BOOTLOADER
config variables to also support EFISTUB cleanly in the code.
Not necessarily automatically out of the box but at least manually
e.g. via something like

USING_UEFI_BOOTLOADER=1
UEFI_BOOTLOADER="EFISTUB"

plus additional conditions in the code that "officially skip" what is not needed
in case of EFISTUB (and perhaps even set NOBOOTLOADER='' to avoid
the non helpful warning that one needs to manually install a boot loader)
instead of now where the code blindly and accidentally lets things
somehow work even for EFISTUB.

This way EFISTUB could be generically supported by ReaR
without a requirement of systemd-boot - except in practice
systemd-boot is always used on systems that boot via EFISTUB.

gozora commented at 2018-10-31 11:15:

@jsmeix
In general I agree.

The reason for using systemd-boot as boot loader, is plain and simple automatic start of boot menu of ReaR recovery system. Without systemd-boot we would need to somehow tell server what kernel + initrd + options to boot. This means that we would either need to somehow mess up with efibootmgr or instruct user around EFI shell, maybe by documenting it (and I don't like any of these options).

From what I've checked so far, systemd-bootx64.efi is standard part of Systemd hence available on Linux distributions using Systemd. So we would not need to install any additional software.

These is my very early idea (which might change over time) how thing could work. I'd need to continue with research on this topic. I keep you posted.

V.

jsmeix commented at 2018-10-31 16:08:

Thanks to mention that 'systemd-boot' actually is systemd-bootx64.efi

https://www.freedesktop.org/wiki/Software/systemd/systemd-boot/
does not tell that.

On my SLES15-like openSUSE Leap 15.0 system I have it

# rpm -qf /usr/lib/systemd/boot/efi/systemd-bootx64.efi
systemd-234-lp150.20.6.1.x86_64

and on my SLES12 system it also exists

# rpm -qf /usr/lib/systemd/boot/efi/systemd-bootx64.efi
systemd-228-150.46.1.x86_64

For the fun of it:
When you "don't like non of these options" you like them all ;-)

gozora commented at 2018-10-31 17:20:

@jsmeix
Sorry for my previous inconsistencies. I'm pretty new in this EFISTUB thingy and I'm still exploring its possibilities and ways how we could use them in ReaR.
The fact that it looks to be widely available across distros, together with fact that kernels are nowadays build with EFISTUB support, could simplify ReaR booting code a lot in the future (at least for systems booting with UEFI).

For the fun of it:
When you "don't like non of these options" you like them all ;-)

Comment updated, thanks for pointing it out ;-).

V.

jsmeix commented at 2018-11-05 12:11:

@gozora
I look forward to your first EFISTUB support in ReaR!

Booting the kernel directly by the UEFI firmware without
a traditional bootloader in between looks right to me
in particular because it avoids RFC 1925 item 6a:
"It is always possible to add another level of indirection."

But booting via EFISTUB could only simplify the ReaR booloader
setup code provided the original system is booted this way.

Because ReaR intends to recreate a system as far as possible as it was before
"rear recover" would have to recreate a system that boots via UEFI
with GRUB2 in between when the original system booted this way.

I assume there are use cases where GRUB2 in between is useful
even on UEFI hardware and EFISTUB support in the kernel.
I guess GRUB2 in between provides features that plain UEFI doesn't.

gozora commented at 2018-11-05 19:35:

@jsmeix

I look forward to your first EFISTUB support in ReaR!

Unfortunately I'm quite short on time last few weeks, I'll try to start my work on EFISTUB support next week ...

Automatically detecting EFISTUB would be similar precarious task as finding right UEFI boot loader, hence I'll most probably let user manually specify UEFI_BOOTLOADER="EFISTUB" at lest for early stages.

V.

jsmeix commented at 2018-11-06 12:13:

@gozora
no rush - take your time!

Yes, first and foremost the user must be able to specify what he wants
so that he has the final power to command what ReaR does.
Then as fallback if nothing was specified a best effort attempt
to guess how the system was booted can be done,
cf. rescue/default/850_save_sysfs_uefi_vars.sh

gozora commented at 2018-11-12 16:26:

I've put together some very basic code for EFISTUB support in ReaR, it currently works only for OUTPUT=ISO and restore (rear recover) part does not work at all.
I've finally decided not to use UEFI_BOOTLOADER="EFISTUB", since it don't feel right and can become quite confusing, since EFISTUB doesn't really use traditional Linux boot loader.

New variable EFI_STUB in global.conf.was introduced and will imply USING_UEFI_BOOTLOADER=1 and UEFI_BOOTLOADER="/usr/lib/systemd/boot/efi/systemd-bootx64.efi" with message in log file in case that user will try to set these variables.

When using EFISTUB=y, UEFI_BOOTLOADER have meaning only for creation process of ReaR recovery system where it is used to display boot menu in same way like Syslinux, Grub or Elilo did.

V.

jsmeix commented at 2018-11-13 08:21:

@gozora
great to have some very basic EFISTUB support in ReaR!
Now there is an entry point where things can be further enhanced as needed.

I do not understand what you wrote
it ... works ... for ... restore and
rear recover part does not work at all
because 'restore' is usually a part of "rear recover" (when 'restore'
means the 'restore' of the backup during "rear recover").

Some initial review of your
https://github.com/rear/rear/compare/master...gozora:EFISTUB
by plain looking at the code:

I think in usr/share/rear/pack/GNU/Linux/500_check_EFISTUB.sh
the 'StopIfError' is not needed and could be simplified to plain 'Error'
because the 'if' condition had already tested that this is the error case
(I wonder why there is $( ... ) command substitution in the 'if' condition).
Because 'Error' exits the whole ReaR I think it could be simplified
to a single COMMAND || Error line for example like

grep -q -i "EFI stub" $KERNEL_FILE || Error "$KERNEL_FILE is not compiled with EFISTUB support"
LogPrint "Using $KERNEL_FILE with EFISTUB support"

or similar.

In usr/share/rear/rescue/default/850_save_sysfs_uefi_vars.sh
the hardcoded

UEFI_BOOTLOADER="/usr/lib/systemd/boot/efi/systemd-bootx64.efi"

should have at least a test that this file exists
as it is done elsewhere in that script like

if is_true $EFI_STUB; then
    # Add checks here
    UEFI_BOOTLOADER="/usr/lib/systemd/boot/efi/systemd-bootx64.efi"
    test -f "$UEFI_BOOTLOADER" && return
    Error "EFISTUB requires $UEFI_BOOTLOADER"
fi

because otherwise it could bindly proceed with a UEFI_BOOTLOADER value
that cannot work and then things may fail later at an unexpected place
cf. "Try to care about possible errors" in
https://github.com/rear/rear/wiki/Coding-Style

gozora commented at 2018-11-13 09:21:

Hello @jsmeix,

Thanks for your inputs ...

I do not understand what you wrote
it ... works ... for ... restore and
rear recover part does not work at all
because 'restore' is usually a part of "rear recover" (when 'restore'
means the 'restore' of the backup during "rear recover").

Let me simplify this ;-), it works currently only to create ReaR recovery system and backup (rear mkbackup/mkrescue).

For hardcoded: UEFI_BOOTLOADER="/usr/lib/systemd/boot/efi/systemd-bootx64.efi" there will be checks for file existence, as you can see in comment:

if is_true $EFI_STUB; then
    # Add checks here
   ...

I was mostly interested about general opinion on introducing $EFI_STUB configuration variable and its functionality when it is internally overriding variables that can be normally set by user (UEFI_BOOTLOADER and USING_UEFI_BOOTLOADER).
I'm not sure if we have some other code in ReaR that behaves like this.
I personally as user, don't like situations where I configure some settings and they are overridden by application for whatever reason without even simple explanation why ...

V.

jsmeix commented at 2018-11-13 09:41:

@gozora
using separated config variables for separated things is "the right thing".

In particular I prefer to Keep Separated Issues Separated (KSIS)
to avoid RFC 1925 item (5) https://tools.ietf.org/html/rfc1925
as a precondition to Keep It Simple and Straightforward (KISS).

We have other cases in ReaR where some config variable sets another
config variable if the other config variable was not explicitly specified
e.g. "If you do not specify OUTPUT_URL variable then by default
it will be aligned to what was defined by variable BACKUP_URL" in
https://github.com/rear/rear/blob/master/doc/rear.8.adoc

I saw your 'Add checks here' comment but misunderstood it
as if it meant that some automated search for systemd-bootx64.efi
could be added later here (something like the other automated
search code in rescue/default/850_save_sysfs_uefi_vars.sh).

gozora commented at 2018-11-13 09:48:

We have other cases in ReaR where some config variable sets another
config variable if the other config variable was not explicitly specified
e.g. "If you do not specify OUTPUT_URL variable then by default
it will be aligned to what was defined by variable BACKUP_URL" in
https://github.com/rear/rear/blob/master/doc/rear.8.adoc

Yes I know about situations like this, but what I'm planning to do, is to override UEFI_BOOTLOADER and USING_UEFI_BOOTLOADER even if they are explicitly set by user in local.conf or site.conf.

V.

schlomo commented at 2018-11-13 10:14:

Looking over your branch and reading through this issue I have a few thoughts/questions:

  • Can we somehow separate between the "rescue media boot method" and "recovered system boot method" topics? They don't have to be same same and IMHO they should not be coupled.
  • Everything that modifies the rescue media should IMHO show up when you do a rear dump. Easiest way to achieve that (and to make it obvious to the user) is to prefix such variables with OUTPUT_ because then they are shown by rear dump with the other output related stuff.
  • Before setting a variable to the full path of systemd-bootx64.efi it might be prudent to check that this file actually exists (or maybe I missed the check).
  • Changing the boot loader of the recovered system is a heavy change and part of a migration. IMHO a variable to control that should start with MIGRATE_ or something like this. Probably it would be good to add a section to the output of rear dump that will print all variables starting from MIGRAT so that users can much easier see migration-related settings.
  • IMHO the only correct way of finding (and not guessing) the EFI System Partition is
    1. find the first GPT partition with a GUID of C12A7328-F81F-11D2-BA4B-00A0C93EC93B (or MBR partition with a type of 0xEF)
    2. find where this partition is mounted in the system
  • Maybe we should have the dump workflow also run SourceStage "prep" initially so that the dump output will actually reflect the calculated variables and not only show the static configuration.

Just for my understanding: Is EFISTUB the default UEFI boot loader in Arch? Does Arch with grub2 on UEFI work already (the find-the-ESP problem should be the same there, no?)?

If the core problem of this issue is "support EFISTUB booting on Arch" then we should maybe rename it so.

jsmeix commented at 2018-11-13 10:27:

@gozora
what the user commands is sacrosanct, cf. "WHAT A MESS!" in
https://github.com/rear/rear/pull/1733#issuecomment-367680598

In current default.conf there is

USING_UEFI_BOOTLOADER=
...
UEFI_BOOTLOADER=""
...
SECURE_BOOT_BOOTLOADER=""

so by default none of those config variables have an actual value.
In this default case you can set their value to whatever looks right
when EFI_STUB has a true value because when a config value has
no real value it means ReaR can do and does automated things.

But when a config variable has a real value this is a command
from the user and then ReaR has to obey where "obey" also means
ReaR has to error out if the user commanded something impossible.

Accordingly if the user specified

EFI_STUB=true
UEFI_BOOTLOADER="/usr/lib/mysystemd/myboot/myefi/mysystemd-bootx64.efi"

and '/usr/lib/mysystemd/myboot/myefi/mysystemd-bootx64.efi' exists
(on Arch Linux things can be anything anywhere) ReaR has to obey
but otherwise (when that file does not exist) ReaR has to error out.

Accordingly I suggest in rescue/default/850_save_sysfs_uefi_vars.sh
code like

for dummy in "once" ; do
    if test -f "$SECURE_BOOT_BOOTLOADER" ; then
        UEFI_BOOTLOADER="$SECURE_BOOT_BOOTLOADER"
    fi

    ...
    test -f "$UEFI_BOOTLOADER" && continue

    if is_true $EFI_STUB; then
        UEFI_BOOTLOADER="/usr/lib/systemd/boot/efi/systemd-bootx64.efi"
        test -f "$UEFI_BOOTLOADER" && return
        Error "EFISTUB requires $UEFI_BOOTLOADER"
    fi

I do not know how SECURE_BOOT_BOOTLOADER and EFI_STUB
could be related to each other - perhaps additional checks are needed?

Furthermore in prep/default/320_include_uefi_env.sh an additional check
is needed at the beginning to error out if USING_UEFI_BOOTLOADER is false
but EFI_STUB is true like

# by default the variable USING_UEFI_BOOTLOADER is empty which means ReaR will decide (this script)
# except when the variable USING_UEFI_BOOTLOADER has an explicit 'false' value set:
if is_false $USING_UEFI_BOOTLOADER ; then
    # we forced the variable to zero (in local.conf) so we do not want UEFI stuff
    # except EFI_STUB is true which conflicts here:
    is_true $EFI_STUB && Error "EFI_STUB true conflicts with USING_UEFI_BOOTLOADER false"
    Log "We do not want UEFI capabilities in ReaR (USING_UEFI_BOOTLOADER=0)"
    return
fi

jsmeix commented at 2018-11-13 10:36:

@gozora
and you can (and should) explain special interdependencies
between config variables (and/or their values) in default.conf
so that users who manually specify those config values are informed.

jsmeix commented at 2018-11-13 10:45:

@schlomo
thanks to mention

separate between the "rescue media boot method"
and "recovered system boot method"

because I fear I already confused both methods here.
I usually confuse them because their config variable names
do not make it obvious whereto they belong but we cannot
change config variable names...

There are no properly and meaningful prefixed
config variables to distinguish between things that belong to
the ReaR recovery system (e.g. SECURE_BOOT_BOOTLOADER)
versus things for the original system (e.g. GRUB_RESCUE)
versus things for the recreated system (e.g. BOOTLOADER)
versus things that belong both to the ReaR recovery system
and to the recreated system (e.g. UEFI_BOOTLOADER).

gozora commented at 2018-11-13 11:17:

Hello @schlomo

IMHO the only correct way of finding (and not guessing) the ...

The thing is, that with EFISTUB we don't need to look for esp, nor doing all the guess work for boot loaders (*.efi) detection.

Just for my understanding: Is EFISTUB the default UEFI boot loader in Arch?

It is hard to say what is actual default on Arch :-). According almighty Arch wiki:

Under UEFI
...

  1. Firmware launches the EFI application.

  2. This could be a boot loader or the Arch kernel itself using EFISTUB.

  3. It could be some other EFI application such as a UEFI shell or a boot manager like systemd-boot or rEFInd.

Does Arch with grub2 on UEFI work already (the find-the-ESP problem should be the same there, no?)?

The main "problem" is that Arch allows you to setup system booting whatever you like. The possibilities are endless and I think ReaR can't cover them all. So if you setup Arch with Grub on UEFI with esp on /boot/efi everything should work just fine.

V.

gozora commented at 2018-11-13 11:22:

@jsmeix your https://github.com/rear/rear/issues/1942#issuecomment-438215291 makes perfect sense.
It is much better to stop execution if configuration values are conflicting or incorrect, than changing things behind the scenes ...

V.

schlomo commented at 2018-11-13 11:28:

The thing is, that with EFISTUB we don't need to look for esp, nor doing all the guess work for boot loaders (*.efi) detection.

What about adding the correct kernel & kernel command line to the UEFI boot menu via efibootmgr? Don't we need to know where to find the ESP for that?

gozora commented at 2018-11-13 11:33:

What about adding the correct kernel & kernel command line to the UEFI boot menu via efibootmgr? Don't we need to know where to find the ESP for that?

No, we could either use efibootmgr -v where all the boot options should be visible (AFAIK they are in unicode, so some additional work would be required), or check /proc/cmdline

V.

schlomo commented at 2018-11-13 11:34:

IMHO checking command line of the currently running kernel is not enough. There could be also other boot entries, e.g. to start a previous kernel version or to start in some safe mode or single user mode.

gozora commented at 2018-11-13 11:38:

You mean that someone could run rear mkbackup when currently booted in single user more, or in previous kernel, ... ?

V.

schlomo commented at 2018-11-13 11:42:

No, I mean that there could be other boot configurations. On SLES, RHEL, Debian for example you will find another boot configuration for the previous kernel (in case the new kernel won't boot) and also additional "safe mode" boot configurations. We should restore them if they exist.

gozora commented at 2018-11-13 11:48:

For something like this, we could either dump whole EFI boot entries related configuration and restore it after rear recover, or introduce some new configuration variable where user could specify efibootmgr command to create entries after rear recover is done ...

But for neither of these cases we need to know location of esp.

V.

jsmeix commented at 2018-11-13 12:37:

Regarding additional special boot options for the bootloader
of the recreated system you may have a look at
https://github.com/rear/rear/issues/1828#issuecomment-399050741
which looks related to things like
"another boot configuration for the previous kernel"

My point therein is

Because on a by "rear recover" recreated system
there is no such thing as a meaningful previous boot
(because "rear recover" recreates a system by reinstalling it
completely from scratch) so that there is no meaningful use-case
to remember any information from a possible previous boot.

In general regarding other boot configurations on SLES and RHEL:
As far as I know at least on SLES only GRUB is used as bootloader
(there is no EFISTUB support on SLES as far as I know)
so that the GRUB config files get restored from the backup
during "rear recover" that should result the same GRUB configuration
as it was on the original system.

gozora commented at 2018-11-18 15:05:

Thanks for all comments so far!
Some of them have already been implemented.

V.

jsmeix commented at 2018-11-19 09:09:

@gozora
thank you for your continuous efforts to add EFISTUB support in ReaR!

I had a quick look at your latest
https://github.com/rear/rear/compare/master...gozora:EFISTUB
and found (I cannot add comments directly there so that I post them here):

(a)
In your usr/share/rear/lib/bootloader-functions.sh there is (excerpts)

    # Use grep ' on / ' with explicitly specified spaces as separators instead of
    # grep -w 'on /' ...
    ...
    echo $(mount | grep -w ' on / ' | ...

so that the code grep -w ' on / ' does no longer match its comment.
What is the reason that the -w option is now needed in addition
to the explicitly specified spaces as separators in ' on / '?
Such a reason should be explained in the comment so that others
know why the code must be this way (and do not attempt to simplify it
into something that does no longer work in this or that special cases).

(b)
In usr/share/rear/output/ISO/Linux-i386/250_populate_efibootimg.sh
there is (excerpt)

# Use 260_populate_efistub.sh instead.
# There much of Grub/Elilo code here exclude it in menaningfull way.
is_true $EFI_STUB || return 0

which skips 250_populate_efibootimg.sh when EFI_STUB is not used
but the comment indicates the opposite that 250_populate_efibootimg.sh
should be skipped when EFI_STUB is used - or what do I misunderstand here?

(c)
In usr/share/rear/pack/GNU/Linux/500_check_EFISTUB.sh

! is_true $USING_UEFI_BOOTLOADER && Error "..."

is there a reason why not using

is_true $USING_UEFI_BOOTLOADER || Error "..."

which would be simpler from my personal point of view (which is
a matter of taste how I personally read code)?

(d)
In usr/share/rear/rescue/default/850_save_sysfs_uefi_vars.sh (excerpts):

    # ... we will use systemd-bootx64.efi (if not overridden in local.conf / site.conf).
    ...
    # This looks to be the best guess for finding our boot loader.
    UEFI_BOOTLOADER="/usr/lib/systemd/boot/efi/systemd-bootx64.efi"

this way '/usr/lib/systemd/boot/efi/systemd-bootx64.efi' is used hardcoded
so that the user cannot override what to use.
Therefore I would suggest something like

    # ... we will use systemd-bootx64.efi (if not overridden in local.conf / site.conf).
    ...
    # If not specified by the user use /usr/lib/systemd/boot/efi/systemd-bootx64.efi
    test -f "$UEFI_BOOTLOADER" || UEFI_BOOTLOADER="/usr/lib/systemd/boot/efi/systemd-bootx64.efi"

gozora commented at 2018-11-19 11:16:

Hello @jsmeix,

Thanks for your valuable inputs, I'll check them later today ...

V.

gozora commented at 2018-11-19 11:21:

@jsmeix I can comment at least for point d):

this way '/usr/lib/systemd/boot/efi/systemd-bootx64.efi' is used hardcoded
so that the user cannot override what to use.
Therefore I would suggest something like

User can override UEFI_BOOTLOADER by code already present in 850_save_sysfs_uefi_vars.sh

for dummy in "once" ; do
    if test -f "$SECURE_BOOT_BOOTLOADER" ; then
      UEFI_BOOTLOADER="$SECURE_BOOT_BOOTLOADER"
    fi

    # When the user has specified UEFI_BOOTLOADER in /etc/rear/local.conf use it if exists and is a regular file.
    # Double quotes are mandatory here because 'test -f' without any (possibly empty) argument results true:
    test -f "$UEFI_BOOTLOADER" && continue <== Here it UEFI_BOOTLOADER is set at this stage, further loop execution will stop ...
    ...

V.

jsmeix commented at 2018-11-19 12:30:

@gozora
you are right - my misunderstanding happened because
I had only looked at the diff but not at the whole file.

gozora commented at 2018-11-19 16:36:

Hello @jsmeix,

About your comment: https://github.com/rear/rear/issues/1942#issuecomment-439820498

First and foremost, thanks for carefully checking my code!

Points a), b) and c) were corrected.

For point a) your are 100% right!
I've mistakenly changed the code to use -w because change in 940_grub2_rescue.sh:

-    root_uuid=$(mount | grep -w 'on /' | awk '{print $1}' | xargs blkid -s UUID -o value)
+    root_uuid=$(get_root_disk_UUID)

but I did not noticed code is using 'on /' instead of ' on / '.

Thanks for pointing this out!

V.

jsmeix commented at 2018-11-20 10:00:

I am happy I was able to contribute at least a little bit here
( when I hear [U]EFI I usually try to escape ;-)

gozora commented at 2018-11-22 19:48:

I'm somehow not OK with using UEFI_BOOTLOADER together with EFISTUB.
It don't feel right as EFISTUB doesn't really use a boot loader (only for auto-magic boot of ReaR rescue system).
I've created another branch where I'm experimenting a bit.
I'm thinking about throwing error when user manually sets UEFI_BOOTLOADER and use new variable (something like REAR_RESCUE_EFISTUB_BOOT_LOADER_LOCATION, or maybe something shorter :-)), to distinguish between booting UEFI with Grub/Elilo and EFISTUB.

V.

schlomo commented at 2018-11-23 09:38:

👍 for quality.

Would it be possible to use variables starting with OUTPUT_ to configure and describe everything related to the operation of the ReaR rescue image? That way a rear dump will actually show them.

gozora commented at 2018-11-23 09:39:

@schlomo sure ...

jsmeix commented at 2018-11-23 10:17:

Is EFISTUB only about EFISTUB booting of the recovery system
(in this case OUTPUT_EFISTUB* variables would be o.k.)
and it will be never ever intended to be also used
to set up EFISTUB booting of the recreated system?

I think usually booting related config variables affect both
booting of the recovery system and booting of the recreated system
(except exceptions), cf. my above
https://github.com/rear/rear/issues/1942#issuecomment-438220832

jsmeix commented at 2018-11-23 10:23:

I wished we had
OUTPUT_BOOT* variables for booting the recovery system
versus
TARGET_BOOT* variables for booting the recreated system

Probably empty OUTPUT_BOOT* variables could inherit
the values from matching TARGET_BOOT* variables
and vice versa for user convenience.

gozora commented at 2018-11-23 10:32:

Is EFISTUB only about EFISTUB booting of the recovery system
(in this case OUTPUT_EFISTUB* variables would be o.k.)
and it will be never ever intended to be also used
to set up EFISTUB booting of the recreated system?

EFISTUB is a great simplification of Linux boot process. Where we don't need to guess, search or build boot loaders on running system (set UEFI_BOOTLOADER during rear mkbackup/mkrescue) neither we need to install boot loaders during rear recover.
All we need to know for successful boot is location of kernel file (that must be compiled with EFISTUB support) and options that should be passed to kernel (initrd, kernel options as we know them from Grub/Elilo).
I'm planning to write code that will be capable of some auto-magic detection of these options or allow user to set them manually.

V.

jsmeix commented at 2018-11-23 13:38:

But I think the user should have a config variable where he could
enforce or forbid ReaR to set up EFISTUB booting.

Assume ReaR determines the kernel is compiled with EFISTUB support
and ReaR does not find a GRUB config, does that mean to set up
EFISTUB booting for the recovery system and the recreated system
in any case and there is no way for the user to get something else?

Assume ReaR determines the kernel is compiled with EFISTUB support
and ReaR finds a GRUB config, does that mean to set up GRUB booting
in any case and there is no way for the user to enforce a migration
to EFISTUB booting for the recreated system?

gozora commented at 2018-11-23 14:53:

EFISTUB has to be activated manually (EFI_STUB=y), because I currently don't know 100% reliable way for detecting EFISTUB (boot loaders might or might not be installed, but not used).
I'm planning to create documentation on this topic though ...

V.

gozora commented at 2018-12-10 19:35:

Just goofing around with EFISTUB code
With enabled settings in local.conf

...
EFI_STUB=y
KERNEL_FILE="/boot/efi/vmlinuz-4.4.21-69-default"
EFI_STUB_EFIBOOTMGR_ARGS="initrd=initrd-4.4.21-69-default root=/dev/mapper/system-root"

ReaR was capable to migrate SLES12 SP2 from Grub to EFISTUB nearly out of the box :-).
Another site effect of EFISTUB is that it might send ebiso into retirement ...

V.

jsmeix commented at 2018-12-11 09:17:

@gozora
awesome!

Could you explan what "nearly out of the box" exactly means?
I.e. did you need to do other manual stuff (except specifying the right config variables)?

FYI
regarding "migrate from Grub to EFISTUB"
we have the related more general issue
https://github.com/rear/rear/issues/1437
so that "migrate from Grub to EFISTUB" could be
a first example where ReaR migrates the bootloader.

jsmeix commented at 2018-12-11 09:39:

@gozora
because I cannot add coments to your
https://github.com/gozora/rear/compare/master...gozora:EFISTUB
I post them here:

In usr/share/rear/finalize/default/050_prepare_checks.sh you set

is_true $EFI_STUB && NOBOOTLOADER=""

I wonder if 050_prepare_checks.sh is the right place for that.
Usually NOBOOTLOADER="" is only set at the end of a script
that actually successfully had installed a bootloader.
Accordingly I think NOBOOTLOADER="" should happen
only after EFISTUP setup was actually done successfully.

In general you use always the test is_true $EFI_STUB.
I did not carefully check the details in your code - I think it is o.k.

In general be careful with using is_true and is_false
because both are not logical inverse, see the comments
in lib/global-functions.sh (excerpt)

'! is_true' is not the same as 'is_false'
and '! is_false' is not the same as 'is_true'

Simply put - also imagine what happens in case of
EFI_STUB='' or EFI_STUB=' ' or EFI_STUB='fubar'
(then both is_true $EFI_STUB and is_false $EFI_STUB return 'false')
even then some reasonable default/fallback behaviour should happen
alternatively you may perhaps add a test in the 'init' stage that errors out
when both is_true $EFI_STUB and is_false $EFI_STUB return 'false'?

gozora commented at 2018-12-11 10:19:

Hello @jsmeix,

Could you explan what "nearly out of the box" exactly means?
I.e. did you need to do other manual stuff (except specifying the right config variables)?

I'm planning to fully explain this in documentation that I'll prepare once I'm OK with EFISTUB code.

But in general I had SLES12SP2 with Grub installed booting using UEFI.
Once of the requirements for EFISTUB will be that your KERNEL_FILE MUST be located on VFAT partition.
Since this requirement was not met, I had to copy KERNEL_FILE and initrd to /boot/efi (VFAT filesystem) and tell ReaR where EFISTUB capable kernel is located (KERNEL_FILE="/boot/efi/vmlinuz-4.4.21-69-default").
After rear recover was over, ReaR recreated initrd what was located on original location (in /boot) so I had to copy it again to /boot/efi, because initrd in /boot/efi is the real initrd that is booted by EFISTUB.

This could be avoided if I would simply migrate /boot from ext3 to vfat ....

I'll send you reply to your https://github.com/rear/rear/issues/1942#issuecomment-446135800 later today (or later tomorrow) once I have a bit more time ...

V.

gozora commented at 2019-01-05 12:56:

Hello @jsmeix,

I wonder if 050_prepare_checks.sh is the right place for that.
Usually NOBOOTLOADER="" is only set at the end of a script
that actually successfully had installed a bootloader.
Accordingly I think NOBOOTLOADER="" should happen
only after EFISTUP setup was actually done successfully.

I've moved "reset" of NOBOOTLOADER variable directly into script responsible for boot entry creation. It really makes more sense ...

general be careful with using is_true and is_false ...

Corrected by commit

Thanks for your inputs.

Soon I'll start writing some small documentation on EFISTUB code describing supported/tested setup and possible behavior during migration to EFISTUB.

V

gozora commented at 2019-02-11 20:15:

@johnny-cash
With https://github.com/rear/rear/pull/1945 and https://github.com/rear/rear/pull/2030 merged, changes that "whatever wild boot UEFI setup" can be successfully restored by ReaR have grown a bit more ;-).

Please test and let us know!

V.

jsmeix commented at 2019-02-22 11:11:

No news are good news.

[Export of Github issue for rear/rear.]