#1204 PR `merged`: Enhancement: find 64-bit systemd UEFI bootloader¶

Labels: enhancement, fixed / solved / done

ProBackup-nl opened issue at 2017-02-22 21:50:¶

As seen on UEFI 64-bit Arch Linux distributions with efibootmgr package installed.
Upon "bootctl install" the .efi files are:
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/systemd-bootx64.efi".
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/BOOTX64.EFI".

When system is booting withing having a listing in the UEFI boot menu (efivars) then the UEFI boot will most likely be booted using file /boot/EFI/BOOT/BOOTX64.EFI.

jsmeix commented at 2017-02-23 08:45:¶

@gozora
I think this one is a successor of
https://github.com/rear/rear/pull/1203

gozora commented at 2017-02-23 08:56:¶

@jsmeix

yes, noticed that ;-). But #1203 was closed during I wrote my https://github.com/rear/rear/pull/1203#discussion_r102582886, so I've stayed and commented, there ...

gozora commented at 2017-02-23 16:55:¶

Uffff, I don't remember last time when Linux distro installation took me more than 15 minutes, oh wait, I know "Linux From Scratch" :-). You should warn me @ProBackup-nl!
But on the other hand, it is nice to see how lazy I've become over time!

I have a question here. From what I've understood you have to prepare everything by your self in Arch live environment. Isn't there some magical command like pacstrap --i-have-no-idea-what-to-do=ofcourse just_do_some_default_partitioning_and_install_my_freaking_efi_system ?

Do I get it right that having whole /boot on vfat was your own decision?

V.

gozora commented at 2017-02-23 19:36:¶

Hello @ProBackup-nl,
Looks like this will require a bit more work to make ReaR running on Arch...

From my first testing:

OUTPUT=ISO does not work at all for me (did not check deeper for reason)
OUTPUT=USB works somehow (when I manually set USING_UEFI_BOOTLOADER=1 which should be IMHO opaque variable)
Patch you've provided did not work you my Arch distro because I had /boot/EFI/grub/grubx64.efi instead of yours BOOTX64.EFI
I did not check deeper for possible problems when /boot/EFI is used directly and not as part of "standard" /boot/efi/EFI

In general I'd say that current ReaR code does not work on Arch Linux.
@gdha, @jsmeix, @schlomo, @didacog or anybody else that might get this message, please let me know if there is someone running ReaR flawlessly on Arch Linux.

I'll continue to work on this topic next days and see how much effort would it require to make ReaR behave correctly on Arch.

I'll keep you posted.

V.

ProBackup-nl commented at 2017-02-23 23:31:¶

@gozora Because UEFI has got a boot menu implementation in itself, I prefer not to chain another boot menu after that. In fact I prefer to most quickly (directly) UEFI boot the Linux kernel (using EFISTUB, which is enabled by default in Arch Linux).
Arch Linux also moved to systemd, so systemd gets installed by default, and that comes with systemd-boot, which can only start EFI executables. EFI booting is all I need. systemd-boot only boots from VFAT. It is my decision to not install an additional boot loader. There is not much choice in using /boot. Anything else is even more manual labour:

Note: systemd-boot cannot load EFI binaries from other partitions. It is therefore recommended to mount your ESP to /boot. In case you want to separate /boot from the ESP see #Manually for more information.

Arch can use many different boot loaders, my interest is only in systemd-boot, UEFI, 64-bit.

gozora commented at 2017-02-24 06:28:¶

@ProBackup-nl
Did you somehow manipulate with USING_UEFI_BOOTLOADER variable?

didacog commented at 2017-02-24 09:05:¶

Hi @gozora,

I've used Archlinux, but the "problem" I see with that distro, from the ReaR point of view, is that you can define boot partition, bootloader, fstypes,... as you want and there is nothing strictly standard like on other distros, with archlinux is always the user choice. Will be more difficult make it work flawessly like on RHEL, Debian, SLES,... that have, more or less, standard installation methods.

gozora commented at 2017-02-24 09:07:¶

Thanks @didacog!
That's what I actually thought! But as I'm Arch noob I wanted to be sure that I did not overlooked something.

V.

didacog commented at 2017-02-24 09:10:¶

@gozora I love Arch, but if you think in standard stuff, will be difficult to find 2 installations with same config... :-P

gozora commented at 2017-02-24 09:30:¶

@didacog, I like the idea of clean system (Arch) as well, it can be certainly useful in certain scenarios. Lets see if I can make ReaR like it as well :-)

jsmeix commented at 2017-02-24 10:03:¶

I think we should better open a separated
ReaR GitHub enhancement issue regarding
"ReaR support for Arch Linux".

In general regarding
https://github.com/rear/rear/pull/1204#issuecomment-282240724

... with that distro, from the ReaR point of view, is
that you can define boot partition, bootloader, fstypes,...
as you want and there is nothing strictly standard like
on other distros ...

For me there are way too much secretly working
"magic automatisms" in ReaR that try to do "things right"
but of course no automatism is infallible in parctice
so that when such automatisms fail it results often
really bad user experience because for the user it looks
as if the "whole ReaR thing" behaves inexplicable.

Therefore I like to get ReaR step by step moved
away from secretly working automatisms towards
procedures that can be fully controlled by the user
via appropriate variable settings if needed but that
also use reasonable defaults or fallback values
when those variables are not specified.

This way the current default behaviour of ReaR
does not change but if needed the user can specify
what ReaR should do in this or that particular case.

When there are no standards in Arch Linux
then Arch Linux is only a good template case
for any specific arbitrary Linux installation
where the admin could have set up anything
in his arbitrary special non-standard way.

Ideally - in the future - ReaR should be able
to also work in such cases - of course not
automatically right out of the box - but the user
should be able to tell ReaR about each and any
of his special non-standard things.

jsmeix commented at 2017-02-24 10:05:¶

@gozora regarding your question in
https://github.com/rear/rear/pull/1204#issuecomment-282096912

I never used Arch Linux.

jsmeix commented at 2017-02-24 10:28:¶

What looks wrong in this particular case is that
in default.conf there is only USING_UEFI_BOOTLOADER
but the actual thing UEFI_BOOTLOADER is set
by a "secretly working automatism".

I would prefer there is also UEFI_BOOTLOADER
documented and specified in default.conf
so that the user knows how to set it right
if needed in /etc/local.conf.

Perhaps - only an offhanded idea - the default setting of
UEFI_BOOTLOADER in default.conf could be a list
or an array of file names or even bash globbing expressions
and the actual code searches in the ordering of that list
for one single file and the first match is then used as
actual UEFI_BOOTLOADER value i.e. something like

test "${UEFI_BOOTLOADER[*]:-}" || Error "Empty UEFI_BOOTLOADER"
local found_uefi_bootloader=""
for uefi_bootloader in "${UEFI_BOOTLOADER[@]}" ; do
    if test -f "$uefi_bootloader" ; then
        UEFI_BOOTLOADER=( "$uefi_bootloader" )
        found_uefi_bootloader="yes"
        LogPrint "Using UEFI bootloader '$uefi_bootloader'"
        break
    fi
done
test "$found_uefi_bootloader" || Error "No UEFI bootloader found"

ProBackup-nl commented at 2017-02-28 13:07:¶

@gozora I started without modifying local.conf. After some error messages I ended up with /etc/rear/local.conf:

USING_UEFI_BOOTLOADER=1
UEFI_BOOTLOADER=/boot/EFI/BOOT/BOOTX64.EFI

jsmeix commented at 2017-02-28 13:33:¶

@ProBackup-nl
do I understand your
https://github.com/rear/rear/pull/1204#issuecomment-283033844
that it works for you without any modifications in the ReaR scripts
only by specifying in /etc/rear/local.conf

USING_UEFI_BOOTLOADER=1
UEFI_BOOTLOADER=/boot/EFI/BOOT/BOOTX64.EFI

?

If yes, I think at least for now it should help to also
document the UEFI_BOOTLOADER variable
in default.conf.

ProBackup-nl commented at 2017-02-28 13:50:¶

@jsmeix Comment https://github.com/rear/rear/pull/1204#issuecomment-283033844 is a reply to https://github.com/rear/rear/pull/1204#issuecomment-282215307:

Did you somehow manipulate with USING_UEFI_BOOTLOADER variable?

After making these changes, Rear still errors out.

jsmeix commented at 2017-02-28 13:58:¶

Regarding
https://github.com/rear/rear/pull/1204#issue-209592311

/boot/EFI/BOOT/BOOTX64.EFI

versus
https://github.com/rear/rear/pull/1204#issuecomment-282096912

/boot/EFI/grub/grubx64.efi

In general how to find a file by its name
regardless of case I suggest to use

find /boot/EFI -iname "BOOTX64.EFI"

cf.
http://unix.stackexchange.com/questions/32155/find-command-how-to-ignore-case

I tested it down to SUSE Linux Enterprise Server 10
and even there find supports '-iname':

# cat /etc/issue
Welcome to SUSE Linux Enterprise Server 10 SP4  (x86_64) ...

# find --version
GNU find version 4.2.27

# find /boot -iname 'system*'
/boot/System.map-2.6.16.60-0.85.1-default

jsmeix commented at 2017-02-28 15:06:¶

@gozora
I also think merging this pull request should not do any harm
so that - if you do not object - I could merge it soon
or you could merge it yourself if you can and like.

Later we can do further enhancements.

jsmeix commented at 2017-02-28 15:53:¶

@gdha
why should stderr be redicrectet to /dev/null ?
I think stderr goes to the ReaR log file and
I would want to see in particular any errors
in the ReaR log file.

For me it behaves well on command line:

# foo=$( find /boot -name "vmlinux*" | tail -1 ) ; echo "'$foo'"
'/boot/vmlinux-3.0.101-0.47.71-pae.gz'

# foo=$( find /boot -name "qqq" | tail -1 ) ; echo "'$foo'"
''

# foo=$( find qqq -name "qqq" | tail -1 ) ; echo "'$foo'"
find: `qqq': No such file or directory
''

gdha commented at 2017-02-28 16:08:¶

@jsmeix redirect to /dev/null makes sense as the next if block checks if the UEFI_BOOTLOADER variable is empty and if so then it will bail out with an error. It is a matter of taste. If you prefer to see an extra error then you make merge the pull request. I have no real objections.

gozora commented at 2017-02-28 16:11:¶

@jsmeix, @gdha,
Of topic, but are there some special rules for merging pull request, or it is enought to hit that big ~~red~~ green button?

V.

jsmeix commented at 2017-02-28 16:20:¶

@gozora
to merge a pull request hit that big green [Merge pull request]
button and then you need (you really need) to provide
a meaningful merge commit log message
(the default message is usually mostly useless)
that tells the essentials about 'what' and 'why' of it
preferably without too bad typos ;-) because one cannot
fix git commit log messages later with reasonable effort.

gozora commented at 2017-02-28 16:24:¶

Ah typos, thanks for pointing this out :-) (You can never use enough spellcheckers)

Would something like this be suitable for this merge message?

fix for finding UEFI boot loader on Arch Linux

jsmeix commented at 2017-02-28 16:29:¶

I agree it is a matter of taste to some degree.
I would prefer to see all stderr messages because
it makes a difference if the named file is not found
or if the directory where find should search does
not exist, cf. my above example
e.g. when there is in the ReaR log

find: `/boot/EFI': No such file or directory
Error "Cannot find a proper UEFI_BOOTLOADER...

versus only

Error "Cannot find a proper UEFI_BOOTLOADER...

jsmeix commented at 2017-02-28 16:32:¶

I think this pull request in not yet a real
"fix for finding UEFI boot loader on Arch Linux"
because currently it only implements to

Try to find UEFI_BOOTLOADER file BOOTX64.EFI in /boot/EFI
where it could be located in particular on Arch Linux.

gozora commented at 2017-02-28 16:41:¶

Right, thanks @jsmeix, for valuable lesson!
I'll watch ReaR commit comments more carefully, and who knows, maybe I will learn something! :-)

I'll use

Try to find UEFI_BOOTLOADER file BOOTX64.EFI in /boot/EFI
where it could be located in particular on Arch Linux.

V.

jsmeix commented at 2017-02-28 17:07:¶

@gozora
only FYI:
My preferred command to view git log messages is something like

git log --format="%ae %ad%n%s :%n%b%n" --graph | fmt -w 120 -u -t | less

cf.
https://github.com/rear/rear/issues/1073#issuecomment-270373403

ProBackup-nl commented at 2017-02-28 20:40:¶

@jsmeix
A clarification.

The location /boot/EFI/ has got very little to do with "Arch Linux", it is the UEFI spec ‘Fallback path’ mount point for most Linux installation:

/boot is the name that most Linux variants use to mount the ESP to
/EFI/ or even more specific /EFI/BOOT/BOOTX64.EFI (or \EFI\BOOT\BOOTX64.EFI from UEFI's point of view) is the location where most (crippled) 64-bit UEFI firmware will only look to further progress boot.

What it actually looks for is \EFI\BOOT\BOOT{machine type short-name}.EFI – ‘x64’ is the “machine type short-name” for x86-64 PCs. The other possibilities are BOOTIA32.EFI (x86-32), BOOTIA64.EFI (Itanium), BOOTARM.EFI (AArch32 – that is, 32-bit ARM) and BOOTAA64.EFI (AArch64 – that is, 64-bit ARM).

This mechanism is not designed for booting permanently-installed OSes. It’s more designed for booting hotpluggable, device-agnostic media, like live images and OS install media...

gozora commented at 2017-02-28 20:57:¶

I'm a bit lazy lately doing partition layouts so I just tell installer "do partitioning your self, I don't care" and here are couple of examples how partitioning was decided by my 8.7 Debian and Centos 7.2

root@debian:~# bdf | grep boot
/dev/sda1                 118M   77M   35M  70% /boot
/dev/sda2                 121M  766K  120M   1% /boot/efi

centos:(/root)(root)# bdf | grep boot
/dev/sda2                 194M   66M  119M  36% /boot
/dev/sda1                  64M   10M   54M  16% /boot/efi

For SLES12 SP1 running SAP HANA it is quite similar

/dev/mapper/vg_root-lv_root    100G   12G   89G  12% /
/dev/sda1                      156M  4,9M  151M   4% /boot/efi

It have nothing to do with technical solution but I simply don't like heaving /boot files (kernel, initrd, ...) on VFAT ...

Here I'd refer to RFC 1925 section 2.3

With sufficient thrust, pigs fly just fine. However, this is
not necessarily a good idea. It is hard to be sure where they
are going to land, and it could be dangerous sitting under them
as they fly overhead.

V.

jsmeix commented at 2017-03-01 08:51:¶

As time permits I will try to enhance
rescue/default/850_save_sysfs_uefi_vars.sh
plus appropriate documentation in default.conf
to make UEFI_BOOTLOADER work in a more general way.

jsmeix commented at 2017-03-01 13:01:¶

For a first proposal how to enhance
rescue/default/850_save_sysfs_uefi_vars.sh
plus appropriate documentation in default.conf see
https://github.com/rear/rear/pull/1212

jsmeix commented at 2017-03-03 14:18:¶

@ProBackup-nl
I would appreciate it if you could test
whether or not it still works for you
with current ReaR GitHub master code
where the follow-up
https://github.com/rear/rear/pull/1212
is merged.

ProBackup-nl commented at 2017-03-03 23:39:¶

@jsmeix
I did a # git stash && git pull

From https://github.com/rear/rear
   e310a6e7..1917742e  master     -> origin/master
Updating e310a6e7..1917742e
Fast-forward
 usr/share/rear/conf/default.conf                          |  20 ++++++++++
 usr/share/rear/prep/RSYNC/GNU/Linux/200_selinux_in_use.sh |   7 ++++
 usr/share/rear/rescue/default/850_save_sysfs_uefi_vars.sh | 194 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------------------------
 3 files changed, 145 insertions(+), 76 deletions(-)

# usr/sbin/rear -v mkrescue (still having syslinux package removed) outputs:

ERROR: Could not find file 'mbr.bin'. Syslinux version 3.08 or newer is required, 4.x prefered!

# tail -n20 /root/rear/var/log/rear/rear-d2.log
2017-03-04 00:35:09 Including prep/GNU/Linux/300_include_grub_tools.sh
2017-03-04 00:35:09 Including prep/GNU/Linux/310_include_cap_utils.sh
2017-03-04 00:35:09 Including prep/default/310_include_uefi_tools.sh
2017-03-04 00:35:09 Including prep/default/320_include_uefi_env.sh
2017-03-04 00:35:09 Including prep/USB/Linux-i386/340_find_mbr_bin.sh
2017-03-04 00:35:09 ERROR: Could not find file 'mbr.bin'. Syslinux version 3.08 or newer is required, 4.x prefered!
==== Stack trace ====
Trace 0: usr/sbin/rear:504 main
Trace 1: /root/rear/usr/share/rear/lib/mkrescue-workflow.sh:12 WORKFLOW_mkrescue
Trace 2: /root/rear/usr/share/rear/lib/framework-functions.sh:85 SourceStage
Trace 3: /root/rear/usr/share/rear/lib/framework-functions.sh:45 Source
Trace 4: /root/rear/usr/share/rear/prep/USB/Linux-i386/340_find_mbr_bin.sh:22 source
Trace 5: /root/rear/usr/share/rear/lib/_input-output-functions.sh:132 StopIfError
Message: Could not find file 'mbr.bin'. Syslinux version 3.08 or newer is required, 4.x prefered!
== End stack trace ==

jsmeix commented at 2017-03-06 09:25:¶

@ProBackup-nl
please do not post new unrelated issues
at arbitrary existing issues.
Create new separated GitHub issues for separated issues.
Your
https://github.com/rear/rear/pull/1204#issuecomment-284101007
is about prep/USB/Linux-i386/340_find_mbr_bin.sh
which was not changed recently.

[Export of Github issue for rear/rear.]

#1204 PR merged: Enhancement: find 64-bit systemd UEFI bootloader¶