#2338 Issue `closed`: Testing only /usr/lib/grub/x86_64-efi/moddep.lst is insufficient - could be also in /boot/grub/ or in /usr/share/grub*/¶

Labels: enhancement, bug, fixed / solved / done

jsmeix opened issue at 2020-03-04 09:58:¶

ReaR version ("/usr/sbin/rear -V"):
current master code
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):
openSUSE Leap 15.1
ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://192.168.122.1/nfs
SSH_ROOT_PASSWORD="rear"
USE_DHCLIENT="yes"
PROGRESS_MODE=plain
PROGRESS_WAIT_SECONDS=3
KEEP_BUILD_DIR="yes"
FIRMWARE_FILES=( no )
COPY_AS_IS+=( /home/JohnDoe )

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
PC real hardware
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86-64
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
UEFI
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
local disk
Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT" or "lsblk" as makeshift):

NAME        KNAME     PKNAME   TRAN   TYPE FSTYPE   SIZE MOUNTPOINT
/dev/sda    /dev/sda           sata   disk        931.5G 
|-/dev/sda1 /dev/sda1 /dev/sda        part vfat     500M /boot/efi
|-/dev/sda2 /dev/sda2 /dev/sda        part ext4   915.5G /
`-/dev/sda3 /dev/sda3 /dev/sda        part swap    15.6G [SWAP]
/dev/sdb    /dev/sdb           sata   disk        238.5G 
|-/dev/sdb1 /dev/sdb1 /dev/sdb        part         1000M 
|-/dev/sdb2 /dev/sdb2 /dev/sdb        part         1000M 
|-/dev/sdb3 /dev/sdb3 /dev/sdb        part            1K 
|-/dev/sdb5 /dev/sdb5 /dev/sdb        part ext2    1000M 
|-/dev/sdb6 /dev/sdb6 /dev/sdb        part         1000M 
`-/dev/sdb7 /dev/sdb7 /dev/sdb        part         1000M 
/dev/sr0    /dev/sr0           sata   rom           3.8G

Description of the issue (ideally so that others can reproduce it):

# usr/sbin/rear -D mkbackup
Relax-and-Recover 2.5 / Git
Using log file: /root/rear.github.master/var/log/rear/rear-g243.log
...
Created initrd.cgz with gzip default compression (136958432 bytes) in 18 seconds
ERROR: grub2-mkstandalone would not make bootable EFI image of GRUB2 (no /usr/lib/grub*/x86_64-efi/moddep.lst file)
Some latest log messages since the last called script 250_populate_efibootimg.sh:
  mkdir: created directory '/tmp/rear.BVHpAOPVK4xhxfw/tmp/mnt/EFI/BOOT/locale'
  '/boot/efi/EFI/opensuse/grubx64.efi' -> '/tmp/rear.BVHpAOPVK4xhxfw/tmp/mnt/EFI/BOOT/BOOTX64.efi'
  /root/rear.github.master/usr/share/rear/lib/_input-output-functions.sh: line 476: type: grub-mkstandalone: not found
  /root/rear.github.master/usr/share/rear/lib/_input-output-functions.sh: line 476: type: grub-probe: not found
                               $gprobe --target=fs "$p"
                               $gprobe --target=partmap "$p" | sed -e 's/^/part_/'
                               $gprobe --target=abstraction "$p"
                           done | sort -u ))
Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-g243.log for details
Exiting rear mkbackup (PID 22270) and its descendant processes ...
Running exit tasks
You should also rm -Rf /tmp/rear.BVHpAOPVK4xhxfw
Terminated

That Error exit happens in the build_bootx86_efi function
in lib/uefi-functions.sh and the log excerpt is

++ build_bootx86_efi /tmp/rear.BVHpAOPVK4xhxfw/tmp/mnt/EFI/BOOT/BOOTX64.efi /tmp/rear.BVHpAOPVK4xhxfw/tmp/mnt/EFI/BOOT/grub.cfg /boot /boot/efi/EFI/opensuse/grubx64.efi
++ local outfile=/tmp/rear.BVHpAOPVK4xhxfw/tmp/mnt/EFI/BOOT/BOOTX64.efi
++ local embedded_config=
++ local gmkstandalone=
++ local gprobe=
++ dirs=()
++ local dirs
++ modules=(${GRUB2_MODULES_LOAD:+"${GRUB2_MODULES_LOAD[@]}"})
++ local modules
++ shift
++ [[ -n /tmp/rear.BVHpAOPVK4xhxfw/tmp/mnt/EFI/BOOT/grub.cfg ]]
++ embedded_config=/boot/grub/grub.cfg=/tmp/rear.BVHpAOPVK4xhxfw/tmp/mnt/EFI/BOOT/grub.cfg
++ shift
++ dirs=(${@:+"$@"})
++ has_binary grub-mkstandalone
++ for bin in $@
++ type grub-mkstandalone
/root/rear.github.master/usr/share/rear/lib/_input-output-functions.sh: line 476: type: grub-mkstandalone: not found
++ return 1
++ has_binary grub2-mkstandalone
++ for bin in $@
++ type grub2-mkstandalone
++ return 0
++ gmkstandalone=grub2-mkstandalone
++ ((  2  ))
++ ((  0  ))
++ has_binary grub-probe
++ for bin in $@
++ type grub-probe
/root/rear.github.master/usr/share/rear/lib/_input-output-functions.sh: line 476: type: grub-probe: not found
++ return 1
++ has_binary grub2-probe
++ for bin in $@
++ type grub2-probe
++ return 0
++ gprobe=grub2-probe
++ '[' -n grub2-probe ']'
++ modules=($( for p in "${dirs[@]}" ; do
                             $gprobe --target=fs "$p"
                             $gprobe --target=partmap "$p" | sed -e 's/^/part_/'
                             $gprobe --target=abstraction "$p"
                         done | sort -u ))
+++ for p in "${dirs[@]}"
+++ grub2-probe --target=fs /boot
+++ sort -u
+++ grub2-probe --target=partmap /boot
+++ sed -e 's/^/part_/'
+++ grub2-probe --target=abstraction /boot
+++ for p in "${dirs[@]}"
+++ grub2-probe --target=fs /boot/efi/EFI/opensuse/grubx64.efi
+++ grub2-probe --target=partmap /boot/efi/EFI/opensuse/grubx64.efi
+++ sed -e 's/^/part_/'
+++ grub2-probe --target=abstraction /boot/efi/EFI/opensuse/grubx64.efi
++ test
++ Error 'grub2-mkstandalone would not make bootable EFI image of GRUB2 (no /usr/lib/grub*/x86_64-efi/moddep.lst file)'

"rear mkbackup" had worked before with some non-current master code
on that system so I guess the recent changes in UEFI area cause it.

The strage thing is that - as far as I can see - the Error exit test
in the build_bootx86_efi function

test /usr/lib/grub*/x86_64-efi/moddep.lst || Error ...

was not changed by a recent change in the UEFI area
so currently I wonder how that has ever worked before for me?

On my openSUSE Leap 15.1 system I have

# find /usr/lib/grub* | grep moddep
[no output]

# find /boot | grep moddep
/boot/grub2/x86_64-efi/moddep.lst

# find /usr | grep moddep
/usr/share/grub2/i386-pc/moddep.lst
/usr/share/grub2/x86_64-efi/moddep.lst

Accordingly the following hack makes "rear mkbackup" work again for me:

--- a/usr/share/rear/lib/uefi-functions.sh
+++ b/usr/share/rear/lib/uefi-functions.sh
@@ -99,10 +99,12 @@ function build_bootx86_efi {
-    test /usr/lib/grub*/x86_64-efi/moddep.lst || Error "$gmkstandalone would not make bootable EFI image of GRUB2 (no /usr/lib/grub*/x86_64-efi/moddep.lst file)"
+    test /usr/lib/grub*/x86_64-efi/moddep.lst || test /boot/grub*/x86_64-efi/moddep.lst || Error "$gmkstandalone would not make bootable EFI image of GRUB2 (no /usr/lib/grub*/x86_64-efi/moddep.lst or /boot/grub*/x86_64-efi/moddep.lst file)"

but I did not try if also "rear recover" works.

I will further investigate as time permits...

pcahyna commented at 2020-03-04 10:04:¶

@jsmeix this test was added in #2013 a year ago - is it possible that the version which worked was older than that?

jsmeix commented at 2020-03-04 10:09:¶

I guess I found the reason on openSUSE Leap 15.1:

# rpm -qf /usr/share/grub2/x86_64-efi/moddep.lst
grub2-x86_64-efi-2.02-lp151.21.9.1.noarch

# rpm -q --changelog grub2-x86_64-efi-2.02-lp151.21.9.1.noarch
...
* Wed Jan 23 2019 rw@suse.com
- Build platform-packages 'noarch' and move to '/usr/share/efi'
  for SUSE Manager.  (FATE#326960)
  * grub2-efi-xen-chainload.patch (bsc#1122563)
  * grub2-efi-xen-removable.patch (refresh)
...

FATE is the SUSE internal feature tracking tool
so it looks like a SUSE specific packaging change
for a SUSE specific feature for SUSE's SUSE Manager product.
Sigh!

Before I had used openSUSE Leap 15.0 and I guess
there it was in /usr/lib/grub2/x86_64-efi/moddep.lst

jsmeix commented at 2020-03-04 10:16:¶

@pcahyna @gozora
do you know if "all the other GRUB2 UEFI stuff" in ReaR could be expected
to "automatically do the right thing" when x86_64-efi/moddep.lst
is in an arbitrary directory that belongs to GRUB2 i.e. so that one
can assume GRUB2 "magically knows" where to find its
x86_64-efi/moddep.lst file when GRUB2 needs it?

Or in other words:
Do you think it should be sufficient for such a test
to find a x86_64-efi/moddep.lst file anywhere on the system
and if yes things look sufficiently OK to assume GRUB2 will work?

jsmeix commented at 2020-03-04 10:20:¶

In our current
https://github.com/rear/rear/blob/master/usr/share/rear/lib/uefi-functions.sh#L99
the comment does no longer match the code
because the comment talks about grub-mkimage
while the code is about $gmkstandalone i.e. grub2-mkstandalone
or grub-mkstandalone.

pcahyna commented at 2020-03-04 10:29:¶

In our current
https://github.com/rear/rear/blob/master/usr/share/rear/lib/uefi-functions.sh#L99
the comment does no longer match the code
because the comment talks about grub-mkimage
while the code is about $gmkstandalone i.e. grub2-mkstandalone
or grub-mkstandalone.

Sorry, I probably missed this comment when updating this code.

jsmeix commented at 2020-03-04 10:33:¶

@pcahyna
no worries - I will fix it by the way...

pcahyna commented at 2020-03-04 10:37:¶

@pcahyna @gozora
do you know if "all the other GRUB2 UEFI stuff" in ReaR could be expected
to "automatically do the right thing" when x86_64-efi/moddep.lst
is in an arbitrary directory that belongs to GRUB2 i.e. so that one
can assume GRUB2 "magically knows" where to find its
x86_64-efi/moddep.lst file when GRUB2 needs it?

Or in other words:
Do you think it should be sufficient for such a test
to find a x86_64-efi/moddep.lst file anywhere on the system
and if yes things look sufficiently OK to assume GRUB2 will work?

@jhlavac any idea on that? By the way, related Red Hat bug reports are https://bugzilla.redhat.com/show_bug.cgi?id=1492177 and https://bugzilla.redhat.com/show_bug.cgi?id=1512493 - the same problem in Fedora (https://bugzilla.redhat.com/show_bug.cgi?id=1419521) has led to #2013 and the test which seems to be the culprit here. I personally would prefer to fix the problem in Fedora Grub packages to DTRT and avoid the test completely.

jsmeix commented at 2020-03-04 10:47:¶

@pcahyna
fixing GRUB issues in GRUB or in its RPM packages
is the right solution for the future but ReaR must work
with what there is on the various Linux systems so
in ReaR theer are "Dirty hacks welcome", cf.
https://github.com/rear/rear/wiki/Coding-Style

I will change that Error into LogPrintError like

    # grub-mkstandalone needs a .../grub*/x86_64-efi/moddep.lst file (cf. https://github.com/rear/rear/issues/1193)
    # At least on SUSE systems that is in different 'grub2' directories (cf. https://github.com/rear/rear/issues/2338)
    # e.g. on openSUSE Leap 15.0 it is in /usr/lib/grub2/x86_64-efi/moddep.lst
    # but on openSUSE Leap 15.1 that was moved to /usr/share/grub2/x86_64-efi/moddep.lst
    # and the one in /boot/grub2/x86_64-efi/moddep.lst is a copy of the one in /usr/*/grub2/x86_64-efi/moddep.lst
    # so we do not error out if we do not find a /x86_64-efi/moddep.lst file because it could be "anywhere else" in the future
    # but we inform the user here in advance about possible problems when there is no /x86_64-efi/moddep.lst file.
    # Careful: usr/sbin/rear sets nullglob so that /usr/*/grub*/x86_64-efi/moddep.lst gets empty if nothing matches
    # and 'test -f' succeeds with empty argument so that we cannot use 'test -f /usr/*/grub*/x86_64-efi/moddep.lst'
    # also 'test -n' succeeds with empty argument but (fortunately/intentionally?) plain 'test' fails with empty argument:
    test /usr/*/grub*/x86_64-efi/moddep.lst || LogPrintError "$gmkstandalone may fail to make a bootable EFI image of GRUB2 (no /usr/*/grub*/x86_64-efi/moddep.lst file)"

which also makes "rear mkbackup" work for me on openSUSE Leap 15.1.

gozora commented at 2020-03-04 10:52:¶

Hello @jsmeix,

Followig commit excluded /boot from search party, could it be related?

https://github.com/rear/rear/commit/946580c40b0e7025e0a4e4ecd93c5ae6b191b2c7

V.

pcahyna commented at 2020-03-04 10:56:¶

Hello @jsmeix,

Followig commit excluded /boot from search party, could it be related?

946580c

It is a similar idea - look into known Grub directories to check whether what we need is there. I removed that code in the meantime because searching at those ever-changing hardcoded paths is not right, and this issues shows why.

pcahyna commented at 2020-03-04 11:00:¶

@pcahyna
fixing GRUB issues in GRUB or in its RPM packages
is the right solution for the future but ReaR must work
with what there is on the various Linux systems so
in ReaR theer are "Dirty hacks welcome", cf.
https://github.com/rear/rear/wiki/Coding-Style

I will change that Error into LogPrintError like

    # grub-mkstandalone needs a .../grub*/x86_64-efi/moddep.lst file (cf. https://github.com/rear/rear/issues/1193)
    # At least on SUSE systems that is in different 'grub2' directories (cf. https://github.com/rear/rear/issues/2338)
    # e.g. on openSUSE Leap 15.0 it is in /usr/lib/grub2/x86_64-efi/moddep.lst
    # but on openSUSE Leap 15.1 that was moved to /usr/share/grub2/x86_64-efi/moddep.lst
    # and the one in /boot/grub2/x86_64-efi/moddep.lst is a copy of the one in /usr/*/grub2/x86_64-efi/moddep.lst
    # so we do not error out if we do not find a /x86_64-efi/moddep.lst file because it could be "anywhere else" in the future
    # but we inform the user here in advance about possible problems when there is no /x86_64-efi/moddep.lst file.
    # Careful: usr/sbin/rear sets nullglob so that /usr/*/grub*/x86_64-efi/moddep.lst gets empty if nothing matches
    # and 'test -f' succeeds with empty argument so that we cannot use 'test -f /usr/*/grub*/x86_64-efi/moddep.lst'
    # also 'test -n' succeeds with empty argument but (fortunately/intentionally?) plain 'test' fails with empty argument:
    test /usr/*/grub*/x86_64-efi/moddep.lst || LogPrintError "$gmkstandalone may fail to make a bootable EFI image of GRUB2 (no /usr/*/grub*/x86_64-efi/moddep.lst file)"

which also makes "rear mkbackup" work for me on openSUSE Leap 15.1.

I will submit to you the downstream patch that the RHEL packages have been carrying and we can discuss it, ok? You can see it here: https://bugzilla.redhat.com/show_bug.cgi?id=1492177#c13 - it replaces the test added in #2013

pcahyna commented at 2020-03-04 11:02:¶

here's the sample output:

WARNING: /usr/lib/grub/x86_64-efi/moddep.lst not found, grub2-mkimage will likely fail. Please install the grub2-efi-x64-modules package to fix this.
ERROR: Error occurred during grub2-mkimage of BOOTX64.efi
Aborting due to an error, check /var/log/rear/rear-localhost.log for details

(grub2-mkimage should be replaced by grub2-mkstandalone. Note the WARNING is output by the downstream-added code, but the subsequent ERROR is from the original code when it actually fails to invoke grub2-mkimage. So in your case there would be a WARNING but no error. That's why the error message says "likely fail".)

jsmeix commented at 2020-03-04 11:11:¶

@gozora
thank you for your
https://github.com/rear/rear/issues/2338#issuecomment-594451960

You are right, on openSUSE Leap 15.1 I have

# find /usr/lib/grub*/x86_64-efi | grep part_gpt
find: ‘/usr/lib/grub*/x86_64-efi’: No such file or directory

# find /usr/*/grub*/x86_64-efi | grep part_gpt
/usr/share/grub2/x86_64-efi/part_gpt.mod
/usr/share/grub2/x86_64-efi/part_gpt.module

gozora commented at 2020-03-04 11:22:¶

So following change:

-     test /usr/lib/grub*/x86_64-efi/moddep.lst || Error "$gmkstandalone would not make bootable EFI image of GRUB2 (no /usr/lib/grub*/x86_64-efi/moddep.lst file)"
+     test /usr/*/grub*/x86_64-efi/moddep.lst || Error "$gmkstandalone would not make bootable EFI image of GRUB2 (no /usr/lib/grub*/x86_64-efi/moddep.lst file)"

Should be ~~temporarily~~ sufficient ...

V.

jsmeix commented at 2020-03-04 11:25:¶

@pcahyna
in general in ReaR we like to avoid the word WARNING, cf.
https://blog.schlomo.schapiro.org/2015/04/warning-is-waste-of-my-time.html
which originated here in ReaR at
https://github.com/rear/rear/issues/564#issuecomment-86418677
and subsequent comments there.

In
https://github.com/rear/rear/issues/1193#issuecomment-452617591
I wrote why I don't like explicit RPM package names.

So better sample output could be something like

Found no /usr/*/grub*/x86_64-efi/moddep.lst file so grub2-mkimage may fail.
ERROR: grub2-mkimage of BOOTX64.efi failed

or perhaps even better

grub2-mkimage may fail (found no /usr/*/grub*/x86_64-efi/moddep.lst)
ERROR: grub2-mkimage of BOOTX64.efi failed

to have the most interesting part grub2-mkimage may fail
at the beginning of the line (and not as "some trailing technical detail").

Furthermore regarding things like PrintIfError in
https://bugzilla.redhat.com/show_bug.cgi?id=1492177#c13
see
https://github.com/rear/rear/pull/2013#discussion_r245909346

jsmeix commented at 2020-03-04 11:32:¶

As a quick first step for now I did
https://github.com/rear/rear/commit/d7729b0baba907a66ccfd976ab133964ae4effbe

But the GRUB2 modules directory part still needs to be fixed, cf.
https://github.com/rear/rear/issues/2338#issuecomment-594461047

jsmeix commented at 2020-03-04 11:34:¶

@gozora
only as a nitpicking side note (not meant seriously) in your
https://github.com/rear/rear/issues/2338#issuecomment-594465632
the error message does not fully match the code ;-)

gozora commented at 2020-03-04 11:49:¶

Hello @jsmeix

Might be ;-), i didn't read the part after pipe ;-).

V.

jsmeix commented at 2020-03-04 11:52:¶

@gozora
you meant that double pipe || ( has much more throughput ;-)

jsmeix commented at 2020-03-04 11:53:¶

Where lib/grub is used in ReaR scripts:

# find usr/sbin/rear usr/share/rear/ -type f | xargs grep 'lib/grub'
usr/share/rear/build/OPALPBA/Linux-i386/095_exclude_non_essential_files.sh:COPY_AS_IS_EXCLUDE+=( /boot /etc/grub.d /etc/default/grub /usr/lib/grub /usr/share/grub )
usr/share/rear/prep/GNU/Linux/300_include_grub_tools.sh:             /usr/lib/grub* /usr/share/grub* /etc/tuned/* )
usr/share/rear/output/RAWDISK/Linux-i386/270_create_grub2_efi_bootloader.sh:[[ -d /usr/lib/grub/x86_64-efi ]] || return 0
usr/share/rear/lib/uefi-functions.sh:            if test /usr/lib/grub*/x86_64-efi/partmap.lst ; then
usr/share/rear/lib/uefi-functions.sh:                modules=( $(cat /usr/lib/grub*/x86_64-efi/partmap.lst) )
usr/share/rear/lib/uefi-functions.sh:    # e.g. on openSUSE Leap 15.0 it is in /usr/lib/grub2/x86_64-efi/moddep.lst

gozora commented at 2020-03-04 11:56:¶

@jsmeix

See? That's what I'm saying. To notice double pipe, I'd need to read behind first pipe :-).

V.

pcahyna commented at 2020-03-04 13:12:¶

@pcahyna
in general in ReaR we like to avoid the word WARNING, cf.
https://blog.schlomo.schapiro.org/2015/04/warning-is-waste-of-my-time.html
which originated here in ReaR at
#564 (comment)
and subsequent comments there.

I understand the reasoning behind it, but here the WARNING provides just an explanation of the following error, so I don't think the reasoning exactly applies.

In
#1193 (comment)
I wrote why I don't like explicit RPM package names.

Our patch was a downstream-specific patch to help with a Fedora and RHEL-specific issues, so it mentions the package name exactly. If we merge it upstream, I understand that you want to make the message more generic according to your reasoning, but the problem is, it is now not much less clear to the user what they should do, in my version it was obvious.

So better sample output could be something like
Found no /usr/*/grub*/x86_64-efi/moddep.lst file so grub2-mkimage may fail.
ERROR: grub2-mkimage of BOOTX64.efi failed
or perhaps even better
grub2-mkimage may fail (found no /usr/*/grub*/x86_64-efi/moddep.lst)
ERROR: grub2-mkimage of BOOTX64.efi failed
to have the most interesting part grub2-mkimage may fail
at the beginning of the line (and not as "some trailing technical detail").

Before #2013 was merged, the error message was:

grub2-mkimage: error: cannot open `/usr/lib/grub/x86_64-efi/moddep.lst': No
such file or directory.
2017-02-06 01:36:32 ERROR: Error occurred during grub2-mkimage of BOOTX64.efi

(issue #1193). I don't see much added value in the message that you propose relative to the original message. Is the issue just that the original message goes only to the log, while the added message gets printed to the terminal?

Furthermore regarding things like PrintIfError in
https://bugzilla.redhat.com/show_bug.cgi?id=1492177#c13
see
#2013 (comment)

Good point, I'll keep that in mind.

pcahyna commented at 2020-03-04 13:19:¶

Shouldn't there be a way to print stderr from a command in case the command fails, in addition to just redirecting it to the log? It would make debugging such failures much easier, without such custom tests that we are discussing here - because often the failed command provides the most accurate information on what went wrong.

jsmeix commented at 2020-03-04 13:36:¶

@pcahyna
in general see "What to do with stdin, stdout, and stderr" in
https://github.com/rear/rear/wiki/Coding-Style

I will think about how to show stderr from a command to the user
plus having that same stderr from the command also in the log.
My offhanded idea is to try out using the tee command
together with some appropriate redirections
e.g. something as in the section
"Generic disaster recovery with the plain SUSE installation system" in
https://en.opensuse.org/SDB:Disaster_Recovery

# Start logging:
my_name=${0##*/}
starting_timestamp=$( date +%Y%m%d%H%M%S )
log_file=$my_name.$starting_timestamp.log
# Have a new file descriptor 3 which is a copy of the stdout file descriptor:
exec 3>&1
# Have a new file descriptor 4 which is a copy of the stderr file descriptor:
exec 4>&2
# Have stdout on the terminal and also in the log file:
exec 1> >( exec -a $my_name:tee tee -a $log_file )
logging_tee_pid=$!
# Make stderr what stdout already is (i.e. terminal and also in the log file):
exec 2>&1

of course that must be done different in ReaR to work
with the existing default stdout/stderr redirections in ReaR.

jsmeix commented at 2020-03-04 14:08:¶

@pcahyna
I think what is shown in
https://github.com/rear/rear/issues/1193#issue-205881890

2017-02-06 01:36:32 Including output/ISO/Linux-i386/250_populate_efibootimg.sh
grub2-mkimage: error: cannot open `/usr/lib/grub/x86_64-efi/moddep.lst': No such file or directory.
2017-02-06 01:36:32 ERROR: Error occurred during grub2-mkimage of BOOTX64.efi

is from ReaR's log file and not what there is on the user's terminal
(I think on the termial there are no "date time" prefixed messages).

My primary intent is to show the user on his terminal
when there are errors instead of let things fail silently
and let the user search through the log file.

My intent is not to show explanations and reasons
or even tell what the user should do.

My intent is to only show the real hard facts
and leave it to the user what to do because

ReaR (at least in its current form) is only for experienced users (in particular for system admins)
personally I really hate it when computers tell me what I should do (i.e. I hate it when computers appear to behave patronizing) - a side-topic is that computers should not beg the user (i.e. no please) or talk to the user personally (e.g. no sorry and no congratulations) - just like technical documentation style that should not be in a personal tone (and never ever private: when you are desperate call our friendly hotline for help ;-)

For example I would do

type -a parted || Error "No parted command found ('type -a parted' failed)"

instead of

type -a parted || Error "Cannot setup disk partitions ('parted' needs to be installed)"

because the former code works correctly in all cases
while in the latter case there are zillions of possible weird reasons
why the error message text is wrong (e.g. when parted is installed
but for whatever awkward reasons type -a parted does not "find" it).
Or in other words:
In the latter case there are implicit hidden assumptions behind
the error message text - i.e. the error message text is the result of an
imagined interpretation by the programmer who wrote the code
at the time when he wrote the code that is limited by his imagination
what all the possible reasons could be why type -a parted failed.

jsmeix commented at 2020-03-04 15:31:¶

Hopefully the GRUB2 modules directory part
in lib/uefi-functions.sh is now fixed via
https://github.com/rear/rear/commit/de9391c01e9492144d50d0c4c8b60db36f653007

jsmeix commented at 2020-03-04 15:42:¶

Hopefully the GRUB2 directories part in
output/RAWDISK/Linux-i386/270_create_grub2_efi_bootloader.sh
is now sufficiently fixed via
https://github.com/rear/rear/commit/5c1e4f19e6cc78de6743289f82c0ae8076a1188c

jsmeix commented at 2020-03-04 15:47:¶

Hopefully the GRUB2 directories part in
build/OPALPBA/Linux-i386/095_exclude_non_essential_files.sh
is now sufficiently fixed via
https://github.com/rear/rear/commit/dff3f6a4c06dc7a75d5ef67ce9aa5fc978828727

jsmeix commented at 2020-03-04 15:50:¶

@OliverO2
FYI see
https://github.com/rear/rear/commit/5c1e4f19e6cc78de6743289f82c0ae8076a1188c
and
https://github.com/rear/rear/commit/dff3f6a4c06dc7a75d5ef67ce9aa5fc978828727
what I changed in your code to hopefully fix GRUB2 directories stuff
(at least for now until GRUB2 things move elsewhere again ;-)

jsmeix commented at 2020-03-04 16:00:¶

With my quick fixes here
https://github.com/rear/rear/commit/d7729b0baba907a66ccfd976ab133964ae4effbe
https://github.com/rear/rear/commit/de9391c01e9492144d50d0c4c8b60db36f653007
https://github.com/rear/rear/commit/5c1e4f19e6cc78de6743289f82c0ae8076a1188c
https://github.com/rear/rear/commit/dff3f6a4c06dc7a75d5ef67ce9aa5fc978828727
I hope the GRUB2 directories stuff is sufficiently fixed for now.

Of course for the future - as time permits - a more generic
and more fail-safe solution would be appreciated, cf.
https://github.com/rear/rear/commit/fe159d11493f447eda8a8feccf050f7f46ad4b84

For the fun of it regadring "as time permits":
I stumbled across this mess today by accident
while I was working on something totally different
so I will try to do that tomorrow - as time permits ;-)

OliverO2 commented at 2020-03-05 16:11:¶

@jsmeix

FYI see
5c1e4f1
and
dff3f6a
what I changed in your code to hopefully fix GRUB2 directories stuff
(at least for now until GRUB2 things move elsewhere again ;-)

Thanks for keeping me posted, your changes look reasonable to me.

I'm a bit puzzled what happend: According to the FHS /usr/share is supposed to contain architecture-independent (shared) data only. x86_64-efi sounds rather architecture-dependent. What's the reasoning in relocating that stuff in openSUSE Leap 15.1?

jsmeix commented at 2020-03-06 10:43:¶

@OliverO2
I am afraid I don't know the actual reason behind why on SUSE
GRUB2 is built 'noarch' with things moved to '/usr/share'.

All what I know is basically
https://github.com/rear/rear/issues/2338#issuecomment-594432946

I had a look at the entries of our FATE#326960
(FATE is the SUSE internal feature tracking tool)
but those entries do not show a reason behind.
This is because usually FATE is not used to discuss
features - this happens usually by the involved parties
outside of FATE - so FATE shows usually only administrative
steps like who must do what until when and things like that.

In the end it does not matter for ReaR if such things make sense.
In the end what matters is that the Relax-and-Recover code works
cf. "Dirty hacks welcome" in
https://github.com/rear/rear/wiki/Coding-Style
and see RFC 1925 item 1
https://tools.ietf.org/html/rfc1925

jsmeix commented at 2020-03-06 14:10:¶

I consider it sufficiently "fixed" for now via my above listed commits.

For the future a more generic and more fail-safe solution would be appreciated.

OliverO2 commented at 2020-03-06 14:15:¶

@jsmeix Thanks for taking care of it and for looking behind the scenes!

[Export of Github issue for rear/rear.]

#2338 Issue closed: Testing only /usr/lib/grub*/x86_64-efi/moddep.lst is insufficient - could be also in /boot/grub*/ or in /usr/share/grub*/¶

jsmeix opened issue at 2020-03-04 09:58:¶

pcahyna commented at 2020-03-04 10:04:¶

jsmeix commented at 2020-03-04 10:09:¶

jsmeix commented at 2020-03-04 10:16:¶

jsmeix commented at 2020-03-04 10:20:¶

pcahyna commented at 2020-03-04 10:29:¶

jsmeix commented at 2020-03-04 10:33:¶

pcahyna commented at 2020-03-04 10:37:¶

jsmeix commented at 2020-03-04 10:47:¶

gozora commented at 2020-03-04 10:52:¶

pcahyna commented at 2020-03-04 10:56:¶

pcahyna commented at 2020-03-04 11:00:¶

pcahyna commented at 2020-03-04 11:02:¶

jsmeix commented at 2020-03-04 11:11:¶

gozora commented at 2020-03-04 11:22:¶

jsmeix commented at 2020-03-04 11:25:¶

jsmeix commented at 2020-03-04 11:32:¶

jsmeix commented at 2020-03-04 11:34:¶

gozora commented at 2020-03-04 11:49:¶

jsmeix commented at 2020-03-04 11:52:¶

jsmeix commented at 2020-03-04 11:53:¶

gozora commented at 2020-03-04 11:56:¶

pcahyna commented at 2020-03-04 13:12:¶

pcahyna commented at 2020-03-04 13:19:¶

jsmeix commented at 2020-03-04 13:36:¶

jsmeix commented at 2020-03-04 14:08:¶

jsmeix commented at 2020-03-04 15:31:¶

jsmeix commented at 2020-03-04 15:42:¶

jsmeix commented at 2020-03-04 15:47:¶

jsmeix commented at 2020-03-04 15:50:¶

jsmeix commented at 2020-03-04 16:00:¶

OliverO2 commented at 2020-03-05 16:11:¶

jsmeix commented at 2020-03-06 10:43:¶

jsmeix commented at 2020-03-06 14:10:¶

OliverO2 commented at 2020-03-06 14:15:¶

#2338 Issue `closed`: Testing only /usr/lib/grub/x86_64-efi/moddep.lst is insufficient - could be also in /boot/grub/ or in /usr/share/grub*/¶