#2508 Issue `closed`: ERROR: Cannot autodetect what is used as bootloader plus LD_LIBRARY_PATH issue with BACKUP=TSM¶

Labels: support / question, fixed / solved / done, external tool

dcz01 opened issue at 2020-11-05 09:45:¶

ERROR: Cannot autodetect what is used as bootloader¶

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.6 / 2020-06-17
OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):

NAME="CentOS Linux"
VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="8"

CentOS Linux release 8.2.2004 (Core)

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

OUTPUT=ISO
OUTPUT_URL=file:///tmp/rear
#BACKUP=NETFS
BACKUP=TSM
#BACKUP_PROG=tar
#BACKUP_PROG_CRYPT_ENABLED=1
#BACKUP_PROG_CRYPT_KEY=<Verschluesselungskennwort>
#BACKUP_PROG_CRYPT_OPTIONS="/usr/bin/openssl aes256 -salt -k"
#BACKUP_PROG_DECRYPT_OPTIONS="/usr/bin/openssl aes256 -d -k"
#BACKUP_URL=nfs://<IP-Adresse oder DNS-Name>/<Freigabepfad>
#BACKUP_URL=cifs://<Server>/<Freigabe>
#BACKUP_OPTIONS="cred=/etc/rear/cifs,sec=ntlmsspi"
#BACKUP_TYPE=incremental
#FULLBACKUPDAY="Sat"
#BACKUP_PROG_EXCLUDE=( '/tmp/*' '/dev/shm/*' $VAR_DIR/output/\* '/opt/tivoli/tsm/rear/*' '/mnt/*' '/media/*' '/var/lib/pgsql/*/data/base/*' '/var/lib/pgsql/*/data/global/*' '/var/lib/pgsql/*/data/pg*/*' )
SSH_ROOT_PASSWORD='$1$HGjk3XUV$lid3Nd3k01Kht1mpMscLw1'
NON_FATAL_BINARIES_WITH_MISSING_LIBRARY='/opt/tivoli/tsm/client/ba/bin/libvixMntapi.so.1.1.0'

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
Virtual Machine on VMware ESXi
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
UEFI with GRUB2
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
Local Disk
Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT" or "lsblk" as makeshift):

NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0  200G  0 disk
├─sda1   8:1    0  200M  0 part /boot/efi
├─sda2   8:2    0  500M  0 part /boot
├─sda3   8:3    0 97,7G  0 part /
├─sda4   8:4    0  7,9G  0 part [SWAP]
├─sda5   8:5    0    1M  0 part
└─sda6   8:6    0 93,8G  0 part /var/lib/pgsql
sr0     11:0    1  624M  0 rom

Description of the issue (ideally so that others can reproduce it):

[root@fbd01pss ~]# rear mkrescue -d
Relax-and-Recover 2.6 / 2020-06-17
Running rear mkrescue (PID 108953)
Using log file: /var/log/rear/rear-fbd01pss.log
Running workflow mkrescue on the normal/original system
Found EFI system partition /dev/sda1 on /boot/efi type vfat
Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)
Using autodetected kernel '/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
ERROR: Cannot autodetect what is used as bootloader, see default.conf about 'BOOTLOADER'
Some latest log messages since the last called script 445_guess_bootloader.sh:
  2020-11-05 10:29:04.880497379 Including layout/save/default/445_guess_bootloader.sh
  4+0 records in
  4+0 records out
  2048 bytes (2.0 kB, 2.0 KiB) copied, 7.9054e-05 s, 25.9 MB/s
Aborting due to an error, check /var/log/rear/rear-fbd01pss.log for details
Exiting rear mkrescue (PID 108953) and its descendant processes ...
Running exit tasks
You should also rm -Rf /tmp/rear.VyWhUaHu9A1hmmx
Beendet



[root@fbd01pss ~]# ls /boot/efi/EFI/
BOOT  centos
[root@fbd01pss ~]# ls /boot/efi/EFI/BOOT/
BOOTX64.EFI  fbx64.efi
[root@fbd01pss ~]# ls /boot/efi/EFI/centos/
BOOTX64.CSV  fonts  grub.cfg  grubenv  grubx64.efi  mmx64.efi  shimx64-centos.efi  shimx64.efi
[root@fbd01pss ~]# ls /boot/efi/
EFI

Workaround, if any:
None
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
rear-fbd01pss.log

dcz01 commented at 2020-11-06 14:30:¶

@jsmeix Sorry for providing the false log without the deeper debuging information.
Here is the complete log with debug option: rear-fbd01pss.log

jsmeix commented at 2020-11-06 15:22:¶

@dcz01
thank you - I will have a look next week (as time permits).

In the meantime you could play around with
the BOOTLOADER config variable, cf.

ERROR: Cannot autodetect what is used as bootloader,
see default.conf about 'BOOTLOADER'

In general when a program fails with an error message
it means that the reason that lead to the error message
is a known case that is handled within the program
so it is normally not a bug in the program because
the program works as intended in this particular case.

Have a relaxing and recovering weekend!

dcz01 commented at 2020-11-09 08:32:¶

@jsmeix Thanks, for your information. I hope you had an nice weekend too.
I tested it again with some new options in my config file:

OUTPUT=ISO
OUTPUT_URL=file:///tmp/rear
#BACKUP=NETFS
BACKUP=TSM
#BACKUP_PROG=tar
#BACKUP_PROG_CRYPT_ENABLED=1
#BACKUP_PROG_CRYPT_KEY=<Verschluesselungskennwort>
#BACKUP_PROG_CRYPT_OPTIONS="/usr/bin/openssl aes256 -salt -k"
#BACKUP_PROG_DECRYPT_OPTIONS="/usr/bin/openssl aes256 -d -k"
#BACKUP_URL=nfs://<IP-Adresse oder DNS-Name>/<Freigabepfad>
#BACKUP_URL=cifs://<Server>/<Freigabe>
#BACKUP_OPTIONS="cred=/etc/rear/cifs,sec=ntlmsspi"
#BACKUP_TYPE=incremental
#FULLBACKUPDAY="Sat"
#BACKUP_PROG_EXCLUDE=( '/tmp/*' '/dev/shm/*' $VAR_DIR/output/\* '/opt/tivoli/tsm/rear/*' '/mnt/*' '/media/*' '/var/lib/pgsql/*/data/base/*' '/var/lib/pgsql/*/data/global/*' '/var/lib/pgsql/*/data/pg*/*' )
SSH_ROOT_PASSWORD='$1$HGjk3XUV$lid3Nd3k01Kht1mpMscLw1'
NON_FATAL_BINARIES_WITH_MISSING_LIBRARY='/opt/tivoli/tsm/client/ba/bin/libvixMntapi.so.1.1.0'
BOOTLOADER="GRUB2-EFI"
UEFI_BOOTLOADER="/boot/efi/EFI/centos/grubx64.efi"

And then i got an new error message:

ReaR recovery system in '/tmp/rear.0Tsyc8vA3jupI3s/rootfs' needs additional libraries, check /var/log/rear/rear-fbd01pss.log for details
Build area kept for investigation in /tmp/rear.0Tsyc8vA3jupI3s, remove it when not needed
ERROR: ReaR recovery system in '/tmp/rear.0Tsyc8vA3jupI3s/rootfs' not usable (required libraries are missing)
Some latest log messages since the last called script 990_verify_rootfs.sh:
  partprobe is /bin/partprobe
  wipefs is /bin/wipefs
  mkfs is /bin/mkfs
  mkfs.vfat is /bin/mkfs.vfat
  mkfs.xfs is /bin/mkfs.xfs
  xfs_admin is /bin/xfs_admin
  mkswap is /bin/mkswap
  ldconfig is /bin/ldconfig
Aborting due to an error, check /var/log/rear/rear-fbd01pss.log for details
Exiting rear mkrescue (PID 437609) and its descendant processes ...
Running exit tasks
You should also rm -Rf /tmp/rear.0Tsyc8vA3jupI3s
Beendet

Also i got an new log with debug option on for you.
rear-fbd01pss.log

jsmeix commented at 2020-11-12 08:09:¶

@dcz01
your https://github.com/rear/rear/files/5509035/rear-fbd01pss.log
shows that you should have got those messages on your terminal
wherefrom "rear -D mkrescue" was run (excerpts):

Testing that the recovery system in /tmp/rear.0Tsyc8vA3jupI3s/rootfs contains a usable system
There are binaries or libraries in the ReaR recovery system that need additional libraries
/bin/gdisk requires additional libraries (fatal error)

and with that inspecting the log file about /bin/gdisk shows (excerpts):

++ chroot /tmp/rear.0Tsyc8vA3jupI3s/rootfs /bin/bash --login -c 'cd /bin && ldd /bin/gdisk'
++ broken_binaries=' /bin/gdisk'
...
+++ chroot /tmp/rear.0Tsyc8vA3jupI3s/rootfs /bin/ldd /bin/gdisk
++ ldd_output=' linux-vdso.so.1 (0x00007ffc0fa94000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007fa1709f4000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fa17065f000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fa1702dd000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa1700c5000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fa16fd03000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa170e2f000)'

and the matching code in usr/share/rear/build/default/990_verify_rootfs.sh
is (excerpts):

    chroot $ROOTFS_DIR /bin/bash --login -c "cd $( dirname $binary ) && ldd $binary" </dev/null 2>/dev/null | grep -q 'not found' && broken_binaries="$broken_binaries $binary"
done
# Restore the LD_LIBRARY_PATH if it was saved above (i.e. when LD_LIBRARY_PATH had been set before)
# otherwise unset a possibly set LD_LIBRARY_PATH (i.e. when LD_LIBRARY_PATH had not been set before):
test $old_LD_LIBRARY_PATH && export LD_LIBRARY_PATH=$old_LD_LIBRARY_PATH || unset LD_LIBRARY_PATH

# Report binaries with 'not found' shared object dependencies:
...
        # Run the same ldd call as above but now keep its whole output:
        ldd_output="$( chroot $ROOTFS_DIR /bin/ldd $binary )"
        # Have the whole ldd output only in the log:
        Log "$ldd_output"
        # Show only the missing libraries to the user to not flood his screen with tons of other ldd output lines:
        PrintError "$( grep 'not found' <<<"$ldd_output" )"

This looks inexplicable on first glance
because the first ldd /bin/gdisk results a 'not found' library
while the second ldd /bin/gdisk does no longer result a 'not found' library.

The only explanation I have is that the second ldd /bin/gdisk
is no longer run with the LD_LIBRARY_PATH setting that was
used for the first ldd /bin/gdisk.

So my conclusions are:

1.)
Your LD_LIBRARY_PATH setting that is needed
for your particular third party backup tool
somehow messes up something with system libraries so that
the first ldd /bin/gdisk with the LD_LIBRARY_PATH setting
results a 'not found' library.

2.)
The second ldd /bin/gdisk should also be run with the LD_LIBRARY_PATH setting
so that results are the same for both ldd runs.

jsmeix commented at 2020-11-12 08:23:¶

@dcz01
to verify if my conclusion is right, manually change your
usr/share/rear/build/default/990_verify_rootfs.sh
as in
https://github.com/rear/rear/pull/2515/files
i.e. remove the

# Restore the LD_LIBRARY_PATH if it was saved above (i.e. when LD_LIBRARY_PATH had been set before)
# otherwise unset a possibly set LD_LIBRARY_PATH (i.e. when LD_LIBRARY_PATH had not been set before):
test $old_LD_LIBRARY_PATH && export LD_LIBRARY_PATH=$old_LD_LIBRARY_PATH || unset LD_LIBRARY_PATH

part from where it currently is
and copy it directly below the

# Report binaries with 'not found' shared object dependencies:
local fatal_missing_library=""
if contains_visible_char "$broken_binaries" ; then
    ...
fi

part so that in the end that part becomes

# Report binaries with 'not found' shared object dependencies:
local fatal_missing_library=""
if contains_visible_char "$broken_binaries" ; then
    ...
fi
# Restore the LD_LIBRARY_PATH if it was saved above (i.e. when LD_LIBRARY_PATH had been set before)
# otherwise unset a possibly set LD_LIBRARY_PATH (i.e. when LD_LIBRARY_PATH had not been set before):
test $old_LD_LIBRARY_PATH && export LD_LIBRARY_PATH=$old_LD_LIBRARY_PATH || unset LD_LIBRARY_PATH

Then re-run "rear -D mkrescue" and attach the new log file.

dcz01 commented at 2020-11-12 13:08:¶

@jsmeix Thanks for your response and the investigation of the code.
I changed the file like you wrote but i got the same error again.
Here is the new log: rear-FBD01PSS.log

jsmeix commented at 2020-11-12 14:17:¶

@dcz01
I wrote

remove the

# Restore the LD_LIBRARY_PATH if it was saved above (i.e. when LD_LIBRARY_PATH had been set before)
# otherwise unset a possibly set LD_LIBRARY_PATH (i.e. when LD_LIBRARY_PATH had not been set before):
test $old_LD_LIBRARY_PATH && export LD_LIBRARY_PATH=$old_LD_LIBRARY_PATH || unset LD_LIBRARY_PATH

part from where it currently is

So your usr/share/rear/build/default/990_verify_rootfs.sh
must be exactly as this one
https://raw.githubusercontent.com/rear/rear/1f409f730d08511fe3f0b6f012f3709806c782c4/usr/share/rear/build/default/990_verify_rootfs.sh
from my https://github.com/rear/rear/pull/2515

This new usr/share/rear/build/default/990_verify_rootfs.sh
will not and cannot fix possible mess with LD_LIBRARY_PATH
but this new usr/share/rear/build/default/990_verify_rootfs.sh
will show the 'ldd' output where things had failed with LD_LIBRARY_PATH set
to show a useful message that should indicate where the root cause could be.

dcz01 commented at 2020-11-12 14:29:¶

@jsmeix Sorry for that, i just commented it out.
But now i cleaned the whole file and pasted your raw output into that file.
Here is the newer log file: rear-FBD01PSS.log

jsmeix commented at 2020-11-12 15:12:¶

@dcz01
now you should have got those messages on your terminal
wherefrom "rear -D mkrescue" was run (excerpts):

Testing that the recovery system in /tmp/rear.xxYYlwefbpPeA96/rootfs contains a usable system
There are binaries or libraries in the ReaR recovery system that need additional libraries
/bin/gdisk requires additional libraries (fatal error)
/bin/gdisk: /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6: version `GLIBCXX_3.4.20'\'' not found (required by /bin/gdisk)
/bin/gdisk: /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6: version `CXXABI_1.3.9'\'' not found (required by /bin/gdisk)
/bin/gdisk: /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6: version `GLIBCXX_3.4.21'\'' not found (required by /bin/gdisk)

and in your https://github.com/rear/rear/files/5531207/rear-FBD01PSS.log
the matching part is

2020-11-12 15:27:22.499599373 /bin/gdisk: /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /bin/gdisk)
/bin/gdisk: /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /bin/gdisk)
/bin/gdisk: /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /bin/gdisk)
        linux-vdso.so.1 (0x00007ffc419b1000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f6d1eea1000)
        libstdc++.so.6 => /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6 (0x00007f6d1eb9e000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f6d1e81c000)
        libgcc_s.so.1 => /opt/tivoli/tsm/client/ba/bin/libgcc_s.so.1 (0x00007f6d1e606000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f6d1e244000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6d1f2dc000)

I cannot tell what could be wrong with your system why it seems
your gdisk needs /opt/tivoli/tsm/client/ba/bin/libstdc++.so.6

On my openSUSE leap 15.1 system gdisk needs this:

# ldd /usr/sbin/gdisk
        linux-vdso.so.1 (0x00007ffebcadd000)
        libuuid.so.1 => /usr/lib64/libuuid.so.1 (0x00007fcad6938000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fcad655e000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fcad6226000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fcad600d000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fcad5c52000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fcad6d6f000)

In ReaR you can use NON_FATAL_BINARIES_WITH_MISSING_LIBRARY
to ignore that error exit in ReaR when you know it is only false alarm, cf.
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L1620

But to be more on the safe side I would recommend to find out
if your shared object dependencies perhaps got somehow messed up
by that LD_LIBRARY_PATH setting.

As I said in
https://github.com/rear/rear/issues/2508#issuecomment-725914211

Your LD_LIBRARY_PATH setting that is needed
for your particular third party backup tool
somehow messes up something with system libraries so that
the first ldd /bin/gdisk with the LD_LIBRARY_PATH setting
results a 'not found' library.

So to be on the safe side I suggest to talk to the provider of your third party backup tool
perhaps even together with CentOS Linux people what you could do about it.

jsmeix commented at 2020-11-12 15:23:¶

I close this issue because it depends on special environment settings
(LD_LIBRARY_PATH) for an external tool (a third party backup tool)
that we at reaR upstream do not have so we cannot reproduce anything
and ReaR provides a workaround to ignore such issues in ReaR.

@dcz01
nevertheless I would appreciate feedback from you
if you find out what the root cause is
and how you could solve it.

jsmeix commented at 2020-11-13 11:25:¶

@dcz01
perhaps something is wrong how build/default/990_verify_rootfs.sh calls 'ldd'
because currently it calls 'ldd' with a special LD_LIBRARY_PATH setting
for all binaries in the recovery system when third-party backup tools are used
that need to run with a special LD_LIBRARY_PATH setting.

But I think in case of third-party backup tools that need to run
with a special LD_LIBRARY_PATH setting
only for the third-party backup tool binaries in the recovery system
'ldd' should be called with the special LD_LIBRARY_PATH setting
but for all the other "normal" binaries in the recovery system
'ldd' should be called without the special LD_LIBRARY_PATH setting.

I will have a closer look next week (as time permits).
Have a relaxing and recovering weekend!

dcz01 commented at 2020-11-17 08:19:¶

@jsmeix Hi Johannes, i finally found the solution of the problem with the missing library.
And now my older case https://github.com/rear/rear/issues/1533 is solved too without the need of the option "NON_FATAL_BINARIES_WITH_MISSING_LIBRARY".
I only added a new file /etc/ld.so.conf.d/TIVsm.conf with that into:

/lib64/
/opt/tivoli/tsm/client/ba/bin/

Later i activated that config with ldconfig.

And then my ldd output looks fine:

[root@FBD01PSS ~]# ldd /opt/tivoli/tsm/client/ba/bin/libvixMntapi.so
        linux-vdso.so.1 (0x00007fff089f3000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f0153d91000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0153b71000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f0153948000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f0153731000)
        libvixDiskLib.so.6 => /opt/tivoli/tsm/client/ba/bin/libvixDiskLib.so.6 (0x00007f015328a000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f0152ef5000)
        libfuse.so.2 => /lib64/libfuse.so.2 (0x00007f0152cb6000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f0152aad000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f0152895000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f01524d3000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f015447e000)
        libcurl.so.4 => /lib64/libcurl.so.4 (0x00007f0152245000)
        libsqlite3.so.0 => /lib64/libsqlite3.so.0 (0x00007f0151f32000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f0151bb0000)
        libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007f0151989000)
        libidn2.so.0 => /lib64/libidn2.so.0 (0x00007f015176b000)
        libssh.so.4 => /lib64/libssh.so.4 (0x00007f01514e0000)
        libpsl.so.5 => /lib64/libpsl.so.5 (0x00007f01512cf000)
        libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007f015103b000)
        libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007f0150b58000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f0150908000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f0150618000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f01503fc000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f01501f8000)
        libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f014ffab000)
        liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f014fd9b000)
        libbrotlidec.so.1 => /lib64/libbrotlidec.so.1 (0x00007f014fb8f000)
        libunistring.so.2 => /lib64/libunistring.so.2 (0x00007f014f80e000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f014f5fd000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f014f3f9000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f014f1e2000)
        libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f014efc4000)
        libbrotlicommon.so.1 => /lib64/libbrotlicommon.so.1 (0x00007f014eda4000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f014eb79000)
        libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f014e8f5000)

But now i get still an error with missing librarys and none is listed.
Look at the new log.
rear-FBD01PSS.log

dcz01 commented at 2020-11-17 13:02:¶

@jsmeix I also received an answer from the IBM Software Support for my ticket with this library.
It is part of the TDP for VMware package.
But we could also only link the library to /lib64 like:
ln -s /opt/tivoli/tsm/client/ba/bin/libvixDiskLib.so.6 /lib64/libvixDiskLib.so.6

If you would like to test with the client package you can download it here for free (there is no need for licensing if you haven't any TSM server): http://public.dhe.ibm.com/storage/tivoli-storage-management/patches/client/v7r1/Linux/LinuxX86/BA/v718/7.1.8.9-TIV-TSMBAC-LinuxX86.tar

jsmeix commented at 2020-11-18 08:54:¶

@dcz01
thank you for your feedback.
I will have a look when time permits.
Currently I am too busy with https://github.com/rear/rear/pull/2514
which is rather complicated so that I have no sufficiently free mind
to also investigate this issue which is also rather complicated.
Better carefully one at a time than several botched in a hurry ;-)

jsmeix commented at 2020-11-23 13:51:¶

@dcz01
could you try out if things behave better for your particular case with
https://github.com/rear/rear/pull/2523

I would suggest to copy the whole
https://raw.githubusercontent.com/rear/rear/a3d0bad726ac76f8dfe15a10f6de73c2c136dc07/usr/share/rear/build/default/990_verify_rootfs.sh
over your currenly existing usr/share/rear/build/default/990_verify_rootfs.sh
and provide feedback how "rear -v mkrescue" behaves for you with the new one.

dcz01 commented at 2020-11-23 15:21:¶

@jsmeix Thanks for your help and the optimization of the code.
I changed the 990_verify_rootfs.sh and tested it with your "rear -v mkrescue".
But i now get another error and the librarys error is gone...

[root@FBD01PSS ~]# rear -v mkrescue
Relax-and-Recover 2.6-git.0.10e049b.unknown.changed / 2020-06-17
Running rear mkrescue (PID 532551)
Using log file: /var/log/rear/rear-FBD01PSS.log
Running workflow mkrescue on the normal/original system
Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)
Using autodetected kernel '/boot/vmlinuz-4.18.0-193.28.1.el8_2.x86_64' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
Using specified bootloader 'GRUB2-EFI'
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct ...
Creating recovery system root filesystem skeleton layout
Adding biosdevname=0 to KERNEL_CMDLINE
Adding net.ifnames=0 to KERNEL_CMDLINE
Using '/boot/efi/EFI/centos/grubx64.efi' as UEFI bootloader file
Copying logfile /var/log/rear/rear-FBD01PSS.log into initramfs as '/tmp/rear-FBD01PSS-partial-2020-11-23T16:08:30+01:00.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/4.18.0-193.28.1.el8_2.x86_64 (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
Broken symlink '/etc/grub2.cfg' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/modules/4.18.0-193.28.1.el8_2.x86_64/build' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/modules/4.18.0-193.28.1.el8_2.x86_64/source' in recovery system because 'readlink' cannot determine its link target
Testing that the recovery system in /tmp/rear.gYWaSSaJMarZWWg/rootfs contains a usable system
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (543328145 bytes) in 80 seconds
grub2-mkstandalone may fail to make a bootable EFI image of GRUB2 (no /usr/*/grub*/x86_64-efi/moddep.lst file)
GRUB2 modules to load: fat part_gpt xfs
ERROR: Failed to make bootable EFI image of GRUB2 (error during grub2-mkstandalone of /tmp/rear.gYWaSSaJMarZWWg/tmp/mnt/EFI/BOOT/BOOTX64.efi)
Some latest log messages since the last called script 250_populate_efibootimg.sh:
  mkdir: created directory '/tmp/rear.gYWaSSaJMarZWWg/tmp/mnt/EFI/BOOT/fonts'
  mkdir: created directory '/tmp/rear.gYWaSSaJMarZWWg/tmp/mnt/EFI/BOOT/locale'
  '/boot/efi/EFI/centos/grubx64.efi' -> '/tmp/rear.gYWaSSaJMarZWWg/tmp/mnt/EFI/BOOT/BOOTX64.efi'
  /usr/share/rear/lib/_input-output-functions.sh: line 476: type: grub-mkstandalone: not found
  /usr/share/rear/lib/_input-output-functions.sh: line 476: type: grub-probe: not found
  2020-11-23 16:10:50.300123883 grub2-mkstandalone may fail to make a bootable EFI image of GRUB2 (no /usr/*/grub*/x86_64-efi/moddep.lst file)
  2020-11-23 16:10:50.303031967 GRUB2 modules to load: fat part_gpt xfs
  grub2-mkstandalone: error: /usr/lib/grub/x86_64-efi/modinfo.sh doesn't exist. Please specify --target or --directory.
Aborting due to an error, check /var/log/rear/rear-FBD01PSS.log for details
Exiting rear mkrescue (PID 532551) and its descendant processes ...
Running exit tasks
Beendet

And i attached a new debug log file.
rear-FBD01PSS.log

jsmeix commented at 2020-11-24 11:56:¶

@dcz01
thank you for testing https://github.com/rear/rear/pull/2523
and your prompt feedback.

Please submit a new separated GitHub issue for your latest
grub2-mkstandalone: error: /usr/lib/grub/x86_64-efi/modinfo.sh doesn't exist
with your CentOS 8 system with UEFI and GRUB2 as bootloader and
re-attach your debug log file to that new issue so that it is directly available there.

I cannot help with CentOS bootloader issues because I do not use it
so I can neither reproduce nor help with such issues on CentOS.
Generic issues like the ldd testing are different but bootloader issues
often depend very much on how things are on a particular Linux distribution.

Therefore a separated issue makes it possible that others who know about CentOS
and in particular its bootloader specific things could notice your new issue.

dcz01 commented at 2020-11-24 12:41:¶

@jsmeix Well then thanks for all your help and the new and better code.
I will open a new issue for that bootloader problem.

jsmeix commented at 2020-11-26 12:39:¶

With https://github.com/rear/rear/pull/2523 merged
the LD_LIBRARY_PATH issue (here in particular with BACKUP=TSM)
with special LD_LIBRARY_PATH setting for third-party backup tools
should be fixed.

[Export of Github issue for rear/rear.]

#2508 Issue closed: ERROR: Cannot autodetect what is used as bootloader plus LD_LIBRARY_PATH issue with BACKUP=TSM¶