#2241 Issue `closed`: Backup restore fails for BACKUP_PROG_COMPRESS_OPTIONS=("--zstd") due to missing zstd binaries in recovery system (probably also for --lzip --lzma --lzop)¶

Labels: enhancement, documentation, needs sponsorship, fixed / solved / done

aasami opened issue at 2019-09-27 13:59:¶

Relax-and-Recover (ReaR) Issue Template¶

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.5 / Git
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):

$ lsb_release -a
LSB Version:    n/a
Distributor ID: ManjaroLinux
Description:    Manjaro Linux
Release:        18.1.0
Codename:       Juhraya

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

$ cat /etc/rear/site.conf
TIMESYNC=NTP

$ cat /etc/rear/local.conf
OUTPUT=RAWDISK
OUTPUT_URL=nfs://nfs.tu/srv/bkp
BACKUP=NETFS
BACKUP_URL=nfs://nfs.tu/srv/bkp
BACKUP_TYPE=differential
FULLBACKUP_OUTDATED_DAYS=92
BACKUP_PROG_COMPRESS_OPTIONS=("--zstd")
BACKUP_PROG_COMPRESS_SUFFIX=".zst"
KERNEL_FILE="/boot/vmlinuz-$( uname -r|cut -d\. -f1-2 )-x86_64"

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):

$ dmidecode -t1
\# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.8 present.

Handle 0x000F, DMI type 1, 27 bytes
System Information
        Manufacturer: LENOVO
        Product Name: 20ET004BXS
        Version: ThinkPad E460
        Serial Number: PF0ISVA2
        UUID: 37a370cc-2181-11b2-a85c-e2db9b721f93
        Wake-up Type: Power Switch
        SKU Number: LENOVO_MT_20ET_BU_Think_FM_ThinkPad E460
        Family: ThinkPad E460

System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):

$ uname -m
x86_64

Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
UEFI + GRUB
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
local SSD
Description of the issue (ideally so that others can reproduce it):
When using option BACKUP_PROG_COMPRESS_OPTIONS=("--zstd")
recovery is not possible due to missing zstd binaries in recovery image.
(The same might be true for options --lzip --lzma and --lzop but I haven't tested it)
Workaround, if any:
Possible: copy missing binary from other location when in recovery mode (not tested)
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

jsmeix commented at 2019-09-27 14:56:¶

@aasami
there is currently no automatism in ReaR that would automatically add
what is needed by tar for special BACKUP_PROG_COMPRESS_OPTIONS
settings into the ReaR recovery system.

When you know what binaries and libraries and other files
are needed by tar when using zstd compression
you can add them into the ReaR recovery system
via generic ReaR functionality using config variables like
COPY_AS_IS, REQUIRED_PROGS, and LIBS
see the default.conf description
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf

After "rear mkrescue/mkbackup" you can check things inside the
ReaR recovery system by using KEEP_BUILD_DIR="yes",
see the KEEP_BILD_DIR description in default.conf
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L128
via chroot $TMPDIR/rear.XXXXXXXXXXXXXXX/rootfs/
and then try to run tar with the options you like
to test things inside the ReaR recovery system
so that you can check more easily and directly
if all what is needed to run tar as you want it
is included in the ReaR recovery system.

Regarding how BACKUP_PROG_COMPRESS_OPTIONS
is actually used for the tar command see the scripts
usr/share/rear/backup/NETFS/default/500_make_backup.sh
https://github.com/rear/rear/blob/master/usr/share/rear/backup/NETFS/default/500_make_backup.sh
and
usr/share/rear/restore/NETFS/default/400_restore_backup.sh
https://github.com/rear/rear/blob/master/usr/share/rear/restore/NETFS/default/400_restore_backup.sh

gdha commented at 2019-10-05 09:55:¶

@aasami Will you make PR when you got it working?

jsmeix commented at 2019-10-16 12:44:¶

Via
https://github.com/rear/rear/commit/9e07846b460976084e2bb2bdec01b901c42ec413
I added explanatory comments to default.conf
about backup restore that may fail for things like
BACKUP_PROG_COMPRESS_OPTIONS=("--zstd")
due to missing zstd binaries in recovery system
(probably also for --lzip --lzma --lzop).

Accordingly this issue is now at least documented
and because I do not plan to implement an automatism in ReaR
that would automatically include additional things
that are needed by tar into the ReaR recovery system
I close this issue hereby and set the 'needs sponsorship' label.

[Export of Github issue for rear/rear.]