Release Notes for Relax-and-Recover version 2.7

This document contains the release notes for the open source project Relax-and-Recover.

Relax-and-Recover website

GitHub project

Product Features

The following features are supported by the most recent releases of Relax-and-Recover. Anything labeled as (New) was added as the most recent release. New functionality for previous releases can be seen in the next chapter that details each release.

The most recent release of Relax-and-Recover is supported on most GNU/Linux based systems with kernel 2.6 or higher. It provides the following functionality:

  • Hot maintenance capability. A recovery/rescue image can be made online while the system is running

  • Command line interface. Relax-and-Recover does not require a graphical interface to run, neither in creation mode, nor in recovery mode (console is enough)

  • Support included for most common file systems, such as ext2, ext3, and ext4. Other filesystems like reiserfs, jfs, xfs, and btrfs are also implemented, but are less tested. (Feedback is appreciated)

  • Selected Hardware RAID and (eg. HP SmartArray) and mirroring solutions (eg. DRBD) are supported

  • NVME and mmcblk disks are supported

  • LVM root volumes are supported

  • Multipath support for SAN storage

  • UEFI support (including UEFI USB booting)

  • Integrates with internal backup programs such as:

  • GNU tar (BACKUP=NETFS, BACKUP_PROG=tar)

  • GNU tar (BACKUP=NETFS, BACKUP_PROG=tar, BACKUP_TYPE=incremental, FULLBACKUPDAY="Mon") for using incremental backups with a weekly full backup. Be aware, old tar archives will not be removed automatically!
  • GNU tar (BACKUP=NETFS, BACKUP_PROG=tar, BACKUP_TYPE=differential, FULLBACKUPDAY="Mon") for using differential backups with a weekly full backup. Be aware, old tar archives will not be removed automatically!
  • GNU tar with openssl encryption (BACKUP=NETFS, BACKUP_PROG=tar, BACKUP_PROG_CRYPT_ENABLED=1)
  • rsync on local devices (BACKUP=NETFS, BACKUP_PROG=rsync), such USB and local disks
  • rsync over the network (BACKUP=RSYNC, BACKUP_PROG=rsync)
  • Multiple backup methods (read the documentation)
  • Any partition (e.g. a Windows partition) via BACKUP=BLOCKCLONE. See the documention about BLOCKCLONE
  • BACKUP=ZYPPER is SLES12 only (Experimental)

  • BACKUP=YUM is for RedHat architectures ony (Experimental)

  • Integrates with external backup solutions such as:

  • Tivoli Storage Manager (BACKUP=TSM)

  • Data Protector (BACKUP=DP)
  • Symantec NetBackup (BACKUP=NBU)
  • Galaxy 5, 6, and 7 (BACKUP=GALAXY)
  • Galaxy 10 [Commvault Simpana] (BACKUP=GALAXY10)
  • Bacula (BACKUP=BACULA)
  • Bareos (BACKUP=BAREOS) (A fork of Bacula)
  • Rsync Backup Made Easy (BACKUP=RBME)
  • Duplicity/Duply (BACKUP=DUPLICITY)
  • EMC Networker, also known as Legato (BACKUP=NSR)
  • EMC Avamar (BACKUP=AVA)
  • SEP Sesam (BACKUP=SESAM)
  • FDR/Upstream (BACKUP=FDRUPSTREAM)
  • Novastor NovaBACKUP DC (BACKUP=NBKDC)
  • Borg Backup (BACKUP=BORG)

  • Rubrik Cloud Data Management (BACKUP=CDM) (New)

  • Integrates with Disaster Recovery Linux Manager (DRLM)

  • Udev support (except for some really ancient udev versions) which is the base for many important features:

  • kernel drivers for network cards and storage adapters are loaded via udev

  • deal with network persistent names in udev rules
  • firmware loading

  • persistent storage device names (though Relax-and-Recover does nothing with this)

  • Systemd support for the more recent Linux distributions

  • System migration and reconfiguration (MIGRATION_MODE)

  • facilitate recovery on hardware, that is not the same as the original system

  • network and storage drivers are adjusted
  • map hard disks if they do not match (e.g. hda -> sda)
  • remap network MAC addresses
  • use another IP address, or using dhcp via templates or from kernel command line
  • rebuild the initial ramdisk if needed (for new storage drivers)

  • migration to SAN storaged

  • Support backup software: Bacula, both locally attached tapes (with bextract) and network-based backups. Also, in combination with OBDR tapes.

  • Create OBDR tapes with method mkbackup and put the backup onto the tape to have a single-tape bootable recovery solution (deprecated)

  • Label the OBDR tape with the method format to avoid accidental overwrites with OBDR

  • Create bootable disk (eSATA, USB ...) medium with the backup included:

    BACKUP_URL=usb:///dev/device

    Together with OUTPUT=USB we have a complete solution on hard disks (booting of it and restoring data).

  • DHCP client support (IPv4 and IPv6). Dhcp client activation can be forced via the variable USE_DHCLIENT=yes (define in /etc/rear/local.conf). It is also possible to force DHCP at boot time with kernel option dhcp

  • USE_STATIC_NETWORKING=y, will cause statically configured network settings to be applied even when USE_DHCLIENT is in effect

  • Save layout and compare layouts for automation of making Relax-and-Recover snapshots (checklayout option)

  • External USB booting uses extlinux (instead of syslinux), and therefore, the USB disk must first be formatted with an ext2, ext3, ext4 or btrfs based file system

  • VLAN tagging, teaming and bridge support

  • Add timestamp of ReaR run with rc code to the syslog or messages file; sending mail report is also possible

  • The possibility to backup any partition (in particular a Windows partition) via the BACKUP type BLOCKCLONE

  • Unattended ReaR recovery has been improved

  • Improved security model related to SSH keys

  • SSH_FILES='avoid_sensitive_files' (see details in /usr/share/rear/conf/default.conf)

  • SSH_UNPROTECTED_PRIVATE_KEYS='no' (see details in /usr/share/rear/conf/default.conf)

  • LUKS2 Support

Note

Features marked Experimental are prone to change with future releases.

ReaR release 2.7 (July 2022)

New features, bigger enhancements, and possibly backward incompatible changes:

  • ReaR's default TMPDIR is now /var/tmp (it was /tmp before) i.e. ReaR's default working area is now /var/tmp/rear.XXXXX because ReaR needs lots of space (from 300MB up to more than 1GB) to build the ReaR rescue/recovery system e.g. as ISO image and even much more space to also store the backup archive e.g. when the backup should be included in the ISO image. The system's default temporary directory /tmp is no longer suited as default temporary directory for ReaR because nowadays /tmp is often a "tmpfs" that is in RAM so /tmp has RAM/swap limitations and file-hierarchy(7) recommends to use /var/tmp for cases as ReaR. If the user sets TMPDIR explicitly, it still takes precedence.

  • Stop ReaR from possibly overwriting its own disk and backup drives for OUTPUT=USB and OUTPUT=RAWDISK via new WRITE_PROTECTED_... config variables (see /usr/share/rear/conf/default.conf) where UUIDs or filesystem labels can be specified so that disks that contain such UUIDs or filesystem labels will be 'write protected' during "rear recover".

  • Added initial LUKS2 support.

  • Overhauled RAID code with changed RAID related entries in /var/lib/rear/layout/disklayout.conf so users who use RAID and a selfmade /etc/rear/disklayout.conf must adapt their RAID related entries

  • Initial preliminary basic support to automatically resize an active last partition on RAID0 and RAID1 disks to be able to automatically resize RAID0 and RAID1 arrays

  • Automatically shrink LVs if needed during "rear recover". This enables to run "rear recover" with automated LVM LVs shrinking as needed on a bit smaller replacement disk (e.g. when a nominally same sized replacement disk is actually a bit smaller than the original disk). This automated LVs shrinking is not intended when disk(s) are substantially smaller. To migrate onto a substantially smaller replacement disk the user must in advance manually adapt his disklayout.conf file before he runs "rear recover".

  • Support to wipe disks before recreating partitions/volumes/filesystems/... (see the new DISKS_TO_BE_WIPED in /usr/share/rear/conf/default.conf). This is currently new and experimental functionality so that currently by default via DISKS_TO_BE_WIPED='false' no disk is wiped to avoid possible regressions until this new feature was more tested by interested users via an explicit DISKS_TO_BE_WIPED setting.

  • Error out when files greater or equal ISO_FILE_SIZE_LIMIT should be included in the ISO: See the reasoning in /usr/share/rear/conf/default.conf why the default ISO_FILE_SIZE_LIMIT is 2GiB and why we error out when files >= ISO_FILE_SIZE_LIMIT should be included in the ISO.

  • Rsync OUTPUT_URLs are now properly supported with BACKUP=RSYNC. Previously the output went to the location specified by BACKUP_URL and OUTPUT_URL was ignored. One exception was OUTPUT=PXE, where the output was uploaded to OUTPUT_URL in addition to BACKUP_URL, but RSYNC_PREFIX was not respected and the interpretation of the URL was different: A URL of the form rsync://[USER@]HOST[:PORT]/PATH was interpreted as using the rsync protocol, while in all other cases such URL would be interpreted as using rsync over ssh. This special handling is now removed: An rsync OUTPUT_URL with OUTPUT=PXE now creates the RSYNC_PREFIX directory at the destination and the URL is interpreted as in all other cases.

  • RAWDISK: Add local rescue partition installation capability: Introduce a configuration variable RAWDISK_INSTALL_GPT_PARTITION_NAME, which is unset by default. When set, the rescue system will be installed to local disk partitions having the corresponding name.

  • Allow setting a PBA-specific firmware configuration via the new OPAL_PBA_FIRMWARE_FILES configuration variable.

  • Support OPAL 2 self-encrypting NVMe disk drives.

  • OPALPBA: Provide a permanent unlocking mode as a workaround when the firmware did not initialize properly during a 'simple' reboot so the only reliable way to boot was a power cycle after Opal disks were unlocked. To reboot with unlocked disks there is now a new unlocking mode "permanent" via OPAL_PBA_UNLOCK_MODE (see default.conf). Reactivating locking is then the responsibility of the user. Additionally enhanced security by disabling shell access via keyboard interrupt and switching to a password hash for OPAL_PBA_DEBUG_PASSWORD.

  • POWER architecture (ppc64) support for BACKUP=DP: Other platforms supporting the Data Protector Disk Agent, but not the Cell Console (GUI) that is only available on i386, x86_64 and ia64 are now supported with ReaR. This includes in particular ppc64. Checks unavailable on clients without the Data Protector Cell Console (GUI) are skipped. In such cases backup restore can be done using Data Protector GUI only.

  • NSR enhancement for point-in-time recovery with EMC Networker client: A new variable NSR_CLIENT_REQUESTRESTORE for the BACKUP=NSR workflow has been introduced to deal with situations in which the NSR client is allowed/granted to perform its recovery action "on its own" but does not have full control of the EMC networker service/environment, for details see its default.conf section.

  • BACKUP=NBKDC: Enhanced ReaR backup and restore to also work with NovaStor DataCenter 8.0 and higher plus fixed wording "NovaBACKUP DC" -> "NovaStor DC"

  • Support for systemd and parallel restore with Data Protector BACKUP=DP: Support starting Data Protector daemon 'omni.socket' via systemd in the recovery system. Restore performance has been increased by doing parallel restores now.

  • Now "rear format" has in addition to the -efi switch a -bios switch. If none is given (i.e. by default) it will now do hybrid formatting with a BIOS boot partition (on GPT) and an EFI system partition. Accordingly the USB_DEVICE_PARTED_LABEL default is no longer "msdos" (see the updated description in /usr/share/rear/conf/default.conf). This is a starting point for implementing OUTPUT=USB support for UEFI and BIOS dual boot from the same medium. See issue 2698, issue 2818 and issue 2829

  • Overhauled serial console support code. A serial console of the ReaR recovery system can now be specified separately for the kernel and the recovery system bootloader via the generic config variables USE_SERIAL_CONSOLE and SERIAL_CONSOLE_DEVICES and specific config variables SERIAL_CONSOLE_DEVICES_KERNEL, SERIAL_CONSOLE_DEVICE_SYSLINUX and SERIAL_CONSOLE_DEVICE_GRUB (see /usr/share/rear/conf/default.conf for details)

  • Support for HTTP sources when using PXE: There is a new config variable PXE_HTTP_URL to specify a HTTP download source for PXE. See the PXE_HTTP_URL description in /usr/share/rear/conf/default.conf

  • In /usr/share/rear/conf/default.conf changed ISO_VOLID from "RELAXRECOVER" to "REAR-ISO" so the first ISO has the label "REAR-ISO" (8 characters) and subsequent ISOs get the labels "REAR-ISO_01" "REAR-ISO_02" ... respectively that have 11 characters (the maximum length for FAT volume names) so things work now by default when the ISO image is used to (manually) create a FAT bootable USB stick Accordingly 'RELAXRECOVER' in variable names was replaced. In particular the user config variable for automated input USER_INPUT_RELAXRECOVER_SYMLINK_TARGET was renamed as USER_INPUT_ISO_SYMLINK_TARGET so only that new name will work.

Details (mostly in chronological order - newest topmost):

  • Refactor rsync URL support, fixes rsync OUTPUT_URL: The code to parse rsync:// URLs was BACKUP_URL specific. If one specified BACKUP=RSYNC and an OUTPUT_URL different from BACKUP_URL, the OUTPUT_URL was ignored and the output files went to BACKUP_URL. Fix by introducing generic functions for rsync URL parsing and use them for both BACKUP_URL and OUTPUT_URL, as appropriate. Replace all uses of global RSYNC_* variables derived from BACKUP_URL by those functions. There also was inconsistent special handling for OUTPUT=PXE which is now removed: An rsync OUTPUT_URL with OUTPUT=PXE now creates the RSYNC_PREFIX directory at the destination and the URL is interpreted as in all other cases. See issue 2831 and issue 2781

  • Support creation of EFI system partition on systems with older 'parted' that do not support the 'esp' keyword: In format/USB/default/300_format_usb_disk.sh try sgdisk ... --typecode="partition_number:EF00" as fallback when parted ... set partition_number esp on failed. See issue 2830

  • Set USB_DEVICE_PARTED_LABEL to match format-workflow.sh: In format/USB/default/300_format_usb_disk.sh it is set depending on the format workflow option -b/--bios or -e/--efi and in in prep/USB/Linux-i386/340_find_mbr_bin.sh try to autodetect what the USB disk partition type is and use a specified USB_DEVICE_PARTED_LABEL if autodetection fails. See issue 2829

  • In usr/sbin/rear enhanced the check for other simultaneously running 'rear' to let it find running 'rear' instances independent of how it was called by the user in particular also when 'rear' is run from a GitHub checkout/clone cf. issue 2826

  • In usr/share/rear/lib/format-workflow.sh do actually recognise -b/--bios options, see issue 2828

  • In layout/prepare/GNU/Linux/110_include_lvm_code.sh use a fail-safe 'yes' pipe for "lvm lvcreate" to pipe as many 'y' as asked for into "lvm lvcreate". See issue 513 and issue 2820

  • Avoid creating a /bin/vim symlink to vi: The symlink is more confusing than helpful: If one has both vi and vim, and they are different (vi usually has less features), one expects to get the more featureful version by executing "vim", but one gets "vi". More importantly, in recent Fedora and RHEL, "vi" is a shell script that executes "vim" if found, so linking "vim" to "vi" leads to an infinite loop. See issue 2822

  • RPM spec: update build requirement for Fedora to unblock Packit build and run make validate after build, see issue 2816

  • In finalize/Linux-i386/660_install_grub2.sh explain that it is also used as fallback to install the nowadays most often used bootloader GRUB2 unless the BOOTLOADER variable tells to install another bootloader (other bootloader install scripts check the BOOTLOADER variable), cf. issue 2871

  • Replace 'RELAXRECOVER' in variable names: In lib/global-functions.sh in the function mount_url() renamed the user config variable for automated input USER_INPUT_RELAXRECOVER_SYMLINK_TARGET as USER_INPUT_ISO_SYMLINK_TARGET because the old name contained a user config value: "RELAXRECOVER" was an old default value of ISO_VOLID, see issue 2457 and issue 2813

  • In default.conf properly describe PRE_BACKUP_SCRIPT and POST_BACKUP_SCRIPT to match what the implementation does, see issue 2735 and for special cases see issue comment

  • Add PRE_RECOVERY_COMMANDS and POST_RECOVERY_COMMANDS as alternative to PRE_RECOVERY_SCRIPT and POST_RECOVERY_SCRIPT see the description in default.conf how to use them and how they work. See issue 2811 and see also issue 2735 therein in particular comment within issue 2735. Additionally use LogPrint to show the user the executed commands, see issue 2789

  • In default.conf describe the new user config variables USB_BOOT_PART_SIZE and USB_DEVICE_BOOT_LABEL - see PR 2660

  • Exclude dev/watchdog* from the ReaR recovery system: In default.conf add dev/watchdog* to COPY_AS_IS_EXCLUDE because watchdog functionality is not wanted in the recovery system because we do not want any automated reboot functionality while disaster recovery happens via "rear recover", see PR 2808. Furthermore having a copy of dev/watchdog* during "rear mkrescue" in ReaR's build area may even trigger a system crash that is caused by a buggy TrendMicro ds_am module touching dev/watchdog in ReaR's build area (/var/tmp/rear.XXX/rootfs), see issue 22798

  • In output/default/940_grub2_rescue.sh for GRUB_RESCUE set root=/dev/ram0 vga=normal rw (the same is already done for other boot media) to avoid a "Kernel panic ... Unable to mount root fs on unknown-block(0,0)" that could otherwise happen in certain cases, see issue 2791

  • In build/GNU/Linux/400_copy_modules.sh always include loaded kernel modules i.e. also for MODULES=() or MODULES=( 'moduleX' 'moduleY' ) include the currently loaded kernel modules, cf. comment of issue 2727

  • In layout/save/GNU/Linux/200_partition_layout.sh ensure $disk_label is one of the supported partition tables (i.e. one of 'msdos' 'gpt' 'gpt_sync_mbr' 'dasd') and ensure syntactically correct 'disk' and 'part' entries in disklayout.conf (each value must exist and each value must be a single non-blank word), see the last part about "error out directly ... when things failed" in issue 28010. But, do not error out when there is no partition label type value for a 'disk' entry in disklayout.conf, because "rear recover" works in a special case without partition label type value when there is only a 'disk' entry but nothing else for this disk exists in disklayout.conf which can happen when /dev/sdX is an empty SD card slot without medium, see issue 2810

  • In default.conf explain how to use LIBS properly, see issue 2743

  • In the function find_syslinux_modules_dir in lib/bootloader-functions.sh the fallback 'find /usr' to find the SYSLINUX modules directory may take a very long time on some systems (up to several hours) so tell the user in debug mode what is going on, see issue 2792 and issue 624

  • In default.conf tell that MODULES=( 'loaded_modules' 'additional_module' ) is not supported (the generic COPY_AS_IS method can be used to include additional modules) cf. issue 2727

  • Use disklayout.conf keyword raidarray instead of raid because we have a new 'raiddisk' keyword and a keyword must not be a leading substring of another keyword so we have now 'raidarray' and 'raiddisk', see issue 2759 and commit

  • In layout/save/default/450_check_bootloader_files.sh use /[e]tc/grub.cfg and /[b]oot/.../grub.cfg with '*' globbing patterns (as in the EFI|GRUB2-EFI case) to find any of grub.cgf or grub2.cfg in /etc/ or in /boot/ (e.g. in openSUSE Leap 15.3 there is /boot/grub2/grub.cfg), cf. issue 2796 comment

  • In default.conf added '[e]tc/crypttab' to FILES_TO_PATCH_PATTERNS, cf. issue 2795 comment

  • New FILES_TO_PATCH_PATTERNS in default.conf to verify file hashes at the end of recover after file restore from backup and show an error when config files were restored from an outdated backup so the restored files do not match the recreated system, see PR 2795 and issue 2785 and issue 2787

  • In backup/RSYNC/default/450_calculate_req_space.sh comment out remote space check with rsync see issue 2760

  • In layout/save/default/450_check_bootloader_files.sh and layout/save/default/450_check_network_files.sh added bash globbing characters [] around the first letter to ensure that with 'shopt -s nullglob' files that do not exist will not appear so nonexistent files are not appended to CHECK_CONFIG_FILES cf. issue 2796 comment

  • In backup/DUPLICITY/default/500_make_duplicity_backup.sh disable SC2068 cf. issue 1040

  • In backup/YUM/default/500_make_backup.sh and backup/NETFS/default/500_make_backup.sh and prep/NETFS/default/070_set_backup_archive.sh fixed SC2068 cf. issue 1040

  • In rescue/default/850_save_sysfs_uefi_vars.sh disable SC2045 cf. issue 1040

  • In layout/save/GNU/Linux/240_swaps_layout.sh and lib/output-functions.sh fixed SC2045 cf. issue 1040

  • In lib/_input-output-functions.sh disable SC2218 cf. issue 1040

  • In layout/save/GNU/Linux/230_filesystem_layout.sh aviod SC1087 cf. issue 1040

  • In default.conf tell that USB_RETAIN_BACKUP_NR is only supported when EXTLINUX is used as bootloader for USB and in output/USB/Linux-i386/300_create_extlinux.sh add an explanatory comment how that code works, cf. issue 2794 comment

  • In layout/save/GNU/Linux/210_raid_layout.sh fixed and overhauled the RAID10 'layout' support code (i.e. what belongs to the mdadm -layout option). This was triggered by ShellCheck SC2034 and SC2066 for the old code, see issue 2758

  • In finalize/Fedora/i386/550_rebuild_initramfs.sh fixed SC2068 and SC2145 via some generic code cleanup cf. issue 1040 comment and see issue 2771

  • In rescue/GNU/Linux/310_network_devices.sh fix SC2091 see issue 2776

  • In default.conf increase USB_UEFI_PART_SIZE to 1024 MiB, cf. https://github.com/rear/rear/pull/1205 in particular to also make things work by default when additional third-party kernel modules and firmware (e.g. from Nvidia) are used, cf. issue 2770 comment

  • In doc/rear.8 and doc/rear.8.adoc fixed typo '/dev/disk/by-path/REAR-000' -> '/dev/disk/by-label/REAR-000'

  • In default.conf better describe USB_DEVICE cf. issue 2770 comment

  • In output/default/940_grub_rescue.sh removed a misleading comment that shows a wrong get_version function call and in output/USB/Linux-i386/850_make_USB_bootable.sh fixed a wrong get_version function call that is needed since the ShellCheck SC2068 fixed get_version function in lib/layout-functions.sh

  • In layout/recreate/default/120_confirm_wipedisk_disks.sh and layout/prepare/Linux-s390/205_s390_enable_disk.sh and rescue/GNU/Linux/310_network_devices.sh and rescue/GNU/Linux/230_storage_and_network_modules.sh and restore/default/990_move_away_restored_files.sh and restore/NBU/default/300_create_nbu_restore_fs_list.sh and skel/default/etc/scripts/system-setup.d/00-functions.sh and skel/default/etc/scripts/dhcp-setup-functions.sh and wrapup/default/990_copy_logfile.sh and usr/share/rear/lib/output-functions.sh and lib/layout-functions.sh and lib/filesystems-functions.sh and lib/linux-functions.sh avoid ShellCheck SC2068 cf. issue 1040 comment

  • In output/ISO/Linux-i386/800_create_isofs.sh avoid ShellCheck SC2068 and SC2145 cf. issue 1040 comment

  • In output/TSM/default/960_dsmc_verify_isofile.sh avoid ShellCheck SC2068 and SC2145 cf. issue 1040 comment plus better error checking and messaging and other code cleanup

  • In output/default/940_grub2_rescue.sh and output/default/940_grub_rescue.sh avoid ShellCheck SC2068 cf. issue 1040 comment

  • In output/default/950_email_result_files.sh avoid ShellCheck SC2068 and SC2145 cf. issue 1040 comment and replaced ...IfError function calls by calling Error

  • In restore/FDRUPSTREAM/default/260_copy_log_and_report.sh fixed ShellCheck reported error SC2199 cf. issue 1040 comment and fixed possible "bash: conditional binary operator expected" error

  • In output/USB/Linux-i386/300_create_extlinux.sh and rescue/GNU/Linux/310_network_devices.sh avoid ShellCheck reported error SC2199 cf. issue 1040 comment

  • In prep/NBKDC/default/400_prep_nbkdc.s and lib/sesam-functions.sh avoid ShellCheck false error indication SC1097 cf. issue 1040 comment

  • In lib/_input-output-functions.sh fixed ShellCheck reported errors SC2145 and SC2068 and avoided ShellCheck false error indication SC1087 cf. issue 1040 comment

  • In usr/sbin/rear avoid ShellCheck SC1075 and fixed ShellCheck SC2145 cf. issue 1040

  • In output/RAWDISK/Linux-i386/260_create_syslinux_efi_bootloader.sh fixed ShellCheck SC2235 cf. issue 1040 comment

  • In rear/lib/sesam-functions.sh fixed ShellCheck SC2221 cf. issue 1040 comment

  • In lib/filesystems-functions.sh in function xfs_parse fixed ShellCheck SC2179 and SC2128 cf. issue 1040 comment furthermore 'xfs_opts' is local in both functions xfs_parse and create_fs in layout/prepare/GNU/Linux/131_include_filesystem_code.sh

  • In lib/layout-functions.sh fixed ShellCheck SC2178 cf. issue 1040 comment

  • In skel/default/etc/scripts/system-setup.d/00-functions.sh ignore ShellCheck SC2119 and SC2120 cf. issue 1040 comment

  • In restore/DUPLICITY/default/150_restore_duply.sh avoid ShellCheck SC2103 cf. issue 1040 comment

  • In layout/prepare/GNU/Linux/100_include_partition_code.sh fixed ShellCheck SC2199 and SC2076 cf. issue 1040 comment

  • In output/ISO/Linux-ia64/400_create_local_efi_dir.sh fixed ShellCheck SC2050 cf. issue 1040 comment

  • In layout/save/GNU/Linux/230_filesystem_layout.sh fixed ShellCheck SC2030 cf. issue 1040 comment

  • In verify/NBU/default/380_request_client_destination.sh fixed ShellCheck SC2018 and SC2019 cf. issue 1040 comment

  • In backup/YUM/default/500_make_backup.sh fixed ShellCheck SC2000 cf. issue 1040 comment

  • Simpler code in prep/Linux-s390/305_include_s390_tools.sh to set a fixed bootdir="/boot" that also fixes ShellCheck SC1066 cf. issue 1040 comment

  • New GRUB2_SEARCH_ROOT_COMMAND in default.conf to overrule the setting in output/USB/Linux-i386/100_create_efiboot.sh see issue 2763 and issue 2500

  • New EXCLUDE_IP_ADDRESSES and EXCLUDE_NETWORK_INTERFACES directives: These new array variables enable to exclude specific IP addresses or network interfaces when building the network configuration used in the rescue environment. This is typically useful when floating IP addresses are used. Not excluding these may lead to outage if the floating IP address is used by another system at time the system is getting recovered, see issue 2736

  • multipath: fix exclusion of still wanted devices: The current code excluding multipath devices is broken when a device being excluded matches other devices. This leads to excluding wanted devices. This happens when having custom alias for multipath devices or there are more than 26 multipath devices and 'mpatha' is getting excluded, which leads to excluding all 'mpathaX' devices are well, see issue 2750

  • Update 06-layout-configuration.adoc: Enhance the "disk layout file syntax" description: Describe that one cannot rely on backward compatibility. Describe positional parameters vs. option=value parameters.

  • Update default.conf: In default.conf describe that when OUTPUT_URL is set OUTPUT_OPTIONS does not inherit the BACKUP_OPTIONS value cf. issue 2753

  • Include dmsetup and dmeventd as PROGS in conf/GNU/Linux.conf because older releases of os-prober (1.74 and below) use dmsetup as a fallback solution for mounting when grub-mount is missing but without dmsetup and dmeventd also in the recovery system that would block indefinitely at "Installing GRUB2 boot loader...", for details see issue 2748

  • In the function cmdline_add_console in lib/serial-functions.sh add 'console=tty0' as fallback to cmdline only if no real serial device was found otherwise on a machine with e.g. ttyS0 and ttyS1 where only ttyS0 is real only kernel messages (but nothing else) would appear on ttyS0, see issue 2749

  • Skip unneeded /usr/lib/syslog-ng/loggen/ from recovery system: In conf/GNU/Linux.conf do no longer copy all in /usr/lib/syslog-ng/ but only copy .so files in /usr/lib/syslog-ng/ (same as for rsyslog). This skips in particular /usr/lib*/syslog-ng/loggen/ because 'loggen' is not included in the recovery system ('loggen' does not appear in any ReaR code). See issue 2743

  • For BACKUP=CDM removed need for the whole /usr/lib64 in the recovery system. See issue 2747 and the related issues 2266, 2314, 2685' and issue 2700

  • Update 400_copy_modules.sh: Do no longer error out if 'cp -a -L' failed to to copy all contents of /lib/modules/... but only tell the user about the issue so he could inspect his system and decide, see issue 2739 comment

  • Support for HTTP sources when using PXE: There is a new config variable PXE_HTTP_URL to specify the HTTP download source for PXE. If the variable is set the PXE config file will generate an additional 3rd boot option (besides the unchanged standard options "rear" via TFTP and "local") namely "rear-http" which includes the HTTP URL information to download the kernel and initrd data. See issue 2738

  • Also add ssh-add prog when ssh-agent is added. Since ssh-agent gets added it makes sense to add ssh-add as well. See PR 2729

  • In layout/prepare/default/420_autoresize_last_partitions.sh automatically resize active last partitions on RAID0 disks, see issue 2696. Therefore in layout/save/GNU/Linux/210_raid_layout.sh a new 'raiddisk' entry that has same syntax as a 'disk' entry is written to disklayout.conf to have size (and partition label) of a RAID device like raiddisk /dev/md127 23599054848 gpt because knowing the RAID device size is a precondition to be able to automatically resize a RAID0 array. See issue 2730

  • In layout/prepare/default/420_autoresize_last_partitions.sh automatically resize active last partitions on RAID1 disks and fixed fallback setting so that now the user can specify AUTORESIZE_EXCLUDE_PARTITIONS=() in his local.conf. Additionally in layout/prepare/default/250_compare_disks.sh show info about old disk and sizes and new disks and sizes in case of MIGRATION_MODE to make it easier to answer the questions to which new disks old disk should be mapped. See PR 2726

  • Fix modinfo_filename() in build/GNU/Linux/400_copy_modules.sh Output the original $module_filename also when it is a symlink because in the code below 'cp -L' copies the symlink taget content as a new regular file with file name as the name of the symlink so the copied content can be still found under its original name, cf. issue 2677 comment Additionally in case of MODULES=( 'all_modules' ) also use 'cp -L' to copy the actual content to avoid dangling symlinks in the recovery system. See PR 2731

  • In usr/share/rear/output/PXE/default/800_copy_to_tftp.sh fixed pxe file cp and permissions for sshfs target to prevent cp error "failed to preserve ownership" for sshfs and to prevent issue with write permissions trying to override image on later runs. See issue 2723

  • Improved wiping disks (see PR 2721): In layout/recreate/default/120_confirm_wipedisk_disks.sh skip disks that do not exist on the bare hardware in the recovery system cf. issue 2715 and exclude disks that are write-protected cf. issue 2703 comment and show in any case a user confirmation dialog for the disks that will be wiped. In layout/recreate/default/150_wipe_disks.sh do no longer open (and close) LUKS volumes because encrypted volumes contain meaningless data unless opened and unencrypted so there is no need to wipe anything inside an encrypted LUKS container, cf. "Regarding LUKS" in issue 2514 comment

  • Overhauled RAID code (see PR 2714) based on https://github.com/rear/rear/pull/2702 that is about initial software RAID IMSM support i.e. with the changes in that pull request: Completely overhauled layout/save/GNU/Linux/210_raid_layout.sh: No longer a subshell that appends all stdout to disklayout.conf but explicit append to disklayout.conf where needed to be safe against accidental things written to disklayout.conf. Handle each mdadm option in one place i.e. parse and prepare output. Handle options ordered by importance, mandatory first, then optional ones. Basic tests that mandatory options are syntactically valid plus Error if not. Overhauled layout/prepare/GNU/Linux/120_include_raid_code.sh: The FEATURE_MDADM_UUID code is meanwhile obsolete because all mdadm versions in supported ReaR Linux distributions support '-uuid'. New layout/recreate/default/220_verify_layout.sh to verify if RAID devices are recreated with the UUIDs in disklayout.conf because mdadm silently ignores this option when creating IMSM arrays (both containers and the volumes inside them) and picks a random UUID cf. issue 2701 comment Support user specified DISKS_TO_BE_WIPED to mitigate issue 2715 see the DISKS_TO_BE_WIPED description in default.conf

  • Update 100_create_layout_file.sh: Additionally show the filesystem LABEL in the 'lsblk' output of the original system as comment in disklayout.conf to make it easier to understand subsequent data in particular for RAID where the array name is shown as LABEL in 'lsblk' for example like "/dev/sda ... linux_raid_member any:raid1sdab" - see also "one cannot see ... /dev/md/ symlinks as NAME ... /dev/md/raid1sdab -> ../md127" in PR 2702 comment and see also commit

  • Update 100_create_layout_file.sh: Additionally show UUID in 'lsblk' output of the original system as comment in disklayout.conf to make it easier to compare UUIDs of the original system with what was recreated, cf. "... shouldn't it [ReaR] restore the [RAID] arrays with the same UUIDs ...?" in https://github.com/rear/rear/pull/2702#issuecomment-968904230 and https://github.com/rear/rear/pull/2714#issuecomment-970279152

  • Update 050_start_required_nfs_daemons.sh: Start rpcbind with the -s option to avoid that in some cases rpcbind is not started / not available in the recovery system cf. https://github.com/rear/rear/issues/2672

  • New function is_multipath_used in layout-functions.sh (see https://github.com/rear/rear/pull/2708) to get 'multipath -l' out of the function is_multipath_path by adding a separated function is_multipath_used that runs 'multipath -l' (which is expensive because it scans all devices) to check if multipath is used and remembers the result in MULTIPATH_IS_USED that is used in subsequent calls of is_multipath_used (without calling 'multipath -l') so we can call is_multipath_used many times via is_multipath_path which is called in layout/save/GNU/Linux/200_partition_layout.sh for each disk device. This should avoid long computing time when there are very may disk devices, cf. https://github.com/rear/rear/pull/2597#issuecomment-814950019

  • Now "rear format" has in addition to the -efi switch a -bios switch. If none is given (i.e. by default) it will now do hybrid formatting with a BIOS boot partition (on GPT) and an EFI system partition. This is a starting point for implementing OUTPUT=USB support for UEFI and BIOS dual boot from the same medium see https://github.com/rear/rear/issues/2698 and https://github.com/rear/rear/pull/2705

  • Update write-protect-functions.sh: In the function is_write_protected_by_id() assume a disk without any of UUID PTUUID PARTUUID WWN is empty and meant to be used to recreate the system so it should not be write-protected cf. https://github.com/rear/rear/pull/2703#discussion_r757393547

  • Enhanced disk write-protection, see https://github.com/rear/rear/pull/2703 and https://github.com/rear/rear/pull/2626 by WRITE_PROTECTED_IDS with generic functionality, cf. https://github.com/rear/rear/pull/2626#issuecomment-950953826 together with the new WRITE_PROTECTED_ID_TYPES which defaults to UUID PTUUID PARTUUID WWN so that the user can specify different lsblk columns as needed in his particular environment cf. https://github.com/rear/rear/pull/2703#issuecomment-962418441

  • Stop ReaR from overwriting its own disk and backup drives for OUTPUT=USB and OUTPUT=RAWDISK via new WRITE_PROTECTED_... config variables (see default.conf) where UUIDs or filesystem labels can be specified so that disks that contain such UUIDs or filesystem labels will be 'write protected' during "rear recover", see https://github.com/rear/rear/issues/1271 and https://github.com/rear/rear/pull/2626

  • Overhauled serial console support code. See https://github.com/rear/rear/pull/2699 A serial console of the ReaR recovery system can now be specified separately for the kernel and the recovery system bootloader via the generic config variables USE_SERIAL_CONSOLE and SERIAL_CONSOLE_DEVICES and specific config variables SERIAL_CONSOLE_DEVICES_KERNEL SERIAL_CONSOLE_DEVICE_SYSLINUX SERIAL_CONSOLE_DEVICE_GRUB see usr/share/rear/conf/default.conf for details cf. https://github.com/rear/rear/issues/2663 and https://github.com/rear/rear/pull/2697

  • More control over serial devices used for console. See https://github.com/rear/rear/pull/2697

  • In layout/prepare/default/300_map_disks.sh overhauled the automapping of original 'disk' devices and 'multipath' devices to current block devices in the currently running recovery system so that now it automatically finds an existing unique disk size mapping also when there is a unique mapping between more than two disks, see https://github.com/rear/rear/issues/2690

  • Fix multiple VG recreation, see https://github.com/rear/rear/pull/2691 Do not use global variables in diskrestore.sh: Fix a problem introduced in commits b184194f37dd22a7e55655ff388579297239e73c and 311bfb3da1d5e47a2ff144123a2457e634f67893 (PR #1806) that shows up when there are multiple VGs to restore. Using variables create_thin_volumes_only and create_logical_volumes to propagate information from VG creation to LV creation does not work well in the case of multiple VGs, because the variables are global and if there are multiple VGs, their values will leak from one VG to another. The generated diskrestore.sh script does not guarantee that the LVs of a given VG are created immediately after their VG and before creating another VG. Currently, the script first creates all VGs and then all LVs, so all the LVs in all VGs will see the value of create_logical_volumes and create_thin_volumes_only from the last VG, not from their own. This matters when different VGs behave differently (typically if one has a thin pool and the other does not). Fix by replacing the scalar values by arrays of VG names. If a given VG is in the array, it is the equivalent of the former scalar value being 1 for the given VG, if it is not in the array, it is an equivalent of a former value of 0. For the create_volume_group variable the change is not needed, but do it nevertheless for symmetry with other variables.

  • Error out in sbin/rear when it failed to source or Source() mandatory files to make it obvious when there are syntax errors in user config files and/or when there are syntax errors in our lib/*-functions.sh code, see https://github.com/rear/rear/issues/2686

  • Overhauled output/USB/Linux-i386/100_create_efiboot.sh In particular no longer use "cp -p" because '-p' may fail because it copies to a VFAT filesystem on the EFI partition see https://github.com/rear/rear/issues/2683 and now always use "cp -L" to ensure the real content gets copied. See https://github.com/rear/rear/pull/2684

  • Overhauled the description in user-guide/05-integration.adoc what "rear savelayout" and "rear checklayout" are menat to do, cf. https://github.com/rear/rear/commit/7a3884bed1fb6cb61654c4e266ba4c35094bd9f4#r56879062

  • Fixed https://github.com/rear/rear/issues/2667 "rear does not entirely remove build area any more" by using double quotes for remove_temporary_mountpoint "$BUILD_DIR/..." and fixed https://github.com/rear/rear/issues/2676 "False ERROR '.../outputfs not empty, cannot remove' when OUTPUT_URL is unset" by making scheme_supports_filesystem() fail-safe if $1 is empty or blank and also making scheme_accepts_files() fail-safe in the same way, see https://github.com/rear/rear/pull/2675

  • On POWER require bootlist & ofpathname if needed: The ofpathname binary is called by grub2-install. Therefore, it is required in the rescue system in order to make the recovered system bootable, except for the PowerNV (Not Virtualized - bare metal) case. Under PowerVM, we also need the bootlist executable to make the system bootable. See https://github.com/rear/rear/pull/2665

  • ReaR's default TMPDIR is now /var/tmp (it was /tmp before) i.e. ReaR's default working area is now /var/tmp/rear.XXXXX ReaR needs lots of space (from 300MB up to more than 1GB) to build the ReaR rescue/recovery system e.g. as ISO image and even much more space to also store the backup archive e.g. when the backup should be included in the ISO image. The system's default temporary directory /tmp is no longer suited as default temporary directory for ReaR because nowadays /tmp is often a "tmpfs" that is in RAM so /tmp has RAM/swap limitations. file-hierarchy(7) recommends to use /var/tmp for such cases. If the user sets TMPDIR explicitly, it still takes precedence. See https://github.com/rear/rear/pull/2664

  • Cleanup of GRUB2 config code in particular for OUTPUT=USB: Cleanup of the create_grub2_cfg function: Describe its usage in a comment. Have its helper functions inside create_grub2_cfg to make them not globally accessible. Initial steps to clean up how the GRUB root device is set via one single method. DebugPrint output how GRUB2 is configured to help the user to find errors or bugs. Better GRUB2 menue entry names. Consistent calling of the create_grub2_cfg function always as create_grub2_cfg "/path/to/kernel" "/path/to/initrd" >/path/to/grub.cfg also in output/ISO/Linux-i386/250_populate_efibootimg.sh. Set GRUB2 timeout to new config variable GRUB2_TIMEOUT that is by default 300 seconds. Set GRUB2 default boot menu entry to GRUB2_DEFAULT_BOOT that is by default 'chainloader' which should usually boot the original system and reference GRUB2 boot menu entries by 'menuentry ... -id=...' to have meaningful names instead of meaningless numbers. Sleep 3 seconds exactly before GRUB2 shows its menu (it replaces what there is on the screen) so that the user could see possible GRUB2 (error) messages on the screen. Additionally cleanup of the get_root_disk_UUID function. See https://github.com/rear/rear/pull/2662

  • Fixed serial console for syslinux: It writes only one serial line matching the configured device (when found) to the config. It also writes it for the syslinux config in case it is used without extlinux. See https://github.com/rear/rear/pull/2650

  • Fix function create_grub2_cfg() for the none EFI cases and adds serial console configuration to the GRUB2 config. See https://github.com/rear/rear/pull/2661

  • Simplifications for USB_BOOTLOADER="grub" and choose the right GRUB2 config /boot/grub/grub.cfg or /boot/grub2/grub.cfg see https://github.com/rear/rear/pull/2659

  • Cleanup USB format scripts and related code: Now OUTPUT=USB should behave sufficiently OK on BIOS systems with syslinux/extlinux as bootloader with a separated boot partition and without it (still the default). To use OUTPUT=USB on BIOS with syslinux/extlinux with a separated boot partition size of e.g. 1024 MiB specify in etc/rear/local.conf something like OUTPUT=USB USB_BOOT_PART_SIZE=1024 OUTPUT_URL=usb:///dev/disk/by-label/REARBOOT BACKUP=NETFS BACKUP_URL=usb:///dev/disk/by-label/REAR-000 See https://github.com/rear/rear/pull/2660

  • Fixed format/USB/default/300_format_usb_disk.sh for none EFI cases by adding a BIOS boot partition and setting the 'legacy_boot' flag in case of GPT partitioning. See https://github.com/rear/rear/pull/2656

  • Added support for GRUB2 as bootloader for OUTPUT=USB via new config variable USB_BOOTLOADER=grub see https://github.com/rear/rear/pull/2655

  • With the new default USER_INPUT_MAX_CHARS=0 input is not truncated and it also makes correcting the input possible (before [Enter] is pressed) see https://github.com/rear/rear/issues/2622

  • Update default.conf: Since https://github.com/rear/rear/pull/2633 export TMPDIR="..." in ReaR config files like etc/rear/local.conf does no longer work so now export TMPDIR="..." must be called before calling 'rear' cf. https://github.com/rear/rear/issues/2654

  • Treat RAID layout value '-unknown-' same as an empty value: In layout/save/GNU/Linux/210_raid_layout.sh treat the RAID layout value '-unknown-' same as an empty value to avoid that layout/prepare/GNU/Linux/120_include_raid_code.sh will create a 'mdadm' command in diskrestore.sh like "mdadm ... -layout=-unknown- ..." which would fail during "rear recover" with something like "mdadm: layout -unknown- not understood for raid0", see https://github.com/rear/rear/issues/2616

  • Avoid false Error() exit in copy_binaries() in build/GNU/Linux/390_copy_binaries_libraries.sh by skipping 'cp' errors in copy_binaries() (regardless what the reason was why 'cp' failed) but then add the affected binary to REQUIRED_PROGS to verify later that the binary actually exists in the recovery system which should be still sufficient to let ReaR error out if something actually went wrong with needed binaries. See https://github.com/rear/rear/pull/2643

  • NSR enhancement for point-in-time recovery with EMC Networker client: A new variable NSR_CLIENT_REQUESTRESTORE for the BACKUP=NSR workflow has been introduced to deal with situations in which the NSR client is allowed/granted to perform its recovery action "on its own" but does not have full control of the EMC networker service/environment, for details see its default.conf section. See https://github.com/rear/rear/pull/2641

  • Workaround against DRBD9 restore error "refusing to be primary while peer is not outdated" see https://github.com/rear/rear/issues/2634

  • Cleanup rsync and fix error reporting: See https://github.com/rear/rear/pull/2632 Cleanups of rsync code to use better variable names and local variables, stop using StopIfError, use better redirections etc. Fixed a problem with rsync error detection that caused rsync errors during backup restore to be ignored, see https://github.com/rear/rear/issues/2612 Now a warning is displayed and if BACKUP_INTEGRITY_CHECK is true, it is elevated to an error and ReaR aborts. Furthermore check fake super support only when RSYNC_PROTO=ssh and removed a broken test for -fake-super arg to rsync, cf. https://github.com/rear/rear/pull/2577

  • Show some stdout and stderr messages also in non-debug modes: In non-debug modes (in particular also in verbose mode) stdout and stderr are redirected to a temporary file STDOUT_STDERR_FILE=TMP_DIR/rear.WORKFLOW.stdout_stderr so in non-debug modes stdout and stderr of all programs is still available for the Error() function to extract some latest messages that get shown on the usrer's terminal and those extracted lines are also copied into the log cf. https://github.com/rear/rear/issues/2623 and https://github.com/rear/rear/issues/2416 and https://github.com/rear/rear/pull/2498 Furthermore the log files and the stdout/stderr file can now only be read by root. Additionally when there is something still mounted within the build area when rear finishes, the user is informed what is mounted and that he must manually umount it before he can (also manually) remove the build area. Finally the Error() and cleanup_build_area_and_end_program() functions are made fail-safe against not yet existing log files and not yet sourced other functions in case of early Error() exits in usr/sbin/rear see https://github.com/rear/rear/pull/2633

  • Fixed accidental destructive backup removal in exit task and cleanup handling of URL mountpoints: See https://github.com/rear/rear/pull/2625 Cleanup of temporary mount point handling, particularly for output. Unification of mount point umount and cleanup move to the mount_url() and umount_url() functions. Replaced the various "rm -rf" of the mountpoint by "rmdir" which fixes https://github.com/rear/rear/issues/2611 Added lazy umount in case normal umount does not succeed. If build dir is kept (cf. KEEP_BUILD_DIR), propose a safe way to remove it to the user via "rm -Rf -one-file-system" instead of just "rm -Rf" where the user risks to remove everything below that mountpoint if still mounted. Fixes also some other bugs noted in the process: Filesystem-specific umount command not called https://github.com/rear/rear/commit/20359a987662cc0c3fcfa52d62d1feac7cd55850#r51319634 Unknown schemes considered invalid, see the discussion under https://github.com/rear/rear/pull/932 Identical scripts under DUPLICITY and YUM replaced by symlinks. Reverted https://github.com/rear/rear/pull/782 that had reintroduced https://github.com/rear/rear/issues/465 which got re-reported as https://github.com/rear/rear/issues/2611 Reverted https://github.com/rear/rear/pull/578 because it is not clear how .lockfile can exist in the unmounted filesystem, and if it does, it is a bug. Reverted https://github.com/rear/rear/commit/d850c4094238a03c9b926b88d7e1582ecd28af52 because it became meanwhile obsoleted by https://github.com/rear/rear/commit/a8fdc445d0d6f7f9184dc6633817928f96aae9bc

  • Add BACKUP_DUPLICITY_OPTIONS to restore/DUPLICITY/default/400_restore_duplicity.sh Additionnal BACKUP_DUPLICITY_OPTIONS options were missing which can cause the recover to fail (options may contain endpoint information for example), see https://github.com/rear/rear/issues/2619

  • In prep/default/400_save_directories.sh also exclude mountpoints that are below mountpoints of "type autofs": Those are below an ancestor mountpoint that is owned/created by the automounter. It is possible to create a sub-mountpoint below an automounted mountpoint but the fact that the sub-mountpoint is not local means it should be excluded (i.e. there is no need to recreate the non-local sub-mountpoint directory), see https://github.com/rear/rear/issues/2610

  • Better description and error checking for GRUB_RESCUE with UEFI plus some alignment with the create_grub2_cfg function, cf. https://github.com/rear/rear/issues/2545 Better describe GRUB_RESCUE with UEFI in default.conf and in output/default/940_grub2_rescue.sh better error checking plus some alignment with how create_grub2_cfg() creates a GRUB2 config file i.e. no longer "insmod" the GRUB2 modules efi_gop efi_uga video_bochs video_cirrus cf. https://github.com/rear/rear/pull/2609#issuecomment-831883795 and the discussion in https://github.com/rear/rear/issues/2388 see https://github.com/rear/rear/pull/2609

  • Fix setting boot path in case of UEFI partition (ESP) on MD RAID: The code finalize/Linux-i386/670_run_efibootmgr.sh did not support Software RAID. It has to execute the efibootmgr command on the members of the Software RAID, see https://github.com/rear/rear/issues/2595 The ESP may be located on a RAID device. In this case, we need to determine the physical RAID components and call efibootmgr on them. In addition, clean up finalize/Linux-i386/670_run_efibootmgr.sh and add more logging to ease analysis when something goes wrong, see https://github.com/rear/rear/pull/2608

  • In prep/RSYNC/default/050_prep_rsync.sh adding sleep period to give rsync daemon time to wake up, see https://github.com/rear/rear/pull/2599

  • Have unused LVM PV devices only as comment in disklayout.conf: PVs that are not part of a VG are documented as comment in disklayout.conf but they are not recreated because they were not used on the original system so there is no need to recreate them by "rear recover" see https://github.com/rear/rear/issues/2596 and https://github.com/rear/rear/pull/2603

  • Automatically shrink LVs if needed during "rear recover". This enables to run "rear recover" with automated LVM LVs shrinking as needed on a bit smaller replacement disk (e.g. when a nominally same sized replacement disk is actually a bit smaller than the original disk). This automated LVs shrinking is not intended when disk(s) are substantially smaller. To migrate onto a substantially smaller replacement disk the user must in advance manually adapt his disklayout.conf file before he runs "rear recover". In layout/prepare/GNU/Linux/110_include_lvm_code.sh assume the command "lvcreate -L 123456b -n LV VG" had failed because of "Volume group ... has insufficient free space" and then try as fallback attempt using all remaining free space in the VG via "lvcreate -l 100%FREE -n LV VG". In layout/save/GNU/Linux/220_lvm_layout.sh sort the 'lvs' output lines by size of the LVs so only some biggest LVs may get automatically shrinked (if needed) because we assume that the data of the backup can still be restored into a big LV after it was shrinked a bit. See https://github.com/rear/rear/pull/2591

  • Replaced possibly misleading "USB device" by generic "USB or disk device" wording in user messages (e.g. when a built-in disk /dev/sdc is used that is no USB device). The word "USB" indicates that those messages belong to the OUTPUT=USB method, cf. https://github.com/rear/rear/pull/2589#issuecomment-805649510 and https://github.com/rear/rear/issues/2588 and https://github.com/rear/rear/pull/2589

  • Wipe disks before recreating partitions/volumes/filesystems/... see https://github.com/rear/rear/issues/799 See the new DISKS_TO_BE_WIPED in default.conf and for details see usr/share/rear/layout/recreate/default/README.wipe_disks This is currently new and experimental functionality so that currently by default via DISKS_TO_BE_WIPED='false' no disk is wiped to avoid possible regressions until this new feature was more tested by interested users via an explicit DISKS_TO_BE_WIPED setting in local.conf see https://github.com/rear/rear/pull/2514

  • In etc/scripts/system-setup.d/41-load-special-modules.sh load the nvram kernel module if possible to make /dev/nvram appear because /dev/nvram should be there when installing GRUB, see https://github.com/rear/rear/issues/2554 and include the nvram kernel module in the recovery system because nvram could be a module in particular on POWER architecture see https://github.com/rear/rear/issues/2554#issuecomment-764720180 and https://github.com/rear/rear/pull/2580

  • Make 400_copy_modules.sh fail-safe for newer modinfo output (in particular modinfo in kmod-27 since SLES15-SP3) because otherwise copying builtin kernel "modules" would let "rear mkrescue" error out with e.g. "ERROR: unix exists but no module file?". Furthermore ensure that kernel modules that should be loaded during recovery system startup (i.e. those in MODULES_LOAD) get always copied into the recovery system. See https://github.com/rear/rear/pull/2579

  • In 300_format_usb_disk.sh adapt USB_UEFI_PART_SIZE fallback value to 512 to be in compliance with the new default.conf setting of 512 MiB since https://github.com/rear/rear/commit/9a6b9a109aa77afc6c96cf05bbd7988cf0310d61 that was done in relation to https://github.com/rear/rear/pull/2576 see also https://github.com/rear/rear/issues/2602

  • Do not specify '-F 16' for mkfs.vfat and also no '-o fat=16' when mounting it but rely on the mkfs.vfat automatic FAT type selection (which is FAT16 up to 511 MiB and FAT32 starting at 512 MiB at least on openSUSE Leap 15.2 with mkfs.vfat from dosfstools-4.1) and rely on the automatic FAT type detection when mounting, cf. https://github.com/rear/rear/issues/2575 and increase the default ESP size to USB_UEFI_PART_SIZE="512" to get by default (via mkfs.vfat) a FAT32 ESP to be in compliance that "the ESP should officially use a FAT32 filesystem" to avoid possible FAT16 ESP issues with certain UEFI firmware. See https://github.com/rear/rear/pull/2576

  • Update 230_filesystem_layout.sh: Increase the 'docker info' timeout value from 5 seconds to 10 seconds because sometimes 'docker info' needs more than 5 seconds to finish, cf. https://github.com/rear/rear/pull/2572#issuecomment-784110872

  • Support loop devices in format_usb_disk see https://github.com/rear/rear/pull/2555

  • Update 110_include_lvm_code.sh: Make sure we delete the volume group before re-creating it. The issue happens in Migration mode when ReaR is not trying to use vgcfgrestore. See https://github.com/rear/rear/pull/2564

  • In usr/share/rear/output/ISO/Linux-i386/700_create_efibootimg.sh add 2 more 32MiB blocks to be on the safe side against inexplicable failures like "cp: error writing '/tmp/rear.XXX/tmp/efi_virt/./EFI/BOOT/elilo.conf': No space left on device" where the above calculated $efi_img_sz is a bit too small in practice, see https://github.com/rear/rear/issues/2552

  • Add terminal password check via 'TTY_ROOT_PASSWORD' and describe it in default.conf, see https://github.com/rear/rear/pull/2539

  • Changes for NetBackup (NBU) support: Copy NetBackup PBX related files to the rescue system and start vxpbx_exchanged on boot. Add /usr/openv/tmp directory to the NBU skeleton. See https://github.com/rear/rear/pull/2544

  • RAWDISK: Add local rescue partition installation capability: Introduce a configuration variable 'RAWDISK_INSTALL_GPT_PARTITION_NAME', which is unset by default. When set, the rescue system will be installed to local disk partitions having the corresponding name. RAWDISK: fix local rescue partition installs (unmount before copying): While the recently created EFI boot partition is still mounted, its file system might not be fully synced. Unmounting ensures that the underlying loop block device is up to date before possibly being copied to local disk partitions. See https://github.com/rear/rear/pull/2538

  • BACKUP=BACULA: Make bacula aware of different location of config files see https://github.com/rear/rear/pull/2535

  • In layout/prepare/GNU/Linux/131_include_filesystem_code.sh fixed vfat mkfs using uuid from layout if possible see https://github.com/rear/rear/pull/2546

  • Completely overhauled USB filesystem kernel module handling: In prep/USB/default/380_copy_usb_fs_module.sh do no longer unconditionally add the USB filesystem to the MODULES and MODULES_LOAD arrays because there are systems where the USB filesystem (e.g. 'ext3') is no kernel module and then (via MODULES_LOAD and /etc/modules) "modprobe ext3" would fail in 40-start-udev-or-load-modules.sh with a false alarm "modprobe: FATAL: ... ext3 not found ..." message that would appear on the user's terminal during recovery system startup. See https://github.com/rear/rear/pull/2537

  • Add DP_LD_LIBRARY_PATH to include shared libraries used by BACKUP=DP see https://github.com/rear/rear/pull/2549

  • POWER architecture (ppc64) support for BACKUP=DP: Other platforms supporting the Data Protector Disk Agent, but not the Cell Console (GUI) that is only available on i386, x86_64 and ia64 are now supported with ReaR. This includes in particular ppc64. Checks unavailable on clients without the Data Protector Cell Console (GUI) are skipped. In such cases backup restore can be done using Data Protector GUI only.

  • BACKUP=NBKDC: Enhanced ReaR backup and restore to also work with NovaStor DataCenter 8.0 and higher plus fixed wording "NovaBACKUP DC" -> "NovaStor DC", see https://github.com/rear/rear/issues/2518

  • Provide final power to the user to skip the ISO_FILE_SIZE_LIMIT test: Skip the assert_ISO_FILE_SIZE_LIMIT function when there is no usable ISO_FILE_SIZE_LIMIT set so the user could specify ISO_FILE_SIZE_LIMIT=0 in his etc/rear/local.conf if he wants to skip the ISO_FILE_SIZE_LIMIT test. Aditionally enforce maximum 2GiB ISO_FILE_SIZE_LIMIT when the MODULES array contains 'loaded_modules' because MODULES+=( udf ) has no effect in this case unless it is loaded (which normally isn't).

  • Error out when files greater or equal ISO_FILE_SIZE_LIMIT should be included in the ISO: See the reasoning in default.conf why the default ISO_FILE_SIZE_LIMIT is 2GiB and why we error out when files >= ISO_FILE_SIZE_LIMIT should be included in the ISO. It is needed to aviod possibly disastrous failures later during "rear recover" that users who exceed that limit in exceptional cases get a hard (error) info to thoroughly test and verify whether or not things actually work for them. See https://github.com/rear/rear/pull/2525

  • Completely overhauled ldd test in 990_verify_rootfs.sh: Now if needed a special LD_LIBRARY_PATH is set only in the bash that runs ldd and that bash exits after ldd finished so the special LD_LIBRARY_PATH setting is gone with it. So there is no need to remember and restore some previously set LD_LIBRARY_PATH because nothing was changed in the bash that runs rear. Furthermore reporting when a binary requires additional libraries happens now in the same 'for' loop after the ldd test was run which further simplifies the code. See https://github.com/rear/rear/pull/2523

  • Skip the ldd test for ReaR files: In build/default/990_verify_rootfs.sh skip the ldd test for ReaR files (mainly bash scripts) where it does not make sense, cf. https://github.com/rear/rear/issues/2519#issuecomment-731196820

  • In 500_clone_keyboard_mappings.sh have more neutral wording in messages about possible keyboard issues and don't be needlessly verbose: It seems newer Debian-based systems (including Ubuntu) no longer contain any keymaps directory as part of the base system by default so including multi-keyboard support into the recovery system is impossible there. If the distro provides console-multi-keyboard support, ReaR includes it (without being verbose). If the distro has decided that this is not necessary, ReaR aligns with it (without being verbose). If the user has installed multi-keyboard support, ReaR aligns with it (without being verbose). Only when including the current keyboard mapping failed (i.e. when 'dumpkeys' failed) it shows subsequent messages on the user's terminal in any case. See https://github.com/rear/rear/issues/2519 and https://github.com/rear/rear/pull/2520

  • Restore LD_LIBRARY_PATH after reporting binaries with 'not found' libraries: In build/default/990_verify_rootfs.sh restore the LD_LIBRARY_PATH after reporting binaries with 'not found' shared object dependencies so that results are the same for the first 'ldd' run that finds binaries with 'not found' shared objects dependencies and the second 'ldd' run that reports those 'not found' shared objects dependencies, see https://github.com/rear/rear/issues/2508#issuecomment-725914211

  • Improved TCG Opal 2 documentation doc/user-guide/13-tcg-opal-support.adoc cf. https://github.com/rear/rear/issues/2511 Better explained OS installation according to the suggestion in https://github.com/Drive-Trust-Alliance/sedutil/issues/301#issuecomment-723010843 Additionally some information from an article on Ask Ubuntu are included https://askubuntu.com/a/1271171/1120528

  • Make recreating LUKS volumes work with optional cryptsetup options: The "cryptseup luksFormat" command does not require any of the type, cipher, key-size, hash, uuid option values because if omitted a cryptseup default value is used, cf. https://github.com/rear/rear/pull/2504#issuecomment-720341023 The right UUID values are mandatory for LUKS volumes that will be mounted during startup of the recreated system. But this does not mean ReaR should error out when there is no cryptsetup uuid value because it is possible to run "rear recover" with enforced MIGRATION_MODE and manually correct the restored /mnt/local/etc/crypttab file to use the new UUIDs before the initrd is recreated and the bootloader is (re)-installed cf. https://github.com/rear/rear/issues/2509

  • Allow setting a PBA-specific firmware configuration via the new OPAL_PBA_FIRMWARE_FILES configuration variable. Add '.../amdgpu/*' firmware files for AMD graphics hardware (if present) on TCG Opal pre-boot authentication (PBA) images, cf. https://github.com/rear/rear/issues/2474

  • Support OPAL 2 self-encrypting NVMe disk drives: Normally the OPAL device itself is used directly but NVMe devices have one or more namespaces per primary device and these namespaces act as disks, cf. https://github.com/rear/rear/issues/2475

  • Added initial LUKS2 support, see https://github.com/rear/rear/issues/2204 Added new parameter 'type' to 'crypt' keyword used in disklayout.conf. Using this parameter allows to recreate the same version of LUKS as on the original system. Added LUKS version detection, parsing depending on version and usage of 'type' parameter.

  • New 036_valid_output_methods.sh to error out for unsupported OUTPUT methods: For "rear mkbackup/mkrescue/mkbackuponly/mkopalpba" (i.e. for all workflows that run the 'prep' stage) check that the OUTPUT method is actually implemented i.e. check that a usr/share/rear/output/$OUTPUT directory exists and error out when an OUTPUT method seems to be not supported to ensure that the user cannot specify a non-working OUTPUT in etc/rear/local.conf (cf. usr/share/rear/prep/default/035_valid_backup_methods.sh) see https://github.com/rear/rear/issues/2501

  • Redirect stdout and stderr to the log only in debug modes. In normal modes stdout and stderr are discarded (go now to /dev/null). The reason is to get rid of messages from command stdout and stderr that don't make sense for the user on their own unless there is more information in the log that provides context about the called command (e.g. what exact command was called in debugscript mode) see https://github.com/rear/rear/issues/2416 and https://github.com/rear/rear/pull/2498

  • More verbose messages when components are excluded so that is is easier for the user to see directly on his terminal what the actual results are when he specified to exclude components in his etc/rear/local.conf and what components are automatically excluded by ReaR. The verbosity depends on if ReaR is run in verbose '-v' mode or even in debug '-d' or '-D' mode.

  • Inform the user about unsupported LUKS2 volumes but do not error out see https://github.com/rear/rear/issues/2491 plus some general code cleanup and improvements in layout/save/GNU/Linux/260_crypt_layout.sh

  • Update 05-integration.adoc: Explain that error messages in ReaR's log may not come from ReaR itself but from programs that are called by ReaR because stdout and stderr are redirected into ReaR's log file, cf. https://github.com/rear/rear/issues/2479

  • Have 'cd WORKING_DIR' in Source() function: Ensure that after each sourced file we are back in ReaR's usual working directory which is the current working directory when usr/sbin/rear is launched that is also the working directory of all the other scripts and config files that get sourced via the Source() function in lib/framework-functions.sh see https://github.com/rear/rear/issues/2461

  • In 400_copy_modules.sh skip copying kernel modules that are builtin modules. The new behaviour is that when modules are listed in modules.builtin and are also shown by modinfo then those modules are now skipped. Before for such modules the modules file(s) would have been included in the recovery system. See https://github.com/rear/rear/issues/2414

  • In backup/NETFS/default/500_make_backup.sh filter purely informational tar messages from output: Suppress purely informational tar messages from output like "tar: Removing leading / from member names" or "tar: Removing leading / from hard link targets" or "tar: /path/to/socket: socket ignored" but keep actual tar error or warning messages like "tar: /path/to/file: file changed as we read it" and show only messages that are prefixed with the name of the failed program (like 'tar:' or 'dd:') which works when tar or dd fail but falsely suppresses messages from openssl which needs to be fixed later, cf. https://github.com/rear/rear/pull/2466#discussion_r466347471

  • Fixed exit code logic in backup/NETFS/default/500_make_backup.sh: Commit https://github.com/rear/rear/commit/2674807f6ad48da80b193ae977d345ad2ef5fdd1 removed the BACKUP_PROG_CRYPT_OPTIONS="cat" dummy code which broke the exit code logic because pipes_rc and backup_prog_shortnames did no longer match. Additionally ensure that the number of elements in both arrays are always the same.

  • Use ReaR specific TMP_DIR (not TMPDIR or hardcoded /tmp) plus some general cleanup work as noticed "by the way" in scripts, see https://github.com/rear/rear/issues/2460 and https://github.com/rear/rear/pull/2462

  • Adaptions and enhancements for BACKUP=CDM because the RBS agent file name changed in Rubrik CDM v5.1 from rubrik-agent-sunos5.10.sparc.tar.gz to rubrik-agent-solaris.sparc.tar.gz see https://github.com/rear/rear/issues/2441

  • Let 'dd' read and write up to 1M=1024*1024 bytes at a time to speed up things cf. https://github.com/rear/rear/issues/2369 and https://github.com/rear/rear/issues/2458

  • In default.conf change ISO_VOLID from "RELAXRECOVER" to "REAR-ISO" so the first ISO has the label "REAR-ISO" (8 characters) and subsequent ISOs get the labels "REAR-ISO_01" "REAR-ISO_02" ... respectively that have 11 characters (the maximum length for FAT volume names) so things work now by default when the ISO image is used to (manually) create a FAT bootable USB stick, cf. https://github.com/rear/rear/issues/1565 and https://github.com/rear/rear/issues/2456

  • OPALPBA: Provide a permanent unlocking mode as a workaround because https://github.com/rear/rear/pull/2426 did not resolve the issue https://github.com/rear/rear/issues/2425 where for an HPE ML10Gen9 server it seems the firmware did not initialize properly during a 'simple' reboot so the only reliable way to boot was a power cycle after Opal disks were unlocked. To reboot with unlocked disks there is now a new unlocking mode "permanent" via OPAL_PBA_UNLOCK_MODE (see default.conf). Reactivating locking is then the responsibility of the user. Additionally enhanced security by disabling shell access via keyboard interrupt and switching to a password hash for OPAL_PBA_DEBUG_PASSWORD.

  • Use single quotes in the password settings examples because single quotes avoid issues with the special bash characters like $ in the password, cf. https://github.com/rear/rear/pull/2178

  • Update format-workflow.sh: For "rear format" show "Use 'rear format - -help' for more information" also when a "rear format" argument is not accepted to help the user what the right syntax is (e.g. "rear format help" and "rear format - help" do not work, only "rear format - -help" works) and provide a more helpful example "rear -v format - -efi /dev/sdX" that is actually neded for UEFI.

  • Support for systemd and parallel restore with Data Protector BACKUP=DP: Support starting Data Protector daemon 'omni.socket' via systemd in the recovery system. Restore performance has been increased by doing parallel restores now. Additional new script usr/share/rear/finalize/DP/default/500_restore_ssc.sh to restore the client certificate when Data Protector Secure Communication is used. Plus several general script cleanup things. See https://github.com/rear/rear/pull/2443

  • Supply default value for root in grub in UEFI ISO: Sometimes the search command in GRUB2 used in UEFI ISO does not find the root device. This was seen at least in Debian Buster running in Qemu (VirtualBox works fine, RHEL/CentOS in Qemu works fine as well). To make ReaR work in this case, set $root to a sensible value before trying search. The GRUB2 image created by grub-mkstandalone has $root set to memdisk, which can't work. This essentially matches how it used to work before https://github.com/rear/rear/pull/2293 and fixes https://github.com/rear/rear/issues/2434

  • Update 300_format_usb_disk.sh: In format/USB/default/300_format_usb_disk.sh when prompting user for size of EFI system partition on USB disk if no valid value is specified also use 400 MiB as default (same as the default value in default.conf) cf. https://github.com/rear/rear/pull/1205#issuecomment-433336977 and https://github.com/rear/rear/commit/89e6a47c82fbfe8d059a4647755f94750fe28acc

  • Replaced $DATE usage with consistent variables START_SECONDS START_DATE_TIME_NUMBER and START_DATE_TIME_STRING see https://github.com/rear/rear/pull/2452

  • Various TCG Opal Improvements: OPALPBA, Ubuntu: Fix incomplete file exclusions: Fixes omissions reported in https://github.com/rear/rear/issues/2436 Uses COPY_AS_IS_EXCLUDE where possible. Removes files from the PBA system's staging directory where not copying could not be configured. OPALPBA: Improve boot splash on Ubuntu 20.04: Takes into account a change from VT1 to VT7 to hide log messages during the boot process. Improves Plymouth boot splash usage to hide log messages during the reboot after unlocking disks. opaladmin: Add sub-commands 'deactivate', 'reactivate': Makes turning on and off hardware encryption easier on provisioned drives. TCG Opal-2: simplify quoting in messages and replace 'StopIfError' use cf. https://github.com/rear/rear/commit/daf35e235d0770c663ff8dba866dddec76586a27 Explicitly issuing an unlock command before reactivating locking ensures that the device remains in an unlocked state when locking is reactivated.

  • Add an explanatory comment in lib/_input-output-functions.sh that using the ...IfError functions can result unexpected behaviour in certain cases cf. https://github.com/rear/rear/pull/2443#pullrequestreview-440609407 and https://github.com/rear/rear/issues/534 and https://github.com/rear/rear/issues/1415#issuecomment-315692391

  • Migrate XFS configuration files. When in MIGRATION_MODE, migrate/rename XFS configuration files so they follow disk mapping set by user. See https://github.com/rear/rear/issues/2333

Supported and Unsupported Operating Systems

We try to keep our wiki page Test Matrix ReaR 2.7 https://github.com/rear/rear/wiki/Test-Matrix-ReaR-2.7 up-to-date with feedback we receive from the community.

ReaR 2.7 is supported on the following Linux based operating systems:

  • Fedora 29, 30, 31, 32, 33, and 34
  • RHEL 6, 7, 8, and 9
  • CentOS 6, 7, and 8
  • Scientific Linux 6 and 7
  • SLES 12 and 15
  • openSUSE Leap 15.x
  • Debian 8, and 9
  • Ubuntu 16, 17, 18 and 20

ReaR 2.7 dropped official support for the following Linux based operating systems:

  • Fedora < 29
  • RHEL < 6
  • CentOS < 6
  • Scientific Linux < 6
  • SLES < 12
  • openSUSE Leap 42.x and before (i.e. openSUSE <= 13)
  • openSUSE Tumbleweed
  • Debian < 8
  • Ubuntu < 16

Usually ReaR 2.7 should also work on newer versions of the above listed supported Linux based operating systems but sometimes arbitrary failures can happen when software that is used by ReaR (like partitioning tools, filesystem tools, bootloader tools, ISO image creating tools, networking tools, and so on) changes in not fully backward compatible ways or when there are innovations of the basic system (like kernel, storage, bootloader, init, networking, and so on) that are not yet supported by ReaR.

In theory ReaR 2.7 should work on openSUSE Tumbleweed but in practice arbitrary failures could happen at any time because the Tumbleweed distribution is a pure rolling release version of openSUSE containing the latest stable versions of all software (cf. https://en.opensuse.org/Portal:Tumbleweed) so arbitrary changes of any software are possible at any time that could arbitrarily break how ReaR works.

ReaR 2.7 may still work for SLES 11 and openSUSE Leap 42.x or even earlier openSUSE versions but it is no longer sufficiently well tested there so arbitrary regressions could appear. For example recreating LUKS encrypted volumes does no longer work with SLES11 SP4 because cryptsetup 1.1.3 in SLES11 SP4 does not support UUID and some other cryptsetup options that are used by default in ReaR 2.7.

ReaR 2.7 and earlier versions are known to no longer work reasonably well for the following Linux based operating systems:

  • RHEL 5 (and probably also CentOS 5): See issue #1766
  • SLES 9 and 10: See issue #1842

If you require support for unsupported Linux operating systems you must acquire a ReaR support contract.

Requests to port ReaR to another operating system (not Linux) can only be achieved with serious sponsoring.

Supported and Unsupported Architectures

ReaR 2.7 is supported on:

  • Intel x86 type of processors
  • AMD x86 type of processors
  • PPC64 processors
  • PPC64LE processors

ReaR 2.7 may or may not work on:

  • Intel Itanium processors
  • ARM type of processors
  • IBM Z "s390x" type of processors

ReaR 2.7 does not support:

  • old PPC (32bit) processors

If you feel the need to get a fully functional ReaR working on one of the above mentioned types of processors please buy consultancy from one of our official developers.

Supported ReaR versions

ReaR has a long history (since 2006) and we cannot support all released versions. If you have a problem we urge you to install the latest stable ReaR version or the development version (available on GitHub) before submitting an issue.

We understand that it is not always possible to install on hundreds of systems the latest version so we are willing to support previous versions of ReaR when you buy a support contract. We cannot handle the big amount of support requests anymore on a voluntary base and we must give paid projects priority, therefore, we urge our customers to buy a support contract for one or more systems. You buy time with our core developers.