This document contains the release notes for the open source project Relax-and-Recover.
This document is distributed with the following license: “Creative Commons Attribution-NoDerivs 3.0 Unported (CC BY-ND 3.0)”. To read the license deed go to http://creativecommons.org/licenses/by-nd/3.0/
Relax-and-Recover is a GNU/Linux system administrator tool used to create disaster recovery images which makes bare metal restore easier. System administrators use Relax-and-Recover as part of disaster recovery policy which does not replace in any way a good backup policy.
The following features are supported on the most recent releases of Relax-and-Recover. Anything labeled as (NEW!) was added as the most recent release. New functionality for previous releases can be seen in the next chapter that details each release.
The most recent release of Relax-and-Recover is supported on most GNU/Linux based systems with kernel 2.6 or higher. It provides the following functionality:
Hot maintenance capability. A rescue image can be made online while the system is running.
Command line interface. Relax-and-Recover doesnot require a graphical interface to run, nor in creation mode, nor in rescue mode (console is enough).
Support included for most common file systems, such as ext2, ext3, ext4 and reiserfs. Other filesystems like jfs, xfs and btrfs are also implemented, but are less tested. (Feedback is appreciated)
Selected Hardware RAID and (eg. HP SmartArray) and mirroring solutions (eg. DRBD) are supported.
LVM root volumes are supported.
Multipath support for SAN storage
UEFI support
Integrates with internal backup solutions such as:
Integrates with external backup solutions such as:
Udev support (except for some really ancient udev versions) which is the base for many new and important features:
System reconfiguration
Support backup software: Bacula, both locally attached tapes (with bextract) and network-based backups. Also, in combination with OBDR tapes.
Create OBDR tapes with method mkbackup and put the backup onto the tape
to have a single-tape bootable recovery solution
Label the OBDR tape with the method format to avoid accidental
overwrites with OBDR
Create bootable disk (eSATA, USB …) media with the backup included:
BACKUP_URL=usb:///dev/device
Together with OUTPUT=USB we have now a complete solution on hard disks
(booting of it and restoring data).
DHCP client support (IPv4 and IPv6). Dhcp client activation
can be forced via the variable USE_DHCLIENT=yes (define in /etc/rear/local.conf).
It is also possible to force DHCP at boot time with kernel option dhcp
Save layout and compare layouts for easy automation of making Relax-and-Recover snapshots (checklayout option)
The layout workflow is the default workflow instead of the deprecated dr workflow. The dr workflow kept all important system information into a directory structure where the new layout workflow use files to keep the information centralized. The dr workflow has been removed in rear v1.14!
External USB booting now uses extlinux instead of syslinux, and therefore, the USB disk must first be formatted with an ext2, ext3, ext4 or btrfs based file system
cron job to check changes in disk layout and trigger rear mkrescue if required
VLAN tagging support (New!)
Add timestamp of rear run with rc code to the syslog or messages file; sending mail report is also possible
NOTE: Features marked experimental are prone to change with future releases.
The first release of Relax-and-Recover, version 1.0, was posted to the web in July 2006. For each release, this chapter lists the new features and defect fixes. Note that all releases are cumulative, and that all releases of Relax-and-Recover are compatible with previous versions of Relax-and-Recover, unless otherwise noted.
The references pointing to fix #nr or issue #nr refer to our issues tracker
The validate rule for xarg bash -n changed into xarg -n bash -n so that rear is working correctly with older bash version 3 as well. Especially required for SLES 10, SLES 11 and EPEL 5 (issue #410).
Adding an updated rear man page and the correct release notes to the 1.16.1 sub-release. In rear 1.16.0 these were forgotten.
Code style improvements (issue #409).
Improvements around btrfs file system recognition and re-creation (issues #408 and #415).
Important fix for grub2 with OpenSuse 13 (changed LOADER into LOADER_TYPE).
Added example prep script to verify if extlinux is present (issue #250) - this type of script could be cloned for other important executables in various workflows.
As usual lots of bug fixes - see the issue tracker.
Network script now deals correctly with VLAN tagging as well
Added 2 new variables to default.conf for TSM: TSM_ARCHIVE_MGMT_CLASS which defines a backup class to use with TSM, and TSM_RM_ISOFILE which triggers (if set to y) to remove the original ISO files from local system once the ISO image has been transferred to TSM
Guessing the bootloader and saving this in a file in the $VAR_LIB/recovery/bootloader (will be picke dup during recovery to assist us in selecting the proper bootloader [ grub, grub2, uefi ])
Various corrections with BACKYP=RSYNC method (issue #200 and #366). In rear 1.16 the old and new style are still recognized, but we might drop the old style in later versions.
OLD STYLE:
BACKUP_URL=[USER@]HOST:PATH # using ssh (no rsh)
with rsync protocol PATH is a MODULE name defined in remote /etc/rsyncd.conf file
BACKUP_URL=[USER@]HOST::PATH # using rsync
NEW STYLE:
BACKUP_URL=rsync://[USER@]HOST[:PORT]/PATH # using ssh
BACKUP_URL=rsync://[USER@]HOST[:PORT]::/PATH # using rsync
Checking the bootloader files during savelayout section - see issue #234
Backup method BACKUP=DUPLICITY has been fully tested and proven to work well (backup and recover). When using duply and automated recovery is possible. Several updates were made to make duplicity works as well with backup and recover
Variable SSH_ROOT_PASSWORD (see default.conf) allows to force a password with the rescue image (issue #362)
TSM version >=6.4 is now supported (issue #356), and point in time restore works better now (issue #358)
Fix the order of fs lines in checklayout.conf file when mor ethen 9 partitions are in use (issue #352)
Avoiding CD/DVD type devices to be added in the disklayout.conf file during savelayout (issue #257)
Removed mingetty executable from REQUIRED_PROGS list as RHEL 7 does not ship it anymore. agetty is getting more and more the standard now and rear will use this one automatically when mingetty is absent
New variable in default.conf file was defined BACKUP_TYPE= (empty is full or incremental; only works with the default BACKUP_PROG=tar)
Avoid stale NFS hangs with df command (issue #350)
For TSM a new variable was introduced in default.conf file: TSM_RESULT_SAVE (to disable the TSM saving; can be used when you saved the result via $ISO_URL, $OUTPUT_URL, $ISO_DIR or something else)
/etc/rear/local.conf is now empty by default and /etc/rear/site.conf now contains OUTPUT=ISO setting. This way we are sure that during an upgrade of rear we do not loose our original user settings (local.conf tends te be overwritten).
Fix for hpacucli where the order logicaldrive and array could be wrong (issue #208)
New variable TMPDIR allows to overrule the default /tmp temporary directory (handy when building big ISO images)
Added option -iso-level 3 to mkisofs command so we can create ISO image bigger then 4 GB (issue #323)
detecting ‘out of space’ errors while building huge ISO images
Add SEP Sesam external backup integration - issues #324 and #325
Add automatic recovery mode for Bareos - issue #311
Add incremental backup type with GNU tar with weekly backups, define in /etc/rear/local.conf the following (issue #294):
BACKUP=NETFS
BACKUP_TYPE=incremental
FULLBACKUPDAY="Mon"
Add support for Gentoo kernels - issue #312
The references pointing to fix #nr or issue #nr refer to our issues tracker
As usual lots of bug fixes - see the issue tracker.
Add EMC NetWorker (Legato) support for doing external backup
Add Bareos (Backup Archive REcovery Open Sourced) support for doing external backup (bareos is a fork of the open source backup suite Bacula)
Align the usage of KEEP_OLD_OUTPUT_COPY and NETFS_KEEP_OLD_BACKUP_COPY and make them behave sane. See also issue #192
Add the –selinux option to be safe with SELinux context restoration (with GNU tar)
Add an option to exclude modules on the rescue system
default.conf: EXCLUDE_MODULES=()
Add integrity check when BACKUP=NETFS and BACKUP_PROG=tar with the option BACKUP_INTEGRITY_CHECK
Add an option to define a root password to allow SSH connection whithout a public/private key pair (#272)
default.conf: SSH_ROOT_PASSWORD=
Implement the “splitted backup to iso” feature. To use that, BACKUP_URL has to use the iso://[subdirs] scheme but a different OUTPUT_URL has to be defined (issues #278 and #287). New parameter ISO_MAX_SIZE introduced.
Fix SELinux autorelabelling (issue #270 and #274)
Rear supports now more then 9 partitions (see issue #263)
systemd support added for Fedora 19/20 and OpenSuse 12.x/13.x
Automatic Recover feature on USB devices (label AUTOMATIC RECOVER in rear recover menu)
Add an option to copy all users and group. It is usefull in the case we want to restore ACL of all users. See default.conf:
CLONE_ALL_USERS_GROUPS=n
Example prep script to detect missing lftp (fix #247)
Using POSIX output format in df.txt (fix #248)
Recognize OracleServer as a Fedora derivate
Allow basic network configuration from the command line
Create directory $PXE_TFTP_LOCAL_PATH if it does not exist (fix #244, issue #244)
Support booting syslinux v5 (issue #238)
Fix installing grub when /boot is inside the root filesystem (issue #210)
Deal with BTRFS subvolumes correctly (issue #233 and #252)
Recognize OracleServer as a Fedora derivate
Added UEFI integration within rear (including booting ISO images)
Improved dataprotector GUI interaction (issue #189)
Secure backup was introduced (issue #196)
Add to /etc/rear/local.conf: BACKUP_PROG_CRYPT_ENABLED=1 BACKUP_PROG_CRYPT_KEY=”secretkey”
Added fixes recomended by RedHat Bugzilla report #882175
OUTPUT_URL scheme approvement (issue #37)
Netbackup improvement for xinetd (issue #180)
LOGFILE variable can be overruled now (issue #56)
Support for RBME restores over NFS
Packaging support introduced for Arch Linux
Added support for sshfs transport with BACKUP=NETFS (issue #171)
Re-introduced /dev/disk/by-id migration which was lost since release 1.7.22 (issue #22)
Updates required for systemd for fedora 18 and 19.
Check if ROOT_FS filesystem was mounted with ‘noexec’ attribute (issue #150)
The references pointing to fix#nr refer to our issues tracker
Lots of minor bugs were fixed, and not all of them are listed in current release notes. For a complete overview see the issue tracker.
Added duply/duplicity with one new backup method duplicity (Experimental)
Systemd supported on Fedora 17 and OpenSuse 12.2 (fix #115, #126)
Create correct yaboot dir on ppc (fix #109)
Add new RAMDISK output method. This writes the kernel and the initramfs to the location given in OUTPUT_URL.
Packaging - introduction of Makefile; cleanup dr workflow (fix #13, #49)
Make rear working from checkout of git, w.o.w. path is relocatable (fix #53)
Add bytes_per_inode information for ext* filesystem to layout.conf (fix #86)
Fixing ebuild to be Gentoo compliant (fix #93)
fix shutdown with upstart (fix #41)
multiarch support library support (fix #82)
Fix serial console on ubuntu 11.04 (fix #83)
Several fixes in layout file (fix #85)
Added fix for DAT320 tape drive (fix for #35)
Use generic grub code for all distributions (fix for #77)
list Xen paravirtualized disks in disklayout.conf (fixes #74 and SF3520992)
Support for multipathing was added
Several improvements and bug fixes to the layout code (especially with parted backwards compatibility).
Added support for ext4 file systems
The OUTPUT=USB with BACKUP=NETFS and BACKUP_PROG=rsync was corrected to avoid duplicate work
Fedora and RHEL will now rebuild the initial ramdisk if needed (on recovered system)
Fix for SF#3475480: datacompression on tape
Fix for SF#3481656: missing bacula-console executable for BACKUP=BACULA workflow
Fix for SF#3479570: /etc/passwd contains :x: without /etc/shadow
Added “migrate HWADDR after cloning” code
Improved the systemd code (parts required by Relax-and-Recover only) for Fedora 16/17
The DHCLIENT variables were moved from local.conf to rescue.conf. This is done automatically, so the end-user, shouldnot be aware of it.
At boot time more kernel options are recognized such as noip, dhcp,
debug. The noip will give you a rescue environment without any attempt to
start networking. The dhcp variable will try to start dhclient on any
network interface it finds activated. The debug variable (which is not new
by the way) will give you the chance to debug the code of our
Relax-and-Recover code.
Relax-and-Recover works again on IA64 architecture (at least with RHEL 5.x). Remember, RHEL 6 is not ported to IA64.
Multiple copies of Relax-and-Recover backups (of the same or different
systems) can be kept on a USB device (with OUTPUT=USB).
(NEW!) BACKUP=RSYNC workflow using rsync executable. Both ssh and rsync methods are supported. E.g.
BACKUP=RSYNC
OUTPUT=ISO
BACKUP_URL=rsync://username@hostname/path
BACKUP_PROG=/usr/local/bin/rsync #(instead of the default rsync)
EXCLUDE_ variables, better control over what is restored:
EXCLUDE_BACKUP excludes components from backupEXCLUDE_RECREATE excludes components from the recreate processEXCLUDE_RESTORE excludes components from the restore processThe layout workflow is now the default instead of the dr workflow. Under /var/lib/rear/layout all information of the system is kept in files.
Arch Linux is now supported with Relax-and-Recover.
The labeltape command has been superseded by the format command. This can be used with tapes and external (USB, eSATA) devices. Usage:
rear format [/dev/st0|/dev/sdx]
Replaced NETFS_URL and ISO_URL by BACKUP_URL and OUTPUT_URL. However, old references will still be recognized and used.
Fedora 16 is supported including GRUB 2, and systemd as init replacement.
Added the BACKUP_URL=file:///PATH with BACKUP=NETFS method (as described in configuration-examples.txt)
Improved multipath functionality
Optional automatic autofs exclusion
The mkobdr command has been removed. OBDR-enabled tapes can now be created using the mkrescue command and by defining the proper variables in /etc/rear/local.conf:
BACKUP=NETFS
OUTPUT=OBDR
BACKUP_URL=tape:///dev/nst0
The site configuration file /etc/rear/site.conf has been removed from the Relax-and-Recover package, but can still be used if end-users want. The purpose of this is to enable sites to distribute this file through RPM or DEB packages that do not have a file conflict with the Relax-and-Recover package. The distribution /etc/rear/local.conf file contains only configuration examples as comments in order to not interfere with configurations in site.conf.
The rear command is by default quiet, which means if you want the same
behavior as in previous versions you need to add the verbose option (-v)
with the rear command
The output workflow now runs before the mkbackup workflow especially done to make OBDR tape creation possible with the mkbackup workflow as the ISO image must be written first on an OBDR aware tape. Please note that this is a fundamental change with regard to previous versions of Relax-and-Recover. While utmost care has been taken that there would be no adverse side effects of this change. We cannot test all possible usage scenarios.
When using OUTPUT=USB then you have to make sure that the destination (USB)
disk is formatted as an ext2, ext3, ext4 or btrfs file-system. Extlinux is
now the only supported boot loader for bootable disks, syslinux is not
supported any more.
The Relax-and-Recover boot now shows a boot menu with options to choose from. The actual content of the menu depends on the available syslinux version and its modules (like menu.c32, hdt.c32, reboot.c32, poweroff.com).
Relax-and-Recover does properly recognize IPv6 addresses and uses these if configured.
NBU backup method now allows to restore to a point in time.
Support Fedora 15 (using systemd to boot-up) and RHEL6 and Scientific Linux 6.
Improved handling of HP SmartArray controllers.
Significantly improved error handling, especially when failing on subshells.
Autologin as root in the rescue media (for upstart and systemd based systems).
EXCLUDE_MOUNTPOINTS should work correctly now (fixed typo).
Support ext4 on RHEL5 and clones.
Ignore known errors when using EXTERNAL backup method (set
EXTERNAL_IGNORE_ERRORS to an array of return codes to ignore).
Use original filesystem mount options for recovery, support attr and facl tools.
Support XEN paravirtualized systems (tested only on RHEL5 so far).
Performance improvements (removed checksum calculation, PID-based locking).
Relax-and-Recover work space is now created with a random part to prevent potential security exploits.
Control exit tasks and subprocesses better. Kill subprocesses before exiting.
Support adding Relax-and-Recover boot files to local GRUB environment
(GRUB_RESCUE) and password protect rescue boot (GRUB_RESCUE_PASSWORD)
to avoid accidential recovery.
The default password is REAR.
(experimental!) Transfer ISO image to remote URL (ISO_URL). Please note
that this feature will be extended to cover all output methods. It has been
renamed to OUTPUT_URL.
Removed various warnings, e.g. about NETFS not being a professional backup method.
Partial support for Arch Linux has been added, more testing required.
(NEW!) shell workflow is now really usable.
Make 32/64 bit handling much more robust, especially on systems having /lib32.
NETFS backup and restore with rsync working now (BACKUP_PROG=rsync).
Support udev on RHEL4.
Development snapshot have now a version like 0.0.REV where REV is the SVN revision used to build the development snapshot.
Greatly reduced log clutter (lvm warnings about leaked file descriptors, which is a bash bug, various irrelevant error and verbose output).
checklayout can now also check arbitrary files (through an md5 checksum),
extend the CHECK_CONFIG_FILES array to use this feature.
An intermediate release only which fixed some hanging issues of version 1.9.0. Also, a RPM upgrade was fixed by this release from 1.7.25 to 1.9.0, which failed because of a wrongly CentOS symbolic link. See bugzilla#680664
With version 1.9.0 some new methods were added, such as:
rear mkobdr : to create an OBDR recovery tape (obsolete since 1.11.0)
rear labeltape : goes together with OBDR tapes. To avoid accidental
overwrites we force the creation of a label before rear mkbackup will
work. (obsolete since 1.11.0)
rear checklayout/savelayout : a new method to save the disk layout
and check if a new rear mkbackup or rear mkrescue is required.
New BACKUP methods were added, Bacula (BACKUP=BACULA) and bextract
(BACKUP=BEXTRACT), both are able to work in conjunction with
output=TAPE. See under the doc directory (or /usr/share/doc/rear-1.9.0/)
the configuration-examples.txt text file for beginners instructions.
OUTPUT=USB method has been extended with BACKUP=NETFS and
NETFS_URL=usb:///dev/<device> which makes it possible that the
complete archive is stored on the /dev/<device> and
Relax-and-Recover will make the USB stick (or disk) bootable too.
Udev support (except for some really ancient udev versions) which is the base for many new and important features, like kernel drivers for network cards and storage adapters are now loaded via udev, or deal with network persistent names in udev rules, and firmware loading.
DHCP client support (IPv4 and IPv6) has been added. Auto detection
is possible with new variable USE_DHCLIENT=yes (define in local.conf),
or one can hard-code your special DHCP client with the variables
DHCLIENT_BIN (for IPv4), and/or DHCLIENT6_BIN (for
IPv6).
Relax-and-Recover version 1.9.0 contain fixes for the following defects:
Missing support for Scientific Linux, LinuxMint
Sourceforge patch ID 2963804 - support for USBFS, but this patch has been
rewritten afterward to incorporate usb support into the NETFS backup method,
instead of having a separate USBFS backup method. Now, by using
NETFS_URL=usb:///dev/<device> and the NETFS backup method we achieve the
same result.
Sourceforge bug 3153027 : support for RHEV virtio device files
Novell bugzilla 581292 : cannot load NIC firmware because of missing udev support. Version 1.9 does have udev support, but firmware loading was broken. The rule in 00-rear.rules has been changed.
Relax-and-Recover version 1.7.26 fixed RedHat bugzilla defect 657174 : rescue image freezes during the boot while executing init. This was due the new upstart mechanism (replaced the sysv init procedure).
Relax-and-Recover version 1.7.25 fixed RedHat bugzilla defect 600217 : Fedora link missing in restore, pack and finalize sub-directories. This broke the proper image building on several Fedora versions.
Relax-and-Recover works on GNU/Linux kernel with version 2.6 and higher. For lower kernel versions Relax-and-Recover cannot be used, and for these systems, mkcdrec is still a good alternative.
As Relax-and-Recover has been solely written in the bash language we need the bash shell which is standard available on all GNU/Linux based systems. The default backup program Relax-and-Recover uses is GNU/tar which is also standard available.
Relax-and-Recover is known to work well on x86 and x86_64 based architectures.
Relax-and-Recover has also been ported to ia64 and ppc architectures, but
these are less tested. Use the ‘rear validate’ command after every
successful DR test please and mail us the results.
The default backup program with Relax-and-Recover is (BACKUP_PROG=tar)
GNU tar and the default compression used with tar is gzip. However, is
using gzip the best choice? We have done some tests and published the
results. See
Relax-and-Recover compression tests
Issue Description: Is there a possibility to add btrfs subvolume to a rsync backup
At present (release 1.16.1) there is no workaround in place. If you happen to know how this could be fixed then add your ideas to issue #417
Issue Description: with RHEL 4 systems extended partitions are recognized as primary partitions
Error: Expecting a partition number
Full details can be read in issue 314.
However, you need to edit /var/lib/rear/layout/disklayout.conf file and change for
partition 4 “primary” to “extended”
Issue Description: System reconfiguration still has some weaknesses.
this has to be tested before relying on it, there are too many unknowns involved so that we cannot guarantee anything in this area. It has been developed mostly as a P2V tool to migrate HP servers to VMware Vms
hard disks need to be at least of the same size and amount as in the original system, ATM this is a simple 1:1 mapping of old to new disks, there is no removal of RAID groups or merging of smaller disks onto a bigger one or making stuff smaller.
any use of /dev/disk/by-path or /dev/disk/by-id is untested and will most likely not work. In some cases Relax-and-Recover will print a warning, but we are not able to detect all cases. Typically this leads to unbootable systems or bad /etc/fstab files
Issue Description: The DHCP client is still a little rough around the edges, especially with complex networking scenarios.
Issue Description: An error is encountered while upgrading rear-1.7.* to rear-1.9.0:
error: unpacking of archive failed on file /usr/share/rear/finalize/CentOS: cpio: rename failed - Is a directory
First remove the older Relax-and-Recover version by hand and then install
the new version. The local.conf is saved (as local.conf.rpmsave) when
we execute rpm -e rear
Issue Description: If SELinux is not disabled during backup (variable
BACKUP_SELINUX_DISABLE= in /etc/rear/local.conf) then we might see
errors in the rear-$(hostname).log file such as:
tar: var/cache/yum/i386/15/updates/packages: Cannot setfilecon: No such file or directory
Make sure the BACKUP_URL destination understands extended attributes
(CIFS is out of the question and NFS is problematic). When using local
disks (or external USB devices) make sure the proper mount options are
given in the BACKUP_OPTIONS variable, e.g.:
BACKUP_OPTIONS="rw,relatime,seclabel,user_xattr,acl,barrier=1,data=ordered"
(TIP) BACKUP_SELINUX_DISABLE=1 variable has been introduced in the
/usr/share/rear/conf/default.conf file to disable SELinux
while the backup is running (default setting).
Issue Description: Is incremental backup possible? With our default
settings (BACKUP=NETFS and BACKUP_PROG=tar) we do not yet support
incremental backups.
However, when we change BACKUP_PROG=rsync we can use rear mkbackuponly
option which is in fact an incremental backup using the rsync program.
The same can be accomplished by using BACKUP=RSYNC and the proper
BACKUP_URL=rsync://hostname/PATH.
Issue Description: Error “Missing usr/lib/systemd/system - too confused to continue”
Above error message might be seen after upgrading rear from a very old version (like rear-1.10) to the latest one.
Use the command rpm -V rear to verify the upgrade went fine. If not, then remove first the old version (with rpm -e rear) before installing the latest version.
Issue Description: ERROR: FindStorageDrivers called but STORAGE_DRIVERS is empty
Above error message might be seen after a fresh installation of the GNU/Linux kernel. Rear got confused between the running kernel version number and the actual fresh kernel available.
Reboot your server before using rear again, which is a good practice anyway after upgrading the GNU/Linux kernel.