#3190 Issue `closed`: Feature Idea: Create portable rear recover solution OUTPUT=PORTABLE¶

Labels: enhancement, sponsored

schlomo opened issue at 2024-04-03 13:36:¶

DeaR @rear/contributors I'd like to collect your thoughts on the following proposal to create a portable rear recovery solution.

Problem¶

The problem to solve is to use rear recover based on another (non-ReaR) rescue system instead of the ReaR rescue system. The reasons are due to corporate requirements of a customer environment or due to software compatibility problems between the ReaR rescue system and proprietary backup software.

Solution Approach¶

The idea is to create a new OUTPUT=PORTABLE method that will produce an "ReaR Portable Recovery" archive containing

/etc/rear
/usr/share/rear
/usr/sbin/rear
/var/lib/rear

For recovery the user would to the following:

start the recovery system from their rescue solution, e.g. the System Rescue CD
configure the system to work in the network and have the same identity as the original system ($HOSTNAME etc.)
install - if needed - the proprietary backup agent or other software required for the recovery
unpack the ReaR portable bundle to /
run rear recover as usual

I expect users to potentially run into trouble with the automated migration of hardware IDs for network, but for simple cases I expect this to "just work".

What do you think? What did I forget?

jsmeix commented at 2024-04-03 13:57:¶

@schlomo
I like the generic idea very much.

My reason (as far as I understand it) is that
with OUTPUT=PORTABLE it is possible to separate what belongs to
the ReaR recovery installer (i.e. what "rear recover" needs)
from what belongs to ReaR's bootable rescue system
so implementing it will help to get our code cleaner
via Keep Separated Items Separated ("KSIS").

jsmeix commented at 2024-04-03 14:09:¶

Only an offhanded thought FYI
regarding

4. unpack the ReaR portable bundle to /

I wonder if this may overwrite things
in the non-ReaR rescue system?

Therefore I am wondering if it is possible
that also "rear recover" could run from within any directory
as "rear mkbackup" can, e.g. in a git checkout directory like

git clone https://github.com/rear/rear.git
cd rear
vi etc/rear/local.conf
usr/sbin/rear mkbackup

If this is possible, then with OUTPUT=PORTABLE
it could be possible to

4. unpack the ReaR portable bundle to any directory
   and change into that directory
5. in that directory run "usr/sbin/rear recover"

What I mean is to get what belongs to
ReaR's recovery installer (i.e. what "rear recover" needs)
separated from what belongs to the non-ReaR rescue system.

The idea behind is that what "rear recover" needs
could be different what there is in the non-ReaR rescue system,
e.g. "rear recover" may need certain specific binaries
like the exact binaries from the original system
and not whatever binaries in the non-ReaR rescue system.

Or in other words:
How to ensure that what there is in the non-ReaR rescue system
is what "rear recover" needs?

schlomo commented at 2024-04-05 17:15:¶

Update: Started to work on this in portable-recovery branch. After some twiddling I got this far with a SystemRescueCD.

In this test it failed to install the boot loaded because ReaR thinks it is running on Arch Linux (SysRescueCD) and not the restored OS.

Looks promising, but I need to make the system read the os.conf stuff from the recovery info and not from the rescue system. Good opportunity to finally get rid of the os.conf in /etc/rear 😃

Your thoughts are welcome.

# hostnamectl hostname rear-ol8u7
# REAR_VAR=/tmp/rear_var ./usr/sbin/rear -p -d recover
Relax-and-Recover 2.7 / Git
Running rear recover (PID 28794 date 2024-04-05 17:06:59)
Command line options: ./usr/sbin/rear -p -d recover
Using log file: /tmp/rear_var/log/rear/rear-rear-ol8u7.log
Using build area: /var/tmp/rear.cKm2s52s4lNQrlg
Setting TMPDIR to '/var/tmp/rear.cKm2s52s4lNQrlg/tmp' (was unset when ReaR was launched)
Running 'init' stage ======================
Running 'setup' stage ======================
Running 'verify' stage ======================
Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
Started RPC portmapper 'rpcbind'.
RPC portmapper 'rpcbind' available.
RPC status rpc.statd available.
Started rpc.idmapd.
Using backup archive '/var/tmp/rear.cKm2s52s4lNQrlg/outputfs/rear-ol8u7/backup.tar.gz'
Will do driver migration (recreating initramfs/initrd)
Calculating backup archive size
Backup archive size is 3.2G     /var/tmp/rear.cKm2s52s4lNQrlg/outputfs/rear-ol8u7/backup.tar.gz (compressed)
Running 'layout/prepare' stage ======================
Comparing disks
Device sda has size 13958643712 bytes but 12884901888 bytes is expected (needs manual configuration)
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 12884901888 (12 GiB) but is now 13958643712 (13 GiB)
/dev/loop0 was not used on the original system and has now 717123584 (0 GiB)
Could not automap /dev/sda (no disk with same size 12884901888 found)
Original disk /dev/sda does not exist (with same size) in the target system
Using /dev/sda (the only available of the disks) for recreating /dev/sda
Current disk mapping table (source => target):
  /dev/sda => /dev/sda

UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /home/schlomo/src/rear/usr/share/rear/layout/prepare/default/300_map_disks.sh line 334
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Confirm identical disk mapping and proceed without manual configuration
3) Edit disk mapping (/tmp/rear_var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /tmp/rear_var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /tmp/rear_var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /tmp/rear_var/lib/rear/layout/config/df.txt
Trying to automatically resize last partition when disk size changed
Examining msdos device /dev/sda to automatically resize its last active partition
New /dev/sda is 1073741824 bytes bigger than old device
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Found 'primary' partition /dev/sda2 as last partition on /dev/sda
Determining if last partition /dev/sda2 is resizeable
Determining new size for last partition /dev/sda2
Determining if last partition /dev/sda2 actually needs to be increased or shrinked
Skip increasing last partition /dev/sda2 (new device less than 10% bigger)
UserInput -I LAYOUT_FILE_CONFIRMATION needed in /home/schlomo/src/rear/usr/share/rear/layout/prepare/default/500_confirm_layout_file.sh line 26
Confirm or edit the disk layout file
1) Confirm disk layout and continue 'rear recover'
2) Edit disk layout (/tmp/rear_var/lib/rear/layout/disklayout.conf)
3) View disk layout (/tmp/rear_var/lib/rear/layout/disklayout.conf)
4) View original disk space usage (/tmp/rear_var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disk layout and continue 'rear recover''
User confirmed disk layout file
Marking component '/dev/sda' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component '/dev/sda1' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component '/dev/sda2' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component 'pv:/dev/sda2' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component '/dev/ol_rear-ol8u7' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/ol_rear--ol8u7-swap' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/ol_rear--ol8u7-root' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component 'fs:/' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component 'fs:/boot' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Marking component 'swap:/dev/mapper/ol_rear--ol8u7-swap' as done in /tmp/rear_var/lib/rear/layout/disktodo.conf
Running 'layout/recreate' stage ======================
UserInput -I LAYOUT_CODE_CONFIRMATION needed in /home/schlomo/src/rear/usr/share/rear/layout/recreate/default/100_confirm_layout_code.sh line 26
Confirm or edit the disk recreation script
1) Confirm disk recreation script and continue 'rear recover'
2) Edit disk recreation script (/tmp/rear_var/lib/rear/layout/diskrestore.sh)
3) View disk recreation script (/tmp/rear_var/lib/rear/layout/diskrestore.sh)
4) View original disk space usage (/tmp/rear_var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disk recreation script and continue 'rear recover''
User confirmed disk recreation script
Disks to be completely overwritten and recreated by /tmp/rear_var/lib/rear/layout/diskrestore.sh:
  /dev/sda 
Determining disks to be wiped ...
UserInput -I WIPE_DISKS_CONFIRMATION needed in /home/schlomo/src/rear/usr/share/rear/layout/recreate/default/120_confirm_wipedisk_disks.sh line 168
Disks to be wiped: /dev/sda 
1) Confirm disks to be wiped and continue 'rear recover'
2) Skip wiping disks and continue 'rear recover'
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disks to be wiped and continue 'rear recover''
User confirmed disks to be wiped
Removed LVM physical volume /dev/sda3
Wiping child devices of /dev/sda in reverse ordering: /dev/sda3 /dev/sda2 /dev/sda1 /dev/sda 
Wiped first 16777216 bytes of /dev/sda3
Wiped last 16777216 bytes of /dev/sda3
Wiped first 16777216 bytes of /dev/sda2
Wiped last 16777216 bytes of /dev/sda2
Wiped first 16777216 bytes of /dev/sda1
Wiped last 16777216 bytes of /dev/sda1
Wiped first 16777216 bytes of /dev/sda
Wiped last 16777216 bytes of /dev/sda
Start system layout restoration.
Disk '/dev/sda': creating 'msdos' partition table
Disk '/dev/sda': creating partition number 1 with name 'primary'
Disk '/dev/sda': creating partition number 2 with name 'primary'
Creating LVM PV /dev/sda2
Creating LVM VG 'ol_rear-ol8u7' (some properties may not be preserved)
Creating LVM volume 'ol_rear-ol8u7/swap' (some properties may not be preserved)
Creating LVM volume 'ol_rear-ol8u7/root' (some properties may not be preserved)
Creating filesystem of type xfs with mount point / on /dev/mapper/ol_rear--ol8u7-root.
Mounting filesystem /
Creating filesystem of type xfs with mount point /boot on /dev/sda1.
Mounting filesystem /boot
Creating swap on /dev/mapper/ol_rear--ol8u7-swap
Disk layout created.
Recreated storage layout:
NAME                                KNAME      TRAN   TYPE FSTYPE      LABEL        SIZE MOUNTPOINTS
/dev/loop0                          /dev/loop0        loop squashfs               683.9M /run/archiso/sfs/airootfs
/dev/sda                            /dev/sda   sata   disk                           13G 
|-/dev/sda1                         /dev/sda1         part xfs                        1G /mnt/local/boot
`-/dev/sda2                         /dev/sda2         part LVM2_member               11G 
  |-/dev/mapper/ol_rear--ol8u7-swap /dev/dm-0         lvm  swap                     1.2G 
  `-/dev/mapper/ol_rear--ol8u7-root /dev/dm-1         lvm  xfs                      9.8G /mnt/local
/dev/sr0                            /dev/sr0   ata    rom  iso9660     RESCUE1002   763M /run/archiso/bootmnt
UserInput -I LAYOUT_MIGRATED_CONFIRMATION needed in /home/schlomo/src/rear/usr/share/rear/layout/recreate/default/200_run_layout_code.sh line 168
Confirm the recreated disk layout or go back one step
1) Confirm recreated disk layout and continue 'rear recover'
2) Go back one step to redo disk layout recreation
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm recreated disk layout and continue 'rear recover''
User confirmed recreated disk layout
Running 'restore' stage ======================
Restoring from '/var/tmp/rear.cKm2s52s4lNQrlg/outputfs/rear-ol8u7/backup.tar.gz' (restore log in /tmp/rear_var/lib/rear/restore/recover.backup.tar.gz.28794.restore.log) ...
Restored 6817 MiB [avg. 86180 KiB/sec] OK
Restored 6893 MiB in 82 seconds [avg. 86089 KiB/sec]
Restoring finished (verify backup restore log messages in /tmp/rear_var/lib/rear/restore/recover.backup.tar.gz.28794.restore.log)
Created SELinux /mnt/local/.autorelabel file : after reboot SELinux will relabel all files
Recreating directories (with permissions) from /tmp/rear_var/lib/rear/recovery/directories_permissions_owner_group
Running 'finalize' stage ======================
Checking if certain restored files are consistent with the recreated system
See /tmp/rear_var/lib/rear/layout/config/files.md5sum what files are checked
Restored files in /mnt/local do not fully match the recreated system
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /mnt/local/src/rear/etc/rear/os.conf: FAILED open or read
  /mnt/local/src/rear/etc/rear/site.conf: FAILED open or read
  /mnt/local/src/rear/etc/rear/local.conf: FAILED open or read
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
/etc/multipath.conf not available in target system, creating it...
Copied /etc/multipath/bindings to target system /mnt/local
UserInput -I RESTORED_FILES_CONFIRMATION needed in /home/schlomo/src/rear/usr/share/rear/finalize/default/520_confirm_finalize.sh line 41
Confirm restored config files are OK or adapt them as needed
1) Confirm it is OK to recreate initrd and reinstall bootloader and continue 'rear recover'
2) Edit restored etc/fstab (/mnt/local/etc/fstab)
3) View restored etc/fstab (/mnt/local/etc/fstab)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm it is OK to recreate initrd and reinstall bootloader and continue 'rear recover''
User confirmed restored files
Installing GRUB2 boot loader...
Failed to generate boot/grub2/grub.cfg in /mnt/local - trying to install GRUB2 nevertheless
Determining where to install GRUB2 (no GRUB2_INSTALL_DEVICES specified)
Found possible boot disk /dev/sda - installing GRUB2 there
Failed to install GRUB2 on /dev/sda
Failed to install GRUB2 - you may have to manually install it
WARNING:
For this system
Arch/10.02 on Linux-i386 (based on 10.02/i386)
there is no code to install a boot loader on the recovered system
or the code that we have failed to install the boot loader correctly.
Please contribute appropriate code to the Relax-and-Recover project,
see http://relax-and-recover.org/development/
Take a look at the scripts in /home/schlomo/src/rear/usr/share/rear/finalize - for example
for PC architectures like x86 and x86_64 see the script
/home/schlomo/src/rear/usr/share/rear/finalize/Linux-i386/660_install_grub2.sh
and for POWER architectures like ppc64le see the script
/home/schlomo/src/rear/usr/share/rear/finalize/Linux-ppc64le/660_install_grub2.sh
---------------------------------------------------
|  IF YOU DO NOT INSTALL A BOOT LOADER MANUALLY,  |
|  THEN YOUR SYSTEM WILL NOT BE ABLE TO BOOT.     |
---------------------------------------------------
You can use 'chroot /mnt/local bash --login'
to change into the recovered system and
manually install a boot loader therein.

Running 'wrapup' stage ======================
Finished 'recover'. The target system is mounted at '/mnt/local'.
Exiting rear recover (PID 28794) and its descendant processes ...
Running exit tasks
To remove the build area you may use (with caution): rm -Rf --one-file-system /var/tmp/rear.cKm2s52s4lNQrlg

schlomo commented at 2024-04-12 17:28:¶

Implemented via a9594b08339e32669d29f555b392606425da977f

[Export of Github issue for rear/rear.]

#3190 Issue closed: Feature Idea: Create portable rear recover solution OUTPUT=PORTABLE¶