... must be sacrosanct¶

Labels: enhancement, fixed / solved / done, critical / security / legal, severe improvement

amtuannguyen opened issue at 2017-03-29 16:16:¶

when restore from USB backup disk that is the same size as the source sda, the USB backup disk is selected as the sda to restore to and therefore overwrite itself.

scenario:

Source system has 1 disk /dev/sda, size is 250GB
Insert a USB disk that is exactly 250GB
Run rear format and rear mkbackup
Take the USB disk to a new machine and attempt to restore
The recovery sees that sda (which is now the USB rear backup disk) is exactly the same as the original and selects it automatically. It does not offer a chance to map drives.
Choose continue and sda is now wiped out along with your rear backup image.

amtuannguyen commented at 2017-03-29 17:09:¶

cat /etc/rear/local.conf
MODULES_LOAD=( "${MODULES_LOAD[@]}" hid_generic usbhid )

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

The restore scripts should NOT offer itself as the drive to restore to. This is a potentially serious issue. Imagine restoring a completely dead machine from a backup usb disk and having that usb disk destroyed during the restore process.

jsmeix commented at 2017-03-30 08:12:¶

The generic root cause is that during "rear recover"
"sda ... is now the USB rear backup disk"
but var/lib/rear/layout/disklayout.conf
lists /dev/sda as the disk device node
that has to be used for recreating the system.

Essentially this issue is a generic problem in ReaR.
For some background information see
https://github.com/rear/rear/issues/1057#issuecomment-258832970
and
https://github.com/rear/rear/issues/1057#issuecomment-259444183

Regarding the particular issue that a disk device node
that is listed in var/lib/rear/layout/disklayout.conf
is during "rear recover" a disk device node
that matches /dev/disk/by-label/REAR-000:

I will implement a special protection against
this particular issue in ReaR.

Simply put:
@amtuannguyen you are right, during "rear recover"
any disk device node that matches /dev/disk/by-label/REAR-000
(or more generally that matches BACKUP_URL)
is sacrosanct.

jsmeix commented at 2017-03-30 08:55:¶

@amtuannguyen
as a workaround for now you need to avoid that
while the recovery system boots, the disk device node
that matches /dev/disk/by-label/REAR-000
becomes a disk device node in var/lib/rear/layout/disklayout.conf
that is listed there as target for the recovery.
My untested proposal how to do that is to have the
USB storage not connected while the recovery system boots
and connect it later and check what device node it gets
before you run "rear recover".

gdha commented at 2017-03-30 09:35:¶

@jsmeix OUTPUT=USB means it boots up from USB ;-/

jsmeix commented at 2017-03-30 10:33:¶

@gdha
I do not understand what you mean.
When a (removable) disk (e.g. a USB mass storage device)
is used whereto "rear mkbackup" will write the recovery system
and the backup.tar.gz, then during "rear mkbackup/mkrescue"
that (removable) disk will be (over)written.
The issue here is that this same disk must be sacrosact
during "rear recover".
With sacrosact I mean that nothing will be (over)written
on that disk - i.e. that disk must be used read-only.
I do not mean that one cannot boot from such a disk.
Basically such a disk must be used like a CDROM.

gdha commented at 2017-03-30 11:08:¶

@jsmeix What I meant was that the USB disk is used to boot from and it also contains the backups, so it cannot be unattached, and therefore, in @amtuannguyen case it will always be sda.
IMHO issue #1063 is the way to go, and we make a blacklist file of disks which are forbidden to use (just like we can do with multipath.conf)

jsmeix commented at 2017-03-30 11:15:¶

@gdha
many thanks for your explanation - it seems I get old
or I should not try to do several things at the same time ;-)

amtuannguyen commented at 2017-03-30 11:21:¶

@jsmeix @gdha Thanks for looking into this. If there is a way to blacklist the backup disk during drive mapping so that it is not offered as an option, that would be ideal. As it is right now, we have to edit the disk layout file at recovery time and replace all occurrences of sda (usb disk) with sdb (internal disk) and the recovery can proceed as normal.

gdha commented at 2017-03-30 11:28:¶

@amtuannguyen Unfortunately, the blacklist code is not yet written. We will put it on wish-list, but cannot predict when it will be written (free of charge). Of course, if you look at the http://relax-and-recover.org/support/sponsors page you can see we have alternatives foreseen

jsmeix commented at 2017-03-30 11:29:¶

Strange:
For me it "just works" (but I did not use an actual USB stick):

I tested it (in the past and right now again)
on two "same" virtual machines where I use
one same second virtual disk /dev/sdb
as /dev/disk/by-label/REAR-000 and for me both
"rear mkbackup" on the one virtual machine and
"rear recover" on the other virtual machine just worked.

On the virtual machine where I run "rear mkbackup"
/dev/sda is my system disk:

# parted -s /dev/sda unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sda: 20480MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 
Number  Start    End       Size      Type     File system     Flags
 1      1.00MiB  1490MiB   1489MiB   primary  linux-swap(v1)  type=82
 2      1490MiB  20480MiB  18990MiB  primary  ext4            boot, type=83

# grep -i USB etc/rear/local.conf 
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
USB_SUFFIX="mystuff"
USB_PARTITION_ALIGN_BLOCK_SIZE="32"
USB_DEVICE_FILESYSTEM_PERCENTAGE=49
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

# usr/sbin/rear -d -D format /dev/sdb
Relax-and-Recover 2.00 / Git
Using log file: /root/rear/var/log/rear/rear-e205.log
USB device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem: Yes
Repartitioning '/dev/sdb'
Creating partition table of type 'msdos' on '/dev/sdb'
Creating ReaR data partition up to 49% of '/dev/sdb'
Setting 'boot' flag on /dev/sdb
Creating ext3 filesystem with label 'REAR-000' on '/dev/sdb1'
Adjusting filesystem parameters on '/dev/sdb1'

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 8192MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 
Number  Start    End      Size     Type     File system  Flags
 1      32.0MiB  4014MiB  3982MiB  primary  ext3         boot, type=83

# usr/sbin/rear -d -D mkbackup
...

# mount /dev/sdb1 /mnt

# find /mnt/rear/e205/
/mnt/rear/e205/
/mnt/rear/e205/mystuff
/mnt/rear/e205/mystuff/kernel
/mnt/rear/e205/mystuff/rear-e205.log
/mnt/rear/e205/mystuff/backup.tar.gz
/mnt/rear/e205/mystuff/backup.log
/mnt/rear/e205/mystuff/initrd.cgz
/mnt/rear/e205/mystuff/syslinux.cfg

I attach this virtual disk as second virtual disk
to the other virtual machine where I run "rear recover"
and when I boot from that disk I get in the recovery system:

# parted -s /dev/sda unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sda: 20480MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 
Number  Start    End       Size      Type     File system     Flags
 1      1.00MiB  1490MiB   1489MiB   primary  linux-swap(v1)  type=83
 2      1490MiB  20480MiB  18990MiB  primary  ext4            boot, type=83

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 8192MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 
Number  Start    End      Size     Type     File system  Flags
 1      32.0MiB  4014MiB  3982MiB  primary  ext3         boot, type=83

# rear -d -D recover
...
Disk configuration is identical, proceeding with restore.
Start system layout restoration.
Creating partitions for disk /dev/sda (msdos)
Creating filesystem of type ext4 with mount point / on /dev/sda2.
...
Finished recovering your system. You can explore it under '/mnt/local'.

# reboot

and the recreated system "just works".

gdha commented at 2017-03-30 11:36:¶

@amtuannguyen Could you show us the output of lsblk -ro TYPE,NAME,SIZE,SERIAL | grep disk ?

amtuannguyen commented at 2017-03-30 11:44:¶

@gdha @jsmeix
Yes, that will work.
But, the problem is only when

the backup+rear+boot disk is a USB disk
and
it is the same size as the original system disk,
and
recovering on a different system with a single internal disk
that is different size than the original system.

In this case the USB disk becomes sda and the internal disk
becomes sdb, but the recovery system chooses sda automatically
because it is identical to the original system disk.

Hope my explanation makes sense.

jsmeix commented at 2017-03-30 13:32:¶

I think I can relatively easily add something to
layout/prepare/default/520_exclude_components.sh
so that any component that matches
USB_DEVICE or BACKUP_URL
gets autmatically excluded from recreation
(plus appropriate user information message
and probably with an Error abort to be safe).

Regarding "matches USB_DEVICE or BACKUP_URL":
My current basic idea is something like

backup_source=$( url_path $BACKUP_URL )
backup_source_partition=$( readlink -e $backup_source )
backup_source_disk=${backup_source_partition%%[1-9]*}

rescue_source_partition=$( readlink -e $USB_DEVICE )
rescue_source_disk=${rescue_source_partition%%[1-9]*}

and then exclude backup_source_disk
and rescue_source_disk from recreation.

gdha commented at 2017-03-30 13:47:¶

@jsmeix No I would look for a disk with the REAR-000 label and exclude it. The source is not important at all as that is always excluded during mkrescue/mkbackup. The problem will only occur in recover mode.

amtuannguyen commented at 2017-03-30 13:49:¶

@jsmeix @gdha Agree. Excluding the REAR-000 label at recover time would be easiest I think.

jsmeix commented at 2017-03-30 14:16:¶

@gdha @amtuannguyen you both assure me that such a disk
must be labeled REAR-000 and not anything else?
;-)

ReaR has already enough overoptimistic code
that mostly works but sometimes fails,
for example see this issue.

For example for me this "just works"
(nothing at all labeled REAR-000):

# grep -i USB etc/rear/local.conf
OUTPUT=USB
USB_DEVICE=/dev/sdb1
USB_SUFFIX="mystuff"
USB_PARTITION_ALIGN_BLOCK_SIZE="32"
USB_DEVICE_FILESYSTEM_PERCENTAGE=49
BACKUP_URL=usb:///dev/sdb1

# e2label /dev/sdb1 mystuff

# ls -l /dev/disk/by-label/
lrwxrwxrwx 1 root root 10 Mar 30 16:03 mystuff -> ../../sdb1

# usr/sbin/rear -d -D mkbackup
...

then on the other system I boot from that sdb disk and

RESCUE e205:~ # ls -l /dev/disk/by-label/
lrwxrwxrwx 1 root root 10 Mar 30 14:12 mystuff -> ../../sdb1

RESCUE e205:~ # rear -d -D recover
...

RESCUE e205:~ # reboot

and the recreated system just works.

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
USB_PREFIX="rear/$HOSTNAME/fixedbackupdir"
BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA

when "rear recover" could blindly overwrite the partition
that matches /dev/disk/by-label/REAR-DATA which is
the most valuable partition because there is the backup.

jsmeix commented at 2017-03-30 14:19:¶

I will try to make code so that "rear recover"
does not overwrite its own sources:
its own backup source (i.e. wherefrom it inputs the backup) or
its own rescue system source (i.e. wherefrom it was launched).

jsmeix commented at 2017-03-30 14:21:¶

But first and foremost I must be able to reproduce it,
then I can make code, then I can test that it works for me,
then there could be a PullRequest, and finally
@amtuannguyen you can verify whether or not
is also works for you.

jsmeix commented at 2017-03-30 14:27:¶

For clarification:
My proposal in
https://github.com/rear/rear/issues/1271#issuecomment-290411502
is about 520_exclude_components.sh which is only
run during "rear recover" (and during "rear layoutonly")

# for w in $( ls usr/share/rear/lib/*-workflow.sh | cut -d '/' -f 5 | cut -d '-' -f 1 ) ; do usr/sbin/rear -s $w | grep _exclude_components && echo $w ; done
Source layout/prepare/default/520_exclude_components.sh
layoutonly
Source layout/prepare/default/520_exclude_components.sh
recover

Currently I have no plans to change anything
during "rear mkbackup/mkrescue".

jsmeix commented at 2017-03-30 15:06:¶

FYI
my first failed attempt to reproduce it
and then how I could reproduce it:

On each of my two virtual machines I have now
two virtual disks with same size (20 GiB)
where the first virtual disk on the one virtual machine
is /dev/sda where the system is installed and
the second virtual disk /deb/sdb is used for ReaR:

# grep -i USB etc/rear/local.conf
OUTPUT=USB
USB_DEVICE=/dev/sdb1
USB_SUFFIX="mystuff"
USB_PARTITION_ALIGN_BLOCK_SIZE="32"
USB_DEVICE_FILESYSTEM_PERCENTAGE=49
BACKUP_URL=usb:///dev/sdb1

# ls -l /dev/disk/by-label/
lrwxrwxrwx 1 root root 10 Mar 30 16:35 mystuff -> ../../sdb1

# rear -d -D mkbackup
...

on the second virtual machine I use the above disk for ReaR
as first virtual disk so that the ReaR disk is /dev/sda there
and I have a second virtual disk /dev/sdb that is intended
for the system:

RESCUE e205:~ # ls -l /dev/disk/by-label/
lrwxrwxrwx 1 root root 10 Mar 30 14:43 mystuff -> ../../sda1

RESCUE e205:~ # rear -d -D recover
Relax-and-Recover 2.00 / Git
Using log file: /var/log/rear/rear-e205.log
Running workflow recover within the ReaR rescue/recovery system
ERROR: Mount command 'mount -v -o rw,noatime /dev/sdb1 /tmp/rear.IWGUskyoFit9VHd/outputfs' failed.
Aborting due to an error, check /var/log/rear/rear-e205.log for details
Terminated

RESCUE e205:~ # less /var/log/rear/rear-e205.log
...
+ source /usr/share/rear/verify/NETFS/default/060_mount_NETFS_path.sh
...
++ mount_cmd='mount -v -o rw,noatime /dev/sdb1 /tmp/rear.IWGUskyoFit9VHd/outputfs'
...
++ eval mount -v -o rw,noatime /dev/sdb1 /tmp/rear.IWGUskyoFit9VHd/outputfs
+++ mount -v -o rw,noatime /dev/sdb1 /tmp/rear.IWGUskyoFit9VHd/outputfs
mount: special device /dev/sdb1 does not exist

This is correct because it is actually /dev/sda1
but the hardcoded /dev/sdb1 in /etc/rear/local.conf
fortunately saved me.

So I must reference the ReaR disk by label
and then it does fail destructively:

RESCUE e205:~ # ls -l /dev/disk/by-label/mystuff
lrwxrwxrwx 1 root root 10 Mar 30 14:43 /dev/disk/by-label/mystuff -> ../../sda1

RESCUE e205:~ # vi /etc/rear/local.conf
...

RESCUE e205:~ # grep -i usb /etc/rear/local.conf
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/mystuff
USB_SUFFIX="mystuff"
USB_PARTITION_ALIGN_BLOCK_SIZE="32"
USB_DEVICE_FILESYSTEM_PERCENTAGE=49
BACKUP_URL=usb:///dev/disk/by-label/mystuff

RESCUE e205:~ # rear -d -D recover
# rear -d -D recover
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.iso885915)
Relax-and-Recover 2.00 / Git
Using log file: /var/log/rear/rear-e205.log
Running workflow recover within the ReaR rescue/recovery system
Using backup archive '/tmp/rear.szmtNVnl86oW9wW/outputfs/rear/e205/mystuff/backup.tar.gz'
Will do driver migration (recreating initramfs/initrd)
Calculating backup archive size
Backup archive size is 1.3G     /tmp/rear.szmtNVnl86oW9wW/outputfs/rear/e205/mystuff/backup.tar.gz (compressed)
Comparing disks.
Disk configuration is identical, proceeding with restore.
Start system layout restoration.
Creating partitions for disk /dev/sda (msdos)
Creating filesystem of type ext4 with mount point / on /dev/sda2.
Mounting filesystem /
Creating swap on /dev/sda1
Disk layout created.
ERROR: Mount command 'mount -v -o ro /dev/disk/by-label/mystuff /tmp/rear.szmtNVnl86oW9wW/outputfs' failed.
Aborting due to an error, check /var/log/rear/rear-e205.log for details
Terminated
RESCUE e205:~ # echo $?
143

RESCUE e205:~ # less /var/log/rear/rear-e205.log
...
+++ mount -v -o ro /dev/disk/by-label/mystuff /tmp/rear.szmtNVnl86oW9wW/outputfs
mount: special device /dev/disk/by-label/mystuff does not exist
...

Now all content on the ReaR disk got destroyed
by creating the disk layout on the ReaR disk.

jsmeix commented at 2017-03-31 07:39:¶

In my above
https://github.com/rear/rear/issues/1271#issuecomment-290440224
I noticed right now

RESCUE e205:~ # rear -d -D recover
...
RESCUE e205:~ # less /var/log/rear/rear-e205.log
...
+ source /usr/share/rear/verify/NETFS/default/060_mount_NETFS_path.sh
...
++ mount_cmd='mount -v -o rw,noatime /dev/sdb1 /tmp/rear.IWGUskyoFit9VHd/outputfs'

I think during "rear recover" there is no need to mount the
NETFS path where the backup is with 'rw'.
I think during "rear recover" nothing should be written there.
I think this violates that during "rear recover"
all ReaR sources must be sacrosanct.
I think I need to enhance and adapt much more to make
the recover (and layoutonly and restoreonly) workflows safe
against accidental damaging any ReaR sources.

@amtuannguyen
for a quick band-aid for your particular use case
(i.e. with a ReaR disk that is labeled "REAR-000")
it should be possible that you to make an appropriate
PRE_RECOVERY_SCRIPT that errors out when
your particular ReaR disk is found in disklayout.conf

jsmeix commented at 2017-03-31 09:00:¶

The following PRE_RECOVERY_SCRIPT
works for my particular use case:

PRE_RECOVERY_SCRIPT='disklayout_file=$( find / | grep disklayout.conf ) ; backup_path=$( url_path $BACKUP_URL ) ; backup_partition=$( readlink -e $backup_path ) ; backup_disk=${backup_partition%%[1-9]*} ; if test $backup_disk ; then for component in disk part fs swap ; do grep -q "^$component $backup_disk" $disklayout_file && Error "Found ReaR source in disklayout.conf as: $component $backup_disk" ; done ; fi'

It checks only BACKUP_URL but it can be easily enhanced
to also check more things like USB_DEVICE as needed.

Now "rear recover" errors out:

RESCUE e205:~ # grep -i USB /etc/rear/local.conf
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/mystuff
USB_SUFFIX="mystuff"
USB_PARTITION_ALIGN_BLOCK_SIZE="32"
USB_DEVICE_FILESYSTEM_PERCENTAGE=49
BACKUP_URL=usb:///dev/disk/by-label/mystuff

RESCUE e205:~ # ls -l /dev/disk/by-label/
lrwxrwxrwx 1 root root 10 Mar 31 08:49 mystuff -> ../../sda1

RESCUE e205:~ # rear -d -D recover
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.iso885915)
Relax-and-Recover 2.00 / Git
Using log file: /var/log/rear/rear-e205.log
Running workflow recover within the ReaR rescue/recovery system
ERROR: Found ReaR source in disklayout.conf as: disk /dev/sda
Aborting due to an error, check /var/log/rear/rear-e205.log for details
Terminated

RESCUE e205:~ # less /var/log/rear/rear-e205.log
...
+ source /usr/share/rear/setup/default/010_pre_recovery_script.sh
...
++ eval 'disklayout_file=$( find / | grep disklayout.conf ) ; backup_path=$( url_path $BACKUP_URL ) ; backup_partition=$( readlink -e $backup_path ) ; backup_disk=${backup_partition%%[1-9]*} ; if test $backup_disk ; then for component in disk part fs swap ; do grep -q "^$component $backup_disk" $disklayout_file && Error "Found ReaR source in disklayout.conf as: $component $backup_disk" ; done ; fi'
++++ grep disklayout.conf
++++ find /
+++ disklayout_file=/root/rear/var/lib/rear/layout/disklayout.conf
++++ url_path usb:///dev/disk/by-label/mystuff
++++ local url=usb:///dev/disk/by-label/mystuff
++++ local url_without_scheme=/dev/disk/by-label/mystuff
++++ echo /dev/disk/by-label/mystuff
+++ backup_path=/dev/disk/by-label/mystuff
++++ readlink -e /dev/disk/by-label/mystuff
+++ backup_partition=/dev/sda1
+++ backup_disk=/dev/sda
+++ test /dev/sda
+++ for component in disk part fs swap
+++ grep -q '^disk /dev/sda' /root/rear/var/lib/rear/layout/disklayout.conf
+++ Error 'Found ReaR source in disklayout.conf as: disk /dev/sda'
...

RESCUE e205:~ # grep -v '^#' /root/rear/var/lib/rear/layout/disklayout.conf
disk /dev/sda 21474836480 msdos
part /dev/sda 1561329664 1048576 primary none /dev/sda1
part /dev/sda 19912458240 1562378240 primary boot /dev/sda2
fs /dev/sda2 / ext4 uuid=46d7e8be-7812-49d1-8d24-e25ed0589e94 label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16377 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
swap /dev/sda1 uuid=28e43119-dac1-4426-a71a-1d70b26d33d7 label=

FYI: I have /root/rear/var/lib/rear/layout/disklayout.conf because
I use a ReaR GitHub checkout in the /root/rear directory.

I decided that it won't help to exclude ReaR sources components
from recreation because this would result an incomplete recreation.

Instead a clear Error abort is right here because in such cases
the user must revise and adapt his disklayout.conf file
in his recovery system.

E.g. in my case:

RESCUE e205:~ # sed -i -e 's/sda/sdb/g' /root/rear/var/lib/rear/layout/disklayout.conf

RESCUE e205:~ # grep -v '^#' /root/rear/var/lib/rear/layout/disklayout.conf
disk /dev/sdb 21474836480 msdos
part /dev/sdb 1561329664 1048576 primary none /dev/sdb1
part /dev/sdb 19912458240 1562378240 primary boot /dev/sdb2
fs /dev/sdb2 / ext4 uuid=46d7e8be-7812-49d1-8d24-e25ed0589e94 label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16377 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
swap /dev/sdb1 uuid=28e43119-dac1-4426-a71a-1d70b26d33d7 label=

RESCUE e205:~ # rear -d -D recover
...
Disk configuration is identical, proceeding with restore.
Start system layout restoration.
Creating partitions for disk /dev/sdb (msdos)
Creating filesystem of type ext4 with mount point / on /dev/sdb2.
Mounting filesystem /
Creating swap on /dev/sdb1
Disk layout created.
Restoring from '/tmp/rear.HOP25kFgcUD7xVG/outputfs/rear/e205/mystuff/backup.tar.gz'...
...
Restoring finished.
...
Running mkinitrd...
Recreated initrd (/sbin/mkinitrd).
Installing GRUB2 boot loader
Finished recovering your system. You can explore it under '/mnt/local'.

RESCUE e205:~ # reboot

and the recreated system "just works".

jsmeix commented at 2017-03-31 11:46:¶

In contrast to my proposal in
https://github.com/rear/rear/issues/1271#issuecomment-290411502
I think now that I will add a separated script
that is run as early as feasible during "rear recover"
(and the other recovery-like workflows layoutonly and restoreonly)
that checks if ReaR sources (i.e. data input for "rear recover"
in particular the source wherefrom the backup will be restored)
are listed as active entries in disklayout.conf and
if yes that new script will Error out, cf.
"Try to care about possible errors" in
https://github.com/rear/rear/wiki/Coding-Style

For the reason for Error abort see my results in
https://github.com/rear/rear/issues/1271#issuecomment-290658377
therein

I decided that it won't help to exclude
ReaR sources components from recreation
because this would result an incomplete recreation.

Instead a clear Error abort is right here because
in such cases the user must revise and adapt
his disklayout.conf file in his recovery system.

jsmeix commented at 2017-05-22 07:29:¶

I postpone it to a ReaR release after 2.1 (for now 2.2)
because for now I won't find the time to implement if for ReaR 2.1
and a workaround via PRE_RECOVERY_SCRIPT is possible cf.
https://github.com/rear/rear/issues/1271#issuecomment-290658377

jsmeix commented at 2017-07-14 10:41:¶

According to
https://github.com/rear/rear/issues/1398#issuecomment-315318878
this issue here is postponed to ReaR v 2.3

jsmeix commented at 2017-10-20 09:59:¶

Since https://github.com/rear/rear/issues/1535
is implemented via https://github.com/rear/rear/pull/1536
there is no longer a hardcoded 'REAR-000' label.

jsmeix commented at 2017-11-20 09:25:¶

I have to postpone it again to a ReaR release after 2.3 (for now 2.4)
because for now I won't find the time to implement if for ReaR 2.3.

jsmeix commented at 2017-11-23 14:33:¶

With https://github.com/rear/rear/pull/1593 merged
issues like this one can and should be avoided.

Now layout/prepare/default/250_compare_disks.sh
is much enhanced to make "rear recover" hopefully
sufficiently safe against blindly proceeding when
"Disk configuration looks identical"
that can destroy things as in this issue here.

Additionally when "Disk configuration looks identical"
there is still a UserInput() confirmation dialog
with 30 seconds timeout to be on the safe side.
In case of automated recovery that timeout
could needlessly delay things. See
https://github.com/rear/rear/pull/1593#issuecomment-346628678
how to avoid that 30 seconds delay.

To be 100% on the safe side the user can now
enforce the manual disk layout configuration mode
via MIGRATION_MODE='true'.

On the other hand when the user knows the content
in his var/lib/rear/layout/disklayout.conf or in his
etc/rear/disklayout.conf file is "the right one" then
he can now enforce to not switch into manual disk layout
configuration mode via MIGRATION_MODE='false'.
In this case "rear recover" blindly recreates according
to the content in disklayout.conf and there is also
no UserInput() confirmation dialog.

Currently I think this issue might be even already
sufficiently fixed (or at least sufficiently avoided)
by https://github.com/rear/rear/pull/1593
but I will keep this issue open for some time
until I got some user feedback how the current
ReaR GitHub master code behaves for your
particular use cases.

jsmeix commented at 2017-11-24 09:53:¶

An addedum FYI:

Perhaps you may wonder why that
UserInput() confirmation dialog (with its delay)
is shown in any case even when it seems
"Disk configuration really is identical"
when on the original system only one single /dev/sda was used
and on the replacement hardware /dev/sda with same size is found:

Even in this case things could get terribly wrong
if "rear recover" would "just proceed" automatically
when on the replacement hardware /dev/sda (with same size)
is not the actually intended target disk but a disk with the
recovery system plus the backup but the actual target disk
is /dev/sdb with different (e.g. bigger) size.

On physical hardware this is probably rather unlikely
but not on virtual machines when one uses virtual disks
with same standard size (e.g. I use by default 20GiB for
the disks on my virtual machines).

Therefore "better safe than sorry" and have
that UserInput() confirmation dialog in any case.

jsmeix commented at 2017-11-28 15:17:¶

Right now I noticed that probably also
https://github.com/rear/rear/issues/1437
is an issue that is related to this issue here:
When the original system was on /dev/sda
but on the replacement hardware /dev/sda is a disk
with the ReaR recovery system and the backup
then https://github.com/rear/rear/issues/1437
might result that the bootloader of the original system
gets installed on that /dev/sda which probably
destroys someting of the ReaR recovery system
(perhaps it is no longer bootable or things like that).

andhoo commented at 2018-03-07 00:39:¶

Years ago I had something similar that the wrong disk got overwritten without user intervention. I made an modification but didn't do a pull request for it. Take a look maybe it can be temporarily solution. https://github.com/andhoo/rear/commit/748b3d76e6dedc76b9608e6eef88d66a23aaabb8

jsmeix commented at 2018-05-03 06:47:¶

Since
https://github.com/rear/rear/issues/1271#issuecomment-346633365
this issue is (hopefully) sufficiently mitigated
so that there is no longer an actual pressure to get it solved for ReaR 2.4
which means I can make it an "enhancement" for a "ReaR future" verion,
cf. https://github.com/rear/rear/issues/1790#issue-319663141

jsmeix commented at 2018-07-06 12:11:¶

@amtuannguyen
see https://github.com/rear/rear/issues/1854
for possible problems that may happen when the ReaR recovery system
is on a normal disk that becomes /dev/sda on the replacement hardware
even if with the current MIGRATION_MODE implementation
the ultimate disaster (destroyed backup) should be avoided.

jsmeix commented at 2020-03-06 14:17:¶

I am not fully satisfied with the current mitigation, cf.
https://github.com/rear/rear/issues/1271#issuecomment-386204739

I hope I may find time to implement a better solution
so I like to keep it open for some more time.

github-actions commented at 2020-07-02 01:33:¶

Stale issue message

jsmeix commented at 2021-04-19 12:24:¶

I reopen the issue as critical bug that I like to fix for ReaR 2.7 because
a user reported to me via personal mail the following special case where
"rear recover" did not automatically go into MIGRATION_MODE
but "just tried" to overwrite the ReaR recovery disk with the backup on it.

What he reported (excerpts):

Backup was done for a notebook with a single disk (320GB);
The backup usb disk with exactly the same size (320GB)
was used for "mkbackup" (=> rescue media and backup tar file);
The disk in the notebook was replaced by an SSD with different size (500GB);
Booting from backup disk and calling "rear recover";
Rear detected exactly one disk with the original size (rescue media!) and
proposed this one for the target disk, without switching to migration mode
(I think so, not sure how to notice this...);
If I remember correctly there was a timeout of 300 seconds
to chose automatically the proposed target disk,
and without interaction the rescue media would be destroyed.
In this case there are no ambiguous target disks
but by detecting a single disk with a correctly matching size
"rear seems to be sure" that this one will be the correct target.

I want to find a solution to avoid that ReaR destroys its own disk.

I need to find a generically and reliable working way
how to detect if a disk is "ReaR's own disk".

Off the top of my head I think the problem is that "ReaR's own disk"
is not mounted while the ReaR recovery system is running
(I did not test it right now) because the ReaR recovery system
is running within a ramdisk.
I think all what happens is that during booting from "ReaR's own disk"
the bootloader on that disk loads the kernel and initrd
(the initrd contains the ReaR recovery system files)
into RAM and then the files of the initrd are unpacked into a ramdisk.
So while the ReaR recovery system is running "ReaR's own disk"
looks like any other "normal" disk.
I would not like to mount all "normal" looking disks to be able
to inspect if their contents look like "ReaR's own disk".

At least a test if a "normal" looking disks is the target
of a /dev/disk/by-label/REAR-000 symlink
should be implemented to avoid at least this common case.

jsmeix commented at 2021-04-19 12:36:¶

OliverO2 commented at 2021-06-03 22:01:¶

These are my recent encounters with RAWDISK output and a separate backup drive (independent of ReaR) which prompted me to try to...

Stop ReaR from overwriting its own disk and backup drives¶

Use Cases¶

I start rear recover on a virtual machine. The original system has three drives (sda, sdb, sdc), the recovery VM has two drives (vda, vdb), plus the recovery partition on another drive (vdc). The intention is to leave sdc unmapped. ReaR asks remapping questions for sda and sdb. Then it decides to map sdc to vdc without even asking. Continuing would overwrite the recovery system. I have to edit /var/lib/rear/layout/disk_mappings manually.
I start rear recover on a virtual machine, with a USB backup drive connected as /dev/sda. The original system's boot drive was also /dev/sda. ReaR asks remapping questions, but I do not notice that the /dev/sdaX partitions on the VM are actually those of the backup drive, while the VM boot drive would have been /dev/vdaX. ReaR happily overwrites the backup drive. This should never be possible.

Some Ideas¶

The idea for use case 1 is to make ReaR remember its own partition table UUID and prevent overwriting the corresponding disk. I'll have to check separately for cases where ReaR resides on a partition of a disk while it is used to restore other partitions on that same disk.
The idea for use case 2 is to have an array variable PROTECTED_FILE_SYSTEM_LABELS containing patterns for file systems labels. File systems with a matching label would then be protected from being overwritten. The user would generally have to specify these labels (we cannot know the labels of backup disks which are independent of ReaR).

I'd like to explore this a bit further and might come up with a PR if that doesn't interfere with other people's work. I'd like to do this in a way that is either generic (not limited to RAWDISK) or at least easily extendable (to USB, for example). No promises at this point (I might not find a solution within the time I have available for this), but: Should I try?

github-actions commented at 2021-08-03 02:19:¶