#2236 Issue `closed`: Multipath disk incorrectly excluded from backup¶

Labels: enhancement, fixed / solved / done

petrpavlu opened issue at 2019-09-12 13:21:¶

ReaR version ("/usr/sbin/rear -V"):
```
# ./usr/sbin/rear -V
Relax-and-Recover 2.5 / Git
```
Git revision a252ca20b5dc9f34f6c8fffb61e055141ca37669 (current HEAD).
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):
```
# cat etc/rear/os.conf 
OS_VENDOR=SUSE_LINUX
OS_VERSION=12.2
```

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

```
# cat etc/rear/local.conf
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=4,nolock"
BACKUP_URL=nfs://192.168.122.1/srv/nfs/multipath-test
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" snapper chattr lsattr )
COPY_AS_IS=( "${COPY_AS_IS[@]}" /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
BACKUP_PROG_INCLUDE=( /var/cache /var/lib/mailman /var/tmp /var/lib/pgsql /usr/local /opt /var/lib/libvirt/images /boot/grub2/i386-pc /var/opt /srv /boot/grub2/x86_64-efi /var/lib/mariadb /var/spool /var/lib/mysql /tmp /home /var/log /var/lib/named /var/lib/machines )
AUTOEXCLUDE_DISKS=y
AUTOEXCLUDE_MULTIPATH=n

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):

KVM guest.
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):

x86-64.
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):

UEFI + GRUB.
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):

Local disk + DM multipath.
Description of the issue (ideally so that others can reproduce it):

When using AUTOEXCLUDE_MULTIPATH=n AUTOEXCLUDE_DISKS=y, it is possible that a used multipath device is excluded from backup.

Problematic configuration:
- Disk /dev/vda holds root partition /dev/vda2.
- Multipath device /dev/mapper/0QEMU_QEMU_HARDDISK_0001 is backed by /dev/sda+/dev/sdb as slaves and contains partition 0QEMU_QEMU_HARDDISK_0001-part1 mounted at /mnt/mp-1.
In the disklayout.conf speak:
```
disk /dev/vda 17179869184 msdos
part /dev/vda 1561329664 1048576 primary none /dev/vda1
part /dev/vda 15617490944 1562378240 primary boot /dev/vda2
fs /dev/mapper/0QEMU_QEMU_HARDDISK_0001-part1 /mnt/mp-1 ext4 uuid=8cf05242-2bd4-404b-b508-5235fd95e7a8 label= blocksize=1024 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=4096 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
fs /dev/vda2 / btrfs uuid=0b200db7-aa8f-4dcb-a8f5-843896599a38 label= options=rw,relatime,space_cache,subvolid=259,subvol=/@/.snapshots/1/snapshot
multipath /dev/mapper/0QEMU_QEMU_HARDDISK_0001 2147483648 /dev/sda,/dev/sdb
part /dev/mapper/0QEMU_QEMU_HARDDISK_0001 209715200 1048576 primary none /dev/mapper/0QEMU_QEMU_HARDDISK_0001-part1
```
In diskdeps.conf:
```
/dev/vda1 /dev/vda
/dev/vda2 /dev/vda
fs:/mnt/mp-1 /dev/mapper/0QEMU_QEMU_HARDDISK_0001-part1
fs:/mnt/mp-1 fs:/
fs:/ /dev/vda2
/dev/mapper/0QEMU_QEMU_HARDDISK_0001 /dev/sda
/dev/mapper/0QEMU_QEMU_HARDDISK_0001 /dev/sdb
/dev/mapper/0QEMU_QEMU_HARDDISK_0001-part1 /dev/mapper/0QEMU_QEMU_HARDDISK_0001
```
With AUTOEXCLUDE_DISKS=y, ReaR excludes fs:/mnt/mp-1, /dev/mapper/0QEMU_QEMU_HARDDISK_0001 and /dev/mapper/0QEMU_QEMU_HARDDISK_0001-part1 from the backup. The following seems to happen:
- Code 320_autoexclude.sh that handles automatic exclusion of disks starts processing fs:/mnt/mp-1. It calls find_disk_and_multipath() -> find_disk() -> get_parent_components().
- Function get_parent_components() calculates all parent elements for fs:/mnt/mp-1. The result is:
```
/dev/mapper/0QEMU_QEMU_HARDDISK_0001-part1 fs:/ /dev/mapper/0QEMU_QEMU_HARDDISK_0001 /dev/vda2 /dev/sda /dev/sdb /dev/vda
```
- Function get_parent_components() scans the ancestors for a correct type. The input type comes from find_disk() and is "disk". This results in get_parent_components() printing "/dev/vda" as its output.
- Function find_disk_and_multipath() sees that find_disk() had a non-zero length output and propagates this result to its caller.
- Code in 320_autoexclude.sh records that /dev/vda is a used disk.
The problem is that find_disk_and_multipath() never returns that /dev/mapper/0QEMU_QEMU_HARDDISK_0001 is needed for fs:/mnt/mp-1, which results in excluding this disk and subsequently also excluding fs:/mnt/mp-1.
Workaround, if any:

The problem can be avoided by using AUTOEXCLUDE_DISKS=n.

jsmeix commented at 2019-09-13 11:39:¶

@petrpavlu
again thank you for your comprehensive issue report
together with your proposed fix in your
https://github.com/rear/rear/pull/2237
plus your description how to avoid it.

From my experience with other issues the backup include/exclude functionality
versus the components include/exclude functionality in ReaR is "hairy".

Currently - as far as I see - there is no simple clear and consistently working way
how the user could specify what disk layout components he wants to get recreated
and what mountpoints or directories he wants to get included in the backup.

Regarding the backup with BACKUP=NETFS I implemented some time ago
BACKUP_ONLY_INCLUDE and BACKUP_ONLY_EXCLUDE
see its documentation in the default.conf file
that should help to specify explicitly what gets included in the backup.

Note that things like BACKUP_ONLY_INCLUDE and BACKUP_ONLY_EXCLUDE
and many other BACKUP_* variables only

apply to all cases of internal Relax-and-Recover backup

For third party backup tools things behave usually totally different,
see the section "Relax-and-Recover versus backup and restore" in
https://en.opensuse.org/SDB:Disaster_Recovery

gdha commented at 2019-10-05 09:58:¶

@petrpavlu @jsmeix is this fixed you think?

jsmeix commented at 2019-10-05 12:14:¶

@gdha
this issue depends on its matching pull request
https://github.com/rear/rear/pull/2237
where @schabrolles would have to have a look, see
https://github.com/rear/rear/pull/2237#issuecomment-531210391

jsmeix commented at 2019-10-18 11:40:¶

With https://github.com/rear/rear/pull/2237 merged
this issue should be fixed.

[Export of Github issue for rear/rear.]