#2853 Issue open: Let ReaR deal with LVM VGs without LVs¶
Labels: enhancement
david-hill opened issue at 2022-08-31 12:36:¶
Relax-and-Recover (ReaR) Issue Template¶
rear should skip mounted loop devices
[dhill@knox rear]$ git diff
diff --git a/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh b/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh
index e01dbf46..dd970944 100644
--- a/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh
+++ b/usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh
@@ -82,6 +82,9 @@ local lvs_exit_code
# Skip lines that are not describing physical devices
# i.e. lines where pdev does not start with a leading / character:
test "${pdev#/}" = "$pdev" && continue
+ if [[ "${pdev}" =~ '/dev/loop' ]] then
+ continue
+ fi
# Output lvmdev header only once to DISKLAYOUT_FILE:
if is_false $header_printed ; then
Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):
-
ReaR version ("/usr/sbin/rear -V"):
All -
OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
RHEL 8.4 -
ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
-
Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
-
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
-
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
-
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
-
Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):
-
Description of the issue (ideally so that others can reproduce it):
-
Workaround, if any:
-
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
2022-08-30 12:46:16.127472961 Verifying that the 'disk' entries in /var/lib/rear/layout/disklayout.conf are correct
1200210141184
2022-08-30 12:46:16.130304394 Verifying that the 'part' entries for /dev/sda in /var/lib/rear/layout/disklayout.conf are correct
1048576
1048576
1200208027136
2097152
2022-08-30 12:46:16.132631819 Verifying that the 'part' entries for /dev/sda in /var/lib/rear/layout/disklayout.conf specify consecutive partitions
2022-08-30 12:46:16.136434519 Verifying that the 'lvm...' entries in /var/lib/rear/layout/disklayout.conf are correct
2022-08-30 12:46:16.142924055 LVM no 'lvmvol /dev/cinder-volumes' for 'lvmdev /dev/cinder-volumes'
2022-08-30 12:46:16.149657393 ERROR:
gdha commented at 2022-09-01 07:14:¶
@david-hill Could you copy/paste the output of the df and mount
command please? Loop devices are normally skipped (on my ubuntu ws I
have more then 10 loop devices which are skipped during a rear
savelayout).
david-hill commented at 2022-09-01 10:41:¶
@david-hill Could you copy/paste the output of the
dfandmountcommand please? Loop devices are normally skipped (on my ubuntu ws I have more then 10 loop devices which are skipped during a rear savelayout).
Is it a LVM mounted on a loop ? It has to be LVM otherwise you won't have any issues ...
gdha commented at 2022-09-01 12:31:¶
@david-hill Still need to see some output to understand it better...
pcahyna commented at 2022-09-08 15:56:¶
@gdha do you know how do the loop devices get skipped? I don't see any
code to skip them under layout/save. There is code in
200_partition_layout.sh that skips generating disk lines for devices
that are not in the known list (currently the list is: hd sd cciss vd
xvd dasd nvme mmcblk ) but there seems to be nothing to prevent fs
lines being generated for them in 230_filesystem_layout.sh ). Surely I
am missing something, as usually it has not been a problem.
pcahyna commented at 2022-09-08 15:59:¶
@david-hill could you please provide the
/var/lib/rear/layout/disklayout.conf file generated when you got the
error?
pcahyna commented at 2022-09-16 17:21:¶
@gdha I don't see that:
dd if=/dev/zero of=/var/tmp/loopfs bs=1024 count=102400
mkfs.xfs /var/tmp/loopfs
mkdir /home/loopfs
mount -o loop /var/tmp/loopfs /home/loopfs
rear savelayout
results in this in layout:
fs /dev/loop0 /home/loopfs xfs uuid=fbb2b182-5f12-49b0-9ff4-9bbf99e3cc35 label= options=rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota
so, loop device is not skipped (unless it changed in ReaR 2.7; this was
with ReaR 2.6). Not sure why loop devices on Ubuntu are skipped, aren't
they mounted under something in AUTOEXCLUDE_PATH ?
pcahyna commented at 2022-09-16 17:40:¶
@david-hill your issue is only tangentially related to loop devices. Using a PV/VG on a disk:
parted --script -a optimal /dev/vdc -- mklabel gpt mkpart extra ext2 1M -1M set 1 lvm on
pvcreate --yes /dev/vdc1
vgcreate sigvg /dev/vdc1
rear savelayout
results in the same error:
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct ...
LVM no 'lvmvol /dev/sigvg' for 'lvmdev /dev/sigvg'
ERROR:
====================
BUG in /usr/share/rear/layout/save/default/950_verify_disklayout_file.sh line 254:
'Entries in /var/lib/rear/layout/disklayout.conf are broken ('rear recover' would fail)'
without any loop device involved. The underlying issue is that ReaR does not like volume groups without logical volumes. Unused volume groups may be unusual, but I think there is nothing really wrong with them, and PR #2603 fixed a bug ( #2596 ) where an unused PV (without VG) aborted mkrescue. So, by analogy, I think we should also allow a VG without any LV (either exclude it, or add it to the layout as empty).
pcahyna commented at 2022-09-16 17:43:¶
What to do with loop devices is a separate question, but I think should be addressed consistently for all storage objects on them (you see above that filesystems on loop devices are not being excluded either).
david-hill commented at 2022-09-16 18:34:¶
I would suggest we just skip /dev/loop ... is there any possibility we have any useful data on there ?
gdha commented at 2022-09-17 08:40:¶
@pcahyna On my ubuntu ws the loop devices are of type squashfs, so this is the reason of exclusion. The one I created manually is included as you can see:
cat /var/lib/rear/layout/disklayout.conf
# Disk layout dated 20220917103612 (YYYYmmddHHMMSS)
# NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT UUID
# /dev/loop0 /dev/loop0 loop squashfs 55.6M /snap/core18/2566
# /dev/loop1 /dev/loop1 loop squashfs 142.2M /snap/chromium/2076
# /dev/loop2 /dev/loop2 loop squashfs 9M /snap/canonical-livepatch/146
# /dev/loop3 /dev/loop3 loop squashfs 63.2M /snap/core20/1623
# /dev/loop4 /dev/loop4 loop squashfs 4K /snap/bare/5
# /dev/loop5 /dev/loop5 loop squashfs 91.7M /snap/gtk-common-themes/1535
# /dev/loop6 /dev/loop6 loop squashfs 162.9M /snap/gnome-3-28-1804/145
# /dev/loop7 /dev/loop7 loop squashfs 400.8M /snap/gnome-3-38-2004/112
# /dev/loop8 /dev/loop8 loop squashfs 114.9M /snap/core/13741
# /dev/loop9 /dev/loop9 loop squashfs 62M /snap/core20/1611
# /dev/loop10 /dev/loop10 loop squashfs 164.8M /snap/gnome-3-28-1804/161
# /dev/loop11 /dev/loop11 loop squashfs 114M /snap/core/13425
# /dev/loop12 /dev/loop12 loop squashfs 55.6M /snap/core18/2560
# /dev/loop13 /dev/loop13 loop squashfs 142.2M /snap/chromium/2082
# /dev/loop14 /dev/loop14 loop squashfs 346.3M /snap/gnome-3-38-2004/115
# /dev/loop15 /dev/loop15 loop squashfs 9M /snap/canonical-livepatch/138
# /dev/loop16 /dev/loop16 loop squashfs 81.3M /snap/gtk-common-themes/1534
# /dev/loop17 /dev/loop17 loop ext4 100M /home/loopfs d89d7f90-1bc9-45a5-a795-de9897577f50
...
fs /dev/loop17 /home/loopfs ext4 uuid=d89d7f90-1bc9-45a5-a795-de9897577f50 label= blocksize=1024 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=4081 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
So, yes it would be better to exclude loop devices if they are not really required.
pcahyna commented at 2022-09-17 12:35:¶
Why shouldn't be any useful data on /dev/loop ? Is it because the
backing files of loop devices can be already saved when the filesystem
where they reside is backed up? This makes sense, but beware: if the
loop devices are mounted or in any other way used (like being PVs for
active VGs, esp. when using thin pools) when the backing files are being
backed up, the backup will very likely be inconsistent. This is a
similar reasoning like why we usually don't back up data by using just
dd on the underlying block device.
Properly saving loop devices and all storage objects on top of them
(instead of skipping) would be a way around the problem, but would
require some special code (to call losetup during layout recreation to
set up the loop devices properly).
Another option would be to exclude them and to warn (or error out) when
backing files of loop devices are being backed up and active read-write
storage objects (filesystems, LVM) reside on top of them. This would
guide the user to think how to temporarily deactivate this stuff during
backup to prevent inconsistent backups (similarly to how one can shut
down database server before backup and start them after to prevent
inconsistent database files, cf the description of POST_BACKUP_SCRIPT
in default.conf).
jsmeix commented at 2022-09-19 09:59:¶
Let's keep this issue to
"Let ReaR deal with LVM VGs without LVs"
in the same way as we did it
for
https://github.com/rear/rear/issues/2596
via
https://github.com/rear/rear/pull/2603
For the separated question what to do with loop devices, cf.
https://github.com/rear/rear/issues/2853#issuecomment-1249628411
I created a separated issue
https://github.com/rear/rear/issues/2865
jsmeix commented at 2022-09-19 10:05:¶
@pcahyna
regarding your above
https://github.com/rear/rear/issues/2853#issuecomment-1250063689
In general we do not know in ReaR what files are in the backup.
In particular not for alomst all non-internal backup methods
because ReaR does not implement making a backup with
almost all non-internal (external) backup methods,
see "BACKUP SOFTWARE INTEGRATION" in
https://github.com/rear/rear/blob/master/doc/rear.8.adoc
So in general we cannot compare what files are in the backup
with what storage devices or filesystems are in disklayout.conf.
Furthermore even with internal backup methods we do not know
during "rear mkrescue" what files are actually in the backup
because no backup is actually made during "rear mkrescue".
github-actions commented at 2022-11-19 02:59:¶
Stale issue message
github-actions commented at 2023-01-28 02:32:¶
Stale issue message
github-actions commented at 2023-04-08 02:08:¶
Stale issue message
[Export of Github issue for rear/rear.]