#2514 PR merged: Wipe disks before recreating partitions/volumes/filesystems/...¶
Labels: enhancement, fixed / solved / done, severe improvement
jsmeix opened issue at 2020-11-11 15:12:¶
New layout/recreate/default/150_wipe_disks.sh to wipe disks before recreating.
-
Type: Enhancement
-
Impact: High
-
Reference to related issue (URL):
https://github.com/rear/rear/issues/799 -
How was this pull request tested?
On my SLES15 test system with LUKS and LVM:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 20G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 20G
`-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2 /dev/dm-0 /dev/sda2 crypt LVM2_member 20G
|-/dev/mapper/system-swap /dev/dm-1 /dev/dm-0 lvm swap 2G [SWAP]
|-/dev/mapper/system-root /dev/dm-2 /dev/dm-0 lvm btrfs 12.6G /
`-/dev/mapper/system-home /dev/dm-3 /dev/dm-0 lvm xfs 5.4G /home
/dev/sdb /dev/sdb ata disk 1G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 307M
| `-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1 /dev/dm-4 /dev/sdb1 crypt ext4 305M /luks1test
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS 409M
- Brief description of the changes in this pull request:
New layout/recreate/default/150_wipe_disks.sh to wipe disks.
The disks that will be completely wiped are those disks
where in diskrestore.sh the create_disk_label function is called
(the create_disk_label function calls "parted -s $disk mklabel
$label")
i.e. the disks that will be completely overwritten by diskrestore.sh.
This implements
https://github.com/rear/rear/issues/799
"Clean up disks before recreating partitions/volumes/filesystems/..."
The intent is to be also used later as a precondition for the future
new 'storage' stage/code as future replacement of the 'layout'
stage/code
cf.
https://github.com/rear/rear/issues/2510
jsmeix commented at 2020-11-11 15:39:¶
The interesting part is that in the ReaR recovery system
on replacement hardware that has the same disk layout
as the original system has (because I did a "rear recover"
of the original system on the replacement hardware)
I do not see all those nested block devices that do actually exist
on the disk of the replacement hardware.
In the ReaR recovery system I only get before 150_wipe_disks.sh is run
rear> lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 20G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 20G 0c58676a-bcb6-42be-8e1c-46a24d954ca7
/dev/sdb /dev/sdb ata disk 1G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 307M fb79b19e-0e6d-4570-aa96-fa968d6e0795
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS 409M 52bc31e4-e9dc-4de0-99cc-d9cca3a886b0
/dev/sr0 /dev/sr0 ata rom iso9660 76.9M 2020-11-11-15-35-02-75
and while 150_wipe_disks.sh is run I get on the terminal
with export MIGRATION_MODE='true' and then rear -D recover
Disks to be overwritten: /dev/sda /dev/sdb
1) Confirm disks to be completely overwritten and continue 'rear recover'
2) Use Relax-and-Recover shell and return back to here
3) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disks to be completely overwritten and continue 'rear recover''
User confirmed disks to be overwritten
Wiping child devices of /dev/sda in reverse ordering: /dev/sda2 /dev/sda1 /dev/sda
Wiping device /dev/sda2
Wiping device /dev/sda1
Wiping device /dev/sda
Wiping child devices of /dev/sdb in reverse ordering: /dev/sdb2 /dev/sdb1 /dev/sdb
Wiping device /dev/sdb2
Wiping device /dev/sdb1
Wiping device /dev/sdb
and after 150_wipe_disks.sh is run I get
rear> lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 20G
/dev/sdb /dev/sdb ata disk 1G
/dev/sr0 /dev/sr0 ata rom iso9660 76.9M 2020-11-11-15-35-02-75
so 150_wipe_disks.sh wiped all what was visible in the recovery system.
Nevertheless there is still metadata on the disk from those nested block
devices
that are invisible in the recovery system
`-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2 /dev/dm-0 /dev/sda2 crypt LVM2_member 20G
|-/dev/mapper/system-swap /dev/dm-1 /dev/dm-0 lvm swap 2G [SWAP]
|-/dev/mapper/system-root /dev/dm-2 /dev/dm-0 lvm btrfs 12.6G /
`-/dev/mapper/system-home /dev/dm-3 /dev/dm-0 lvm xfs 5.4G /home
...
| `-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1 /dev/dm-4 /dev/sdb1 crypt ext4 305M /luks1test
so that metadata from nested block devices could still get in the way
when recreating things exactly as they have been before on the disk.
jsmeix commented at 2020-11-11 16:11:¶
FYI
excerpts from dialog that I had yesterday with a SUSE colleague
who is an expert in storage setup:
I asked him:
I have a question about RAID devices deep inside a disk partition:
I would assume when I wipe the first and last 16 MiB of a partition
then sufficient RAID metadata is wiped so that automatisms could
no longer detect that there is anything RAID related deep inside
the partition.
I assume the automatisms would not scan the whole partition
to find RAID related things deep inside
(i.e. I assume when nothing is found at the beginning or end
the whole partition is ignored).
He repiled:
Suppose you have a mirroring RAID,
and inside the RAID two partitions,
the second one starts at 1GB and includes a XFS.
The signature of that XFS is now not within the first 16 MB
of the underlying partition of the RAID.
So sda1 and sdb1 make up md0, that has md0p1 and md0p2.
Cleaning 16 MB of sda1 and sdb1 will not clean md0p2.
I replied
Yes, but my expectation was that cleaning 16 MB of sda1 and sdb1
will clean md0 so that md0p1 and md0p2 get ignored.
But now I see an issue even if md0 was cleaned this way.
When the same RAID setup is recreated on that disk
with exactly same values
then after recreating md0 the old md0p1 and md0p2
may reappear instantly
which lets the commands that intend to recreate md0p1
and md0p2 fail and
also the old XFS may reappear but here enforced
"mksf.xfs -f" helps (which is done e.g. in ReaR).
Then I had "A totally offhanded crazy idea"
A totally offhanded crazy idea:
Perhaps it works best in practice to not care about old stuff on the disk
but recreate the disk partitions with an offset of e.g. 1 MiB
(or 4 MiB or 8 MiB to be more on the safe side for SSDs alignments).
Who would care (in particular in case of disaster recovery) in practice
about 8 MiB less disk space for the last partition on each disk?
Of course only optionally when requested so the user can decide.
After sleeping over it I think that "totally offhanded crazy idea"
is perhaps not so "totally crazy" because such a thing is needed
anyway
for properly migrating from rotating rust disks to SSDs:
When migrating from rotating rust disks to SSDs new disk alignment
is usually needed so that the partition boundaries on the SSDs
match the phsyical "sector size" or "segment size" or "erase block
size"
(i.e. the physical read/write unit size) of the SSD
which is 4 MiB or 8 MiB or even more on nowadays SSDs
cf. USB_PARTITION_ALIGN_BLOCK_SIZE in default.conf
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L853
So when migrating from rotating rust disks to SSDs
the partition begin values must be adapted to be
multiples of the physical read/write unit size of the SSD
i.e. multiples of 4 MiB or 8 MiB or even more.
So when migrating from rotating rust disks to SSDs the partition begin
values
(that are usually multiples of 1 MiB or less on rotating rust disks)
need to be shifted to multiples of 4 MiB or 8 MiB or more
which means in the end to recreate the disk partitions with an offset.
So an option to recreate the disk partitions with an offset
is needed anyway in ReaR and such an option can then
also be (mis)-used to avoid issues when recreating on a used disk
the same storage objects at exact same positions on the disk
(i.e. to avoid exact same positions when recreating same storage
objects)
in particular when recreating on the same hardware (i.e. when
the replacement hardware is the original system hardware).
Of course when we recreate the disk partitions with an offset
the next needed feature is to also adapt the sizes of nested
storage objects inside disk partitions like LVM volumes
which is yet another feature that we need since a long time
to make migration easier for the user.
jsmeix commented at 2020-11-12 14:35:¶
FYI and for completeness here a full "rear -D recover" terminal output
when recreating this storage structure
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 20G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 20G 0c58676a-bcb6-42be-8e1c-46a24d954ca7
`-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2 /dev/dm-0 /dev/sda2 crypt LVM2_member 20G zJalOt-2mjE-OrW4-MRxO-ErID-beeL-rRwzOp
|-/dev/mapper/system-swap /dev/dm-1 /dev/dm-0 lvm swap 2G [SWAP] a88670c6-43c7-4024-822f-f0fa0d00cfbc
|-/dev/mapper/system-root /dev/dm-2 /dev/dm-0 lvm btrfs 12.6G / bd326c18-0806-47d7-a740-97d5047d7de4
`-/dev/mapper/system-home /dev/dm-3 /dev/dm-0 lvm xfs 5.4G /home c9a5ebc9-3eac-4aa8-a768-731295af64a8
/dev/sdb /dev/sdb ata disk 1G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 307M fb79b19e-0e6d-4570-aa96-fa968d6e0795
| `-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1 /dev/dm-4 /dev/sdb1 crypt ext4 305M /luks1test 745a0d13-2b73-4a81-a81c-96906c45ef5a
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS 409M 30376f43-60fd-4fc7-af0c-fad8063d5a1a
`-/dev/mapper/luks2test /dev/dm-5 /dev/sdb2 crypt ext4 405M /luks2test 850e0fcc-6739-4190-9940-0b27cb82ee66
/dev/sr0 /dev/sr0 ata rom iso9660 657M 2020-01-08-15-17-54-22
/dev/sr1 /dev/sr1 ata rom iso9660 8.2G 2020-01-08-15-53-34-34
on replacement hardware where this storage structure
had been already recreated by a "rear recover" run before.
In the recovery system on the replacement hardware
only this partial storage structure is visible (i.e. only disks with
their partitions):
RESCUE linux-uxxi:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 20G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 20G 0c58676a-bcb6-42be-8e1c-46a24d954ca7
/dev/sdb /dev/sdb ata disk 1G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 307M fb79b19e-0e6d-4570-aa96-fa968d6e0795
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS 409M 30376f43-60fd-4fc7-af0c-fad8063d5a1a
/dev/sr0 /dev/sr0 ata rom iso9660 76.9M 2020-11-12-13-37-02-39
RESCUE linux-uxxi:~ # rear -D recover
Relax-and-Recover 2.6 / Git
Running rear recover (PID 810 date 2020-11-12 15:18:47)
Using log file: /var/log/rear/rear-linux-uxxi.log
Running workflow recover within the ReaR rescue/recovery system
Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
Started RPC portmapper 'rpcbind'.
RPC portmapper 'rpcbind' available.
Started rpc.statd.
RPC status rpc.statd available.
Using backup archive '/tmp/rear.0pEcVkENRxqJSI5/outputfs/linux-uxxi/backup.tar.gz'
Will do driver migration (recreating initramfs/initrd)
Calculating backup archive size
Backup archive size is 1.5G /tmp/rear.0pEcVkENRxqJSI5/outputfs/linux-uxxi/backup.tar.gz (compressed)
Comparing disks
Device sda has expected (same) size 21474836480 bytes (will be used for 'recover')
Device sdb has expected (same) size 1073741824 bytes (will be used for 'recover')
Disk configuration looks identical
UserInput -I DISK_LAYOUT_PROCEED_RECOVERY needed in /usr/share/rear/layout/prepare/default/250_compare_disks.sh line 148
Proceed with 'recover' (yes) otherwise manual disk layout configuration is enforced
(default 'yes' timeout 30 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: No choices - result is 'yes'
User confirmed to proceed with 'recover'
Marking component '/dev/sda' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sda1' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sda2' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sdb' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sdb1' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sdb2' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'pv:/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/system' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/system-home' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/system-root' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/system-swap' as done in /var/lib/rear/layout/disktodo.conf
Doing SLES-like btrfs subvolumes setup for /dev/mapper/system-root on / (BTRFS_SUBVOLUME_SLES_SETUP contains /dev/mapper/system-root)
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/mapper/system-root (default subvolume path contains '@/.snapshots/')
Marking component 'fs:/' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'fs:/luks1test' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'fs:/home' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/.snapshots' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/var' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/usr/local' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/srv' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/boot/grub2/x86_64-efi' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/root' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/tmp' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/boot/grub2/i386-pc' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'btrfsmountedsubvol:/opt' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'swap:/dev/mapper/system-swap' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/luks2test' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'fs:/luks2test' as done in /var/lib/rear/layout/disktodo.conf
Wiping child devices of /dev/sda in reverse ordering: /dev/sda2 /dev/sda1 /dev/sda
Wiped first 16777216 bytes of /dev/sda2
Wiped last 16777216 bytes of /dev/sda2
Wiped first 8388608 bytes of /dev/sda1
Skip wiping at the end of /dev/sda1 (dvice size 8388608 not greater than the bytes that were wiped)
Wiped first 16777216 bytes of /dev/sda
Wiped last 16777216 bytes of /dev/sda
Wiping child devices of /dev/sdb in reverse ordering: /dev/sdb2 /dev/sdb1 /dev/sdb
Wiped first 16777216 bytes of /dev/sdb2
Wiped last 16777216 bytes of /dev/sdb2
Wiped first 16777216 bytes of /dev/sdb1
Wiped last 16777216 bytes of /dev/sdb1
Wiped first 16777216 bytes of /dev/sdb
Wiped last 16777216 bytes of /dev/sdb
Start system layout restoration.
Disk '/dev/sda': creating 'gpt' partition table
Disk '/dev/sda': creating partition number 1 with name ''sda1''
Disk '/dev/sda': creating partition number 2 with name ''sda2''
Disk '/dev/sdb': creating 'gpt' partition table
Disk '/dev/sdb': creating partition number 1 with name ''sdb1''
Disk '/dev/sdb': creating partition number 2 with name ''sdb2''
Creating LUKS volume cr_ata-QEMU_HARDDISK_QM00004-part1 on /dev/sdb1
Set the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00004-part1 (for 'cryptsetup luksFormat' on /dev/sdb1):
Enter passphrase for /dev/sdb1:
Enter the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00004-part1 (for 'cryptsetup luksOpen' on /dev/sdb1):
Enter passphrase for /dev/sdb1:
Creating LUKS volume cr_ata-QEMU_HARDDISK_QM00001-part2 on /dev/sda2
Set the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00001-part2 (for 'cryptsetup luksFormat' on /dev/sda2):
Enter passphrase for /dev/sda2:
Enter the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00001-part2 (for 'cryptsetup luksOpen' on /dev/sda2):
Enter passphrase for /dev/sda2:
Creating LVM PV /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2
Restoring LVM VG 'system'
Sleeping 3 seconds to let udev or systemd-udevd create their devices...
Creating filesystem of type btrfs with mount point / on /dev/mapper/system-root.
Mounting filesystem /
Running snapper/installation-helper
Creating filesystem of type ext4 with mount point /luks1test on /dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1.
Mounting filesystem /luks1test
Creating filesystem of type xfs with mount point /home on /dev/mapper/system-home.
Mounting filesystem /home
Creating swap on /dev/mapper/system-swap
Creating LUKS volume luks2test on /dev/sdb2
Set the password for LUKS volume luks2test (for 'cryptsetup luksFormat' on /dev/sdb2):
Enter passphrase for /dev/sdb2:
Enter the password for LUKS volume luks2test (for 'cryptsetup luksOpen' on /dev/sdb2):
Enter passphrase for /dev/sdb2:
Creating filesystem of type ext4 with mount point /luks2test on /dev/mapper/luks2test.
Mounting filesystem /luks2test
Disk layout created.
Restoring from '/tmp/rear.0pEcVkENRxqJSI5/outputfs/linux-uxxi/backup.tar.gz' (restore log in /var/lib/rear/restore/recover.backup.tar.gz.810.restore.log) ...
Backup restore program 'tar' started in subshell (PID=4932)
Restored 128 MiB [avg. 43877 KiB/sec]
...
Restored 3073 MiB [avg. 34581 KiB/sec]
OK
Restored 3073 MiB in 94 seconds [avg. 33477 KiB/sec]
Restoring finished (verify backup restore log messages in /var/lib/rear/restore/recover.backup.tar.gz.810.restore.log)
Created SELinux /mnt/local/.autorelabel file : after reboot SELinux will relabel all files
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
Replacing restored udev rule '/mnt/local//etc/udev/rules.d/70-persistent-net.rules' with the one from the ReaR rescue system
Migrating restored network configuration files according to the mapping files ...
Running mkinitrd...
Recreated initrd (/sbin/mkinitrd).
Installing GRUB2 boot loader...
Determining where to install GRUB2 (no GRUB2_INSTALL_DEVICES specified)
Found possible boot disk /dev/sda - installing GRUB2 there
Finished 'recover'. The target system is mounted at '/mnt/local'.
Exiting rear recover (PID 810) and its descendant processes ...
Running exit tasks
You should also rm -Rf /tmp/rear.0pEcVkENRxqJSI5
Still in the recovery system on the replacement hardware
after "rear recover" the whole storage structure is visible again:
RESCUE linux-uxxi:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 20G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 20G 0c58676a-bcb6-42be-8e1c-46a24d954ca7
`-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2
/dev/dm-1 /dev/sda2 crypt LVM2_member 20G zJalOt-2mjE-OrW4-MRxO-ErID-beeL-rRwzOp
|-/dev/mapper/system-swap /dev/dm-2 /dev/dm-1 lvm swap 2G a88670c6-43c7-4024-822f-f0fa0d00cfbc
|-/dev/mapper/system-home /dev/dm-3 /dev/dm-1 lvm xfs 5.4G /mnt/local/home c9a5ebc9-3eac-4aa8-a768-731295af64a8
`-/dev/mapper/system-root /dev/dm-4 /dev/dm-1 lvm btrfs 12.6G /mnt/local bd326c18-0806-47d7-a740-97d5047d7de4
/dev/sdb /dev/sdb ata disk 1G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 307M fb79b19e-0e6d-4570-aa96-fa968d6e0795
| `-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1
| /dev/dm-0 /dev/sdb1 crypt ext4 305M /mnt/local/luks1test 745a0d13-2b73-4a81-a81c-96906c45ef5a
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS 409M 30376f43-60fd-4fc7-af0c-fad8063d5a1a
`-/dev/mapper/luks2test /dev/dm-5 /dev/sdb2 crypt ext4 405M /mnt/local/luks2test 850e0fcc-6739-4190-9940-0b27cb82ee66
/dev/sr0 /dev/sr0 ata rom iso9660 76.9M 2020-11-12-13-37-02-39
jsmeix commented at 2020-11-16 14:50:¶
With
https://github.com/rear/rear/pull/2514/commits/e107f317088a5578940b8130f155e049f01b5403
I get LVM remainders well removed on a used disk.
For now I tested without plain LVM that is not inside LUKS:
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 20G
`-/dev/sda1 /dev/sda1 /dev/sda part LVM2_member 20G
|-/dev/mapper/system-root /dev/dm-0 /dev/sda1 lvm ext4 7.2G /
|-/dev/mapper/system-swap /dev/dm-1 /dev/sda1 lvm swap 2G [SWAP]
`-/dev/mapper/system-home /dev/dm-2 /dev/sda1 lvm ext4 10.8G /home
Removing things inside LUKS comes next...
Here my "rear recover" excerpts
(again on a ystem where "rear recover" was already run before):
RESCUE # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 20G
`-/dev/sda1 /dev/sda1 /dev/sda part LVM2_member 20G
/dev/sr0 /dev/sr0 ata rom udf 71M
RESCUE # lvscan
inactive '/dev/system/home' [10.79 GiB] inherit
inactive '/dev/system/root' [7.20 GiB] inherit
inactive '/dev/system/swap' [2.00 GiB] inherit
RESCUE # vgscan
Reading all physical volumes. This may take a while...
Found volume group "system" using metadata type lvm2
RESCUE # pvscan
PV /dev/sda1 VG system lvm2 [20.00 GiB / 8.00 MiB free]
Total: 1 [20.00 GiB] / in use: 1 [20.00 GiB] / in no VG: 0 [0 ]
RESCUE # rear -D recover
...
User confirmed to proceed with 'recover'
Marking component '/dev/sda' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sda1' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'pv:/dev/sda1' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/system' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/system-home' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/system-root' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/system-swap' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'fs:/' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'fs:/home' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'swap:/dev/mapper/system-swap' as done in /var/lib/rear/layout/disktodo.conf
Removed LVM logical volume /dev/system/home
Removed LVM logical volume /dev/system/root
Removed LVM logical volume /dev/system/swap
Removed LVM volume group system
Removed LVM physical volume /dev/sda1
Wiping child devices of /dev/sda in reverse ordering: /dev/sda1 /dev/sda
Wiped first 16777216 bytes of /dev/sda1
Wiped last 16777216 bytes of /dev/sda1
Wiped first 16777216 bytes of /dev/sda
Wiped last 16777216 bytes of /dev/sda
Start system layout restoration.
Disk '/dev/sda': creating 'msdos' partition table
Disk '/dev/sda': creating partition number 1 with name 'primary'
Creating LVM PV /dev/sda1
Restoring LVM VG 'system'
Sleeping 3 seconds to let udev or systemd-udevd create their devices...
Creating filesystem of type ext4 with mount point / on /dev/mapper/system-root.
Mounting filesystem /
Creating filesystem of type ext4 with mount point /home on /dev/mapper/system-home.
Mounting filesystem /home
Creating swap on /dev/mapper/system-swap
Disk layout created.
Restoring from '/tmp/rear.NpjvSjqHZzWRnWe/outputfs/linux-u89h/backup.tar.gz' ...
jsmeix commented at 2020-11-17 15:34:¶
It could cause horrible damage if all LVM PVs VGs and LVs get removed
that pvscan/vgscan/lvscan can detect in the recovery system:
Assume there are three disks:
A 100 GiB /dev/sda where the basic operating system is stored
plus a 2 TiB /dev/sdb and a 3 TiB /dev/sdc where the actual data
is stored using LVM with PVs on /dev/sdb and /dev/sdc.
"rear mkbackup" only creates a backup of the files on /dev/sda.
For the about 5 TiB of actual data a separated backup is used.
Assume disk /dev/sda broke down and gets replaced
but /dev/sdb and /dev/sdc had no issues and are kept as is.
So "rear recover" must not touch anything on /dev/sdb or /dev/sdc
but pvscan/vgscan/lvscan detect LVM objects on all disks
in the recovery system.
Accordingly only those PVs get removed that are on a disk which will be
wiped.
Disks which will be wiped are the ones where
parted -s $disk mklabel $label
will be called in diskrestore.sh - i.e. disks that will be completely
overwritten.
This is the main reason behind for
https://github.com/rear/rear/pull/2514/commits/ae48d3ca8d8c251c71c6322bc99ee46ac7b61fa5
Additionally neither any VGs nor any LVs get removed because VGs and LVs
could be
spread over several disks so that it is not easily possible to determine
which disks belong
to VGs and LVs (stricly speaking it seems to be not possible with
reasonable effort,
at least not for now).
If time permits I will investigate if that is possible with reasonable
effort as follows:
LVs belong to a VG and a VG has several PVs that are on one or more
disks.
Those disks where the PVs for a VG are located are the disks that belong
to that VG
and the same disks belong to all LVs in that VG.
When all disks that belong to a VG will be wiped
then this VG and its LVs can be removed.
The reason behind why it is better to remove first LVs, then VGs, and
finally PVs is
that this way things work cleanly (no longer needed to forcefully remove
PVs with
loud WARNING messages that look as if things went wrong or are done
wrong).
jsmeix commented at 2020-11-17 16:00:¶
Another part that is implemented in
https://github.com/rear/rear/pull/2514/commits/ae48d3ca8d8c251c71c6322bc99ee46ac7b61fa5
is removing nested storage objects inside LUKS volumes.
To make nested storage objects in LUKS volumes visible
the LUKS volumes must be opened via
cryptsetup luksOpen $luks_device $luks_mapping_name
then nested storage objects in LUKS volumes can be removed
and finally those LUKS volumes must be closed again
via cryptsetup luksClose $luks_mapping_name
because leaving LUKS volumes open leads to errors later
when "parted -s $disk mklabel $label" is called in the diskrestore.sh
script
because then 'parted' fails with the following error message:
Partitions ... have been written, but we have been unable to inform
the kernel of the change, probably because it/they are in use.
As a result, the old partition(s) will remain in use.
You should probably reboot now before making further changes.
In this case it works to reboot the recovery system
and then a second "rear recover" usually succeeds
because after plain wiping a disk with LUKS volumes
the LUKS metadata/signatures are no longer there
so that LUKS cannot be in use in the rebooted recovery system
and "parted -s $disk mklabel $label" can succeed.
But we like to do all in one single "rear recover" run
so we need to clean up LUKS storage objects properly.
jsmeix commented at 2020-12-09 08:52:¶
@gdha
thank you for looking at this one!
In its current state it cleans up old LVM stuff on the disk
so you could try it out in its current state.
The next thing I like to add is "final power to the user"
i.e. a config variable so that the user can specify
if he wants to let ReaR wipe the disks before recreating
or what exact disks ReaR should wipe
or not wipe any disk at all.
With that I could merge it with by default disabled disk wipe
functionality
to get current things into master code so that interested users could
try it out
and provide feedback how good or bad (it could destroy unwanted stuff)
it works.
To try out this particular git branch that contains the current state of
this pull request
you could do something like
# git clone --single-branch --branch jsmeix-wipe-disks-before-recreating https://github.com/rear/rear.git
gdha commented at 2020-12-09 15:19:¶
@jsmeix will ask to redo my test next week to see if the wipe disks resolves the below issue during recovery.
+++ lvm vgcreate --physicalextentsize 4096k vg00 /dev/sda2 /dev/sde
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
A volume group called vg00 already exists.
The current work-around is the do a vgremove vg --force on the command
line before rear recover or by adding the next code in the vgcreation
part of diskrestore.sh like:
lvm vgremove vg02 --force || true
Both work-arounds are equal for success, however, I think your code
would have the same result.
gdha commented at 2020-12-09 15:29:¶
@jsmeix The file README.wipe_disks would probably better move to the
ReaR User Guide as it
is a bit invisible /usr/share/rear/layout/recreate/default directory
to the user.
jsmeix commented at 2020-12-09 15:33:¶
I placed the README.wipe_disks intentionally next to the scripts
because
it is not meant for the user but as background information for further
developers
of such scripts (I don't what to have all that as a comment in the
script itself ;-)
OliverO2 commented at 2020-12-10 22:46:¶
I have just stumbled into this and might not have seen enough. I
understand that wipefs is missing recursion and LUKS inspection
capabilities, and for example one needs to wipe GPT partitions before
wiping a GPT disk. But it is also my understanding that wipefs knows
(via libblkid) all "filesystem, raid or partition-table signatures".
So I wonder: Why is it not sufficient to use wipefs in a bottom-up
fashion with all of these file systems, volumes and partitions?
jsmeix commented at 2020-12-11 08:24:¶
In general see my personal experiences that I described in
layout/recreate/default/README.wipe_disks
and also in the comments in
layout/recreate/default/150_wipe_disks.sh
In particular wiping LVM stuff is horrible according to my personal
experience.
I found no simple and straightforward way how to generically wipe LVM
stuff.
To see the block devices that belong to LVM the LVM things need to be
activated
cf. layout/recreate/default/README.wipe_disks
but when LVM things are activated (in use) one cannot wipefs their
block devices
(similar as I won't wipefs a disk partition while its filesystem is
still mounted).
So - according to my personal experience - the only way to get rid of
LVM stuff
is to deconstruct the LVM stuff with LVM tools step by step from LVs to
VGs to PVs
cf. layout/recreate/default/150_wipe_disks.sh
But deconstructing LVM stuff step by step from LVs to VGs to PVs gets
impossible
when there are LVs that belong to VGs that contain disks that should not
be wiped.
So in my current layout/recreate/default/150_wipe_disks.sh I only
remove PVs
because things could get too complicated with VGs and LVs - at least too
complicated
for my current understanding - I am not a real LVM expert.
For example assume there is
a smaller /dev/sda for the operating system on LVs that are in a VG
"system"
and a huge /dev/sdb for a database on LVs that are in a VG "data".
The "rear mkbackup" backup is only for the operating system.
The database backup is a separated backup.
Over the time /dev/sda became too small for the operating system
so as band aid a PV /dev/sdb123 was added to the VG "system"
to enlarge the LVs that belong to the VG "system"
so some operating system LVM stuff is then also on sdb.
Years later that bad aid is still there - of course!
Then some soft-error destroyed the operating system
and it should be recreated with "rear recover" on same hardware.
Currently I don't know how to properly deconstruct
only the LVM stuff that belongs to the operating system
but not touch the LVM stuff that belongs to the database
in an automated way.
The "automated" is the crucial point here.
The admin who knows his system can of course manually deconstruct
only his operating system's LVM stuff before he runs "rear recover".
But when the LVM stuff is inside a LUKS volume and the one who runs
"rear recover"
does not know the LUKS password (e.g. because he got the replacement
hardware
with its used disk "as is") it is impossible to see the block devices
that belong to LVM
so one needs a low-level generic method to wipe things which is dd
cf. layout/recreate/default/README.wipe_disks
When one has the dd method there is no longer a need to use wipefs
because the dd method wipes more than what wipefs would do and
the dd method "just works" and doesn't sometimes ask the user
questions
or refuses to work as wipefs woud do unless called with -f or
wipes
incomplete unless called with -a or more unexpected things like that.
I think wipefs is meant to be used within a systern that currently
runs
so it must operate carefully to not destroy the running system.
In contrast what we need during "rear recover" is some generic "brutal"
method
that "just works" (as far as possible with reasonable effort) to wipe
the target disk.
If the recovery system and the backup is on a disk
(e.g. via OUTPUT=USB plus BACKUP_URL=usb:///)
we must carefully avoid to apply that generic "just works" method
by accident also on the disk where the backup is.
So my main problem is:
On the one hand I need a generic "brutal" method that "just works".
On the other hand that method must be very careful and not wipe
by accident things on wrong disks (cf. the above example where
that PV /dev/sdb123 is on a wrong disk).
I.e. I am looking for a "careful and brutal" method that "just works"
;-)
OliverO2 commented at 2020-12-11 23:43:¶
I see. I have never tried to wipe LVM stuff, so I wonder why this is really different. I've tried to play a bit with that stuff, so here are my thoughts:
-
I was wondering whether a limited amount of
ddoverwriting would be sufficient. For example, Btrfs can position an additional superblock at a location slightly above 25 TB(!):The primary superblock is located at 0x1 0000 (6410 KiB). Mirror copies of the superblock are located at physical addresses 0x400 0000 (6410 MiB) and 0x40 0000 0000 (25610 GiB), if these locations are valid. Superblock copies are updated simultaneously.
On the other hand:
Note that btrfs only recognizes disks with a valid 0x1 0000 superblock; otherwise, there would be confusion with other filesystems.
So if there is no code making the kernel look for signatures at high locations, all will be fine, but do we really know?
-
My experiences with wiping 'regular' disks have been positive so far as long as wipefs is used 'correctly':
-
wipefs /dev/XXX(without-a) always just prints the signatures it has found and does not actually wipe anything. I always use it for simulation like I would use other tools with a--dry-runoption. -
wipefs -a /dev/XXXwipes just that device or partition specified. It does so reliably. That device or partition will not come back. But for a device, any partitions inside have not been wiped and will now exist as 'ghost' partitions. They will come back once a matching partition table is re-created. -
To wipe a multi-partition disk, I always use
wipefs -a /dev/XXX?* /dev/XXX, which will wipe partitions first, then the partition table. No ghosts anymore. -
The
wipefsmanual page says about-f, --force:Force erasure, even if the filesystem is mounted. This is required in order to
erase a partition-table signature on a block device.This seems to be wrong. I was able to successfully wipe a GPT partition table on a block device without
--forceas long as no partition was actually mounted. -
wipefseven correctly detects and wipes Btrfs signatures on a Btrfs RAID1 combination. I was able to unmount the Btrfs file system, then wipe one drive, then usebtrfs replaceto bring it back. And Btrfs is quite picky. It will scan for file systems regardless of any attempts to actually mount them. And it will group former RAID1 member disks if they happen to contain the same file system UUID (so one must be carful when cloning disks).
-
-
Regarding LVM, this worked:
root@test-u20-04-s:~/rear# lvs -v LV VG #Seg Attr LSize Maj Min KMaj KMin Pool Origin Data% Meta% Move Cpy%Sync Log Convert LV UUID LProfile db vg_data 1 -wi-a----- 199.00g -1 -1 253 1 8lPIgd-l3RL-J3ve-dX0V-gMX8-sbcU-0SmtKi root vg_system 1 -wi-a----- 255.00g -1 -1 253 0 5cg23v-H5iU-gySJ-lj8x-sNOU-JT8D-W3hPsZ root@test-u20-04-s:~/rear# vgs -v VG Attr Ext #PV #LV #SN VSize VFree VG UUID VProfile vg_data wz--n- 4.00m 1 1 0 <200.00g 1020.00m rJFtmB-SKVi-9T4D-Fraz-7Xo7-rbmn-iQIUET vg_system wz--n- 4.00m 2 1 0 311.99g 56.99g seaOqS-AUiz-511n-GIav-Bgv9-RXht-3yZcKK root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk LVM2_member sdc 8:32 0 256G 0 disk ├─sdc1 8:33 0 200G 0 part LVM2_member Linux filesystem 47d3f897-a5d9-473e-982a-be7184df8b33 │ └─vg_data-db 253:1 0 199G 0 lvm └─sdc2 8:34 0 56G 0 part LVM2_member Linux filesystem 31e597a5-9fa4-4dfd-91cf-8059de6e4641 root@test-u20-04-s:~/rear# lvremove /dev/vg_*/* Do you really want to remove and DISCARD active logical volume vg_data/db? [y/n]: y Logical volume "db" successfully removed Do you really want to remove and DISCARD active logical volume vg_system/root? [y/n]: y Logical volume "root" successfully removed root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk LVM2_member sdc 8:32 0 256G 0 disk ├─sdc1 8:33 0 200G 0 part LVM2_member Linux filesystem 47d3f897-a5d9-473e-982a-be7184df8b33 └─sdc2 8:34 0 56G 0 part LVM2_member Linux filesystem 31e597a5-9fa4-4dfd-91cf-8059de6e4641 root@test-u20-04-s:~/rear# vgremove vg_data vg_system Volume group "vg_data" successfully removed Volume group "vg_system" successfully removed root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk LVM2_member sdc 8:32 0 256G 0 disk ├─sdc1 8:33 0 200G 0 part LVM2_member Linux filesystem 47d3f897-a5d9-473e-982a-be7184df8b33 └─sdc2 8:34 0 56G 0 part LVM2_member Linux filesystem 31e597a5-9fa4-4dfd-91cf-8059de6e4641 root@test-u20-04-s:~/rear# pvremove /dev/sdb /dev/sdc? Labels on physical volume "/dev/sdb" successfully wiped. Labels on physical volume "/dev/sdc1" successfully wiped. Labels on physical volume "/dev/sdc2" successfully wiped. root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk sdc 8:32 0 256G 0 disk ├─sdc1 8:33 0 200G 0 part Linux filesystem 47d3f897-a5d9-473e-982a-be7184df8b33 └─sdc2 8:34 0 56G 0 part Linux filesystem 31e597a5-9fa4-4dfd-91cf-8059de6e4641 root@test-u20-04-s:~/rear# wipefs -a /dev/sdb /dev/sdc?* /dev/sdc /dev/sdc: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54 /dev/sdc: 8 bytes were erased at offset 0x3ffffffe00 (gpt): 45 46 49 20 50 41 52 54 /dev/sdc: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa /dev/sdc: calling ioctl to re-read partition table: Success root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk sdc 8:32 0 256G 0 disk -
Regarding LUKS, you wrote in
README.wipe_disks:then it is impossible to wipe nested block devices inside LUKS volumes.
But: There is no need to wipe anything inside an encrypted LUKS container. Once the container is destroyed, there is no way that any 'ghost' structures inside might come back. Their data and metadata was encrypted and cannot be decrypted ever once the LUKS container's information is gone. So destructing a LUKS container leaves only harmless garbled data behind with no need to wipe any further. Rebuilding a new LUKS container on the same disk/partition will not interpret any data which might have been stored there previously.
And
wipefserases a LUKS container properly:root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk └─sdb1 8:17 0 256G 0 part 2ae405db-4889-45b1-8d45-8809bc842a11 sdc 8:32 0 256G 0 disk root@test-u20-04-s:~/rear# cryptsetup -v --cipher aes-xts-plain64 --key-size 512 --hash sha256 --iter-time 2000 --use-random --verify-passphrase luksFormat /dev/sdb1 WARNING! ======== This will overwrite data on /dev/sdb1 irrevocably. Are you sure? (Type uppercase yes): YES Enter passphrase for /dev/sdb1: Verify passphrase: Key slot 0 created. Command successful. root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk └─sdb1 8:17 0 256G 0 part crypto_LUKS 2ae405db-4889-45b1-8d45-8809bc842a11 sdc 8:32 0 256G 0 disk root@test-u20-04-s:~/rear# wipefs -a /dev/sdb? /dev/sdb1: 6 bytes were erased at offset 0x00000000 (crypto_LUKS): 4c 55 4b 53 ba be /dev/sdb1: 6 bytes were erased at offset 0x00004000 (crypto_LUKS): 53 4b 55 4c ba be root@test-u20-04-s:~/rear# lsblk -I 8 -o +FSTYPE,LABEL,PARTLABEL,PARTUUID NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT FSTYPE LABEL PARTLABEL PARTUUID sda 8:0 0 256G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi vfat 90ea8592-8662-4456-9c11-e27c08ebdcc4 ├─sda2 8:2 0 1G 0 part vfat RESCUE SYS Rescue System 9aa84d1c-10aa-49f4-b6ae-12819208115e └─sda3 8:3 0 254.5G 0 part / btrfs 1c8be966-ab4b-4afd-9dbd-114b68526e45 sdb 8:16 0 256G 0 disk └─sdb1 8:17 0 256G 0 part 2ae405db-4889-45b1-8d45-8809bc842a11 sdc 8:32 0 256G 0 disk
gdha commented at 2020-12-12 13:08:¶
@OliverO2 one remark about vgremove - if you use vgremove --force then
it will remove all lvols within the VG and remove also the UUID of the
VG.
gdha commented at 2020-12-12 13:10:¶
@jsmeix upcoming Thursday I have a change planned to test this wipe script during a recovery which is currently missing in the rear-2.4 version of RHEL 7/8 and then I have a case towards RedHat.
gdha commented at 2020-12-17 13:19:¶
@jsmeix As promised I give you an update of my tests:
- used your updated wipe scripts from this PR to the rear-2.4 (/usr/share/rear/layout/recreate/default/120_confirm_wipedisk_disks.sh, /usr/share/rear/layout/recreate/default/150_wipe_disks.sh, /usr/share/rear/lib/layout-functions.sh and /usr/share/rear/layout/prepare/GNU/Linux/100_include_partition_code.sh) and created an ISO + backup
- did a recover test (putty log: https://gist.github.com/gdha/0c8304dbbbaeebf51e74ad01bf8f2b63) - we see device sda gets wiped out, but then we are getting an error:
+++ lvm vgcreate --physicalextentsize 4096k vg00 /dev/sda2 /dev/sde
A volume group called vg00 already exists.
- When we edit script:
/usr/share/rear/layout/prepare/GNU/Linux/110_include_lvm_code.shand add line:
lvm vgremove $vg --force --yes >&2 || true
at line 136 it works fine - see gist https://gist.github.com/gdha/504ee70b3dda0867f1dfa7298f5604f3 that contains the restore log file, the diskrestore.sh script and the modification explanation details.
github-actions commented at 2021-02-16 02:01:¶
Stale pull request message
jsmeix commented at 2021-02-23 10:40:¶
When I find a bit more time for ReaR I will continue here
jsmeix commented at 2021-03-16 13:53:¶
With recent commits here there is DISKS_TO_BE_WIPED='false' in
default.conf
so that by default no disk is wiped to avoid regressions until this
feature was more tested.
jsmeix commented at 2021-03-16 13:56:¶
@rear/contributors
if there are no objections I would like to merge it tomorrow afternoon
in its current incomplete state so that it can be tested by interested
users who try out our current ReaR upstream master code
via explicit DISKS_TO_BE_WIPED='' in etc/rear/local.conf
jsmeix commented at 2021-03-17 13:14:¶
Works sufficiently well for me for my recent tests today
(i.e. works as the current code is intended to do)
so I will merge it now.
jsmeix commented at 2021-03-17 13:52:¶
@OliverO2
thank you for your valuable input in your comment above
https://github.com/rear/rear/pull/2514#issuecomment-743483945
As time permits I will have a closer look how I could improve my current code.
I think my current code is overcomplicated regarding LUKS encrypted
voulmes
i.e. I think you are right that one does not need to care about what is
inside
a LUKS encrypted voulme.
Regarding "safely and properly deconstruct/remove LVM things":
My primary concern is to ensure that never ever an automatism in ReaR
may accidentally destroy things on disks that should not be touched by
ReaR,
cf.
https://github.com/rear/rear/pull/2514#issuecomment-729009274
To avoid that basic storage objects like partitions get destroyed
on disks that should not be touched by ReaR I set
DISKS_TO_BE_WIPED to those disks where
in diskrestore.sh the create_disk_label function is called
(the create_disk_label function calls "parted -s $disk mklabel
$label")
i.e. those disks where things will be completely recreated from scratch.
I was thinking I could do the same for LVM storage objects:
Inspect diskrestore.sh and determine those LVs VGs and PVs
that will be completely recreated from scratch.
As far as I see this is basically what
https://github.com/rear/rear/pull/2564
implements (but there only for VGs).
BUT:
The LVs VGs and PVs in diskrestore.sh that will be completely recreated
from scratch
don't need to be those that exist on an already used disk that is used
for recovery.
Only when the system should be recreated on the unchanged hardware
of the original system (e.g. to recover from soft errors), then
LVs VGs and PVs exist on the unchanged system disk(s)
which match the LVs VGs and PVs in diskrestore.sh
that will be completely recreated from scratch.
So removing the LVs VGs and PVs in diskrestore.sh
that will be completely recreated from scratch can be done in any case
and it helps to recreate a system on the unchanged hardware of the
original system
but this is not a complete solution to get rid of existing LVM stuff
on an arbitrary already used replacement disk.
On the other hand existing LVM stuff on an arbitrary already used
replacement disk
should usually not get in the way when it is not the unchanged disk of
the original system
because partitions are likely recreated at different locations compare
to where they
have been before on the already used replacement disk so the existing
LVM stuff
will not automatically re-appear because the LVM metadata will not
re-appear
at the expected place in the recreated partitions.
So removing the LVs VGs and PVs in diskrestore.sh
that will be completely recreated from scratch is likely a
sufficiently complete way to get rid of existing LVM stuff in practice.
I will think a bit more about that...
jsmeix commented at 2021-06-25 06:11:¶
Currently we have in usr/share/rear/layout/recreate/default/150_wipe_disks.sh
# Wipe RAID plus LVM plus LUKS metadata.
# To wipe RAID Superblocks it is sufficient to wipe 133 KiB at the beginning and at the end of the device.
# To wipe to wipe LVM metadata is should be sufficient to wipe 4 MiB at the beginning and at the end of the device.
# To wipe LUKS headers is should be sufficient to wipe 8 MiB at the beginning of the device.
# To wipe RAID superblocks plus LVM metadata plus LUKS headers it should be sufficient to
# wipe 8 MiB + 4 MiB + 1 MiB = 13 MiB at the beginning of the device and to
# wipe 4 MiB + 1 MiB = 5 MiB at the end of the device.
# To be future proof (perhaps LUKS may add a backup header at the end of the device)
# wiping 16 MiB at the beginning and at the end of the device should be sufficiently safe.
based on what I collected about size and location of metadata in
usr/share/rear/layout/recreate/default/README.wipe_disks
Now I spotted by luck (more pecisely because of
https://github.com/rear/rear/issues/2638)
https://github.com/rear/rear/issues/540#issuecomment-71819883
that reads
Regarding clean up DRBD I found in
https://www.suse.com/documentation/sle_ha/book_sleha/data/sec_ha_drbd_overview.html
DRBD uses the last 128 MB of the raw device for metadata
So wiping 16 MiB at the beginning and at the end of the device
seems to be not sufficiently safe in case of DRBD.
To enhance usr/share/rear/layout/recreate/default/150_wipe_disks.sh
to also wipe DRBD (plus RAID plus LVM plus LUKS) metadata
wiping 128 MiB (at the beginning and) at the end of the device
should be sufficiently safe.
[Export of Github issue for rear/rear.]