#2726 PR merged: Automatically resize active last partition on RAID1¶
Labels: enhancement, fixed / solved / done
jsmeix opened issue at 2021-12-09 14:43:¶
Automatically resize active last partitions
also on RAID1 array component disks
-
Type: Enhancement
-
Impact: High
-
Reference to related issue (URL):
https://github.com/rear/rear/issues/2696 -
How was this pull request tested?
Works well for me with my simple RAID1 setup,
see my results in the comments below.
jsmeix commented at 2021-12-09 14:54:¶
Original system:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk linux_raid_member any:raid1sdab 12G
`-/dev/md127 /dev/md127 /dev/sda raid1 12G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11.9G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11.9G /
/dev/sdb /dev/sdb ata disk 5G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part 8M
|-/dev/sdb2 /dev/sdb2 /dev/sdb part vfat REAR-EFI 512M
|-/dev/sdb3 /dev/sdb3 /dev/sdb part ext2 MY-BOOT 1G
`-/dev/sdb4 /dev/sdb4 /dev/sdb part ext3 MY-DATA 3.5G
/dev/sdc /dev/sdc ata disk linux_raid_member any:raid1sdab 12G
`-/dev/md127 /dev/md127 /dev/sdc raid1 12G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11.9G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11.9G /
/dev/sdd /dev/sdd ata disk 1G
`-/dev/sdd1 /dev/sdd1 /dev/sdd part swap 1023M [SWAP]
# grep -v '^#' etc/rear/local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://192.168.122.1/nfs
REQUIRED_PROGS+=( snapper chattr )
PROGS+=( lsattr )
COPY_AS_IS+=( /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
BACKUP_PROG_INCLUDE=( /boot/grub2/x86_64-efi /home /boot/grub2/i386-pc /root /srv /opt /tmp /usr/local /var )
POST_RECOVERY_SCRIPT=( 'if snapper --no-dbus -r $TARGET_FS_ROOT get-config | grep -q "^QGROUP.*[0-9]/[0-9]" ; then snapper --no-dbus -r $TARGET_FS_ROOT set-config QGROUP= ; snapper --no-dbus -r $TARGET_FS_ROOT setup-quota && echo snapper setup-quota done || echo snapper setup-quota failed ; else echo snapper setup-quota not used ; fi' )
SSH_ROOT_PASSWORD='rear'
USE_DHCLIENT="yes"
FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )
PROGRESS_MODE="plain"
PROGRESS_WAIT_SECONDS="3"
LUKS_CRYPTSETUP_OPTIONS+=" --force-password"
GRUB2_INSTALL_DEVICES="/dev/sda /dev/sdc"
DISKS_TO_BE_WIPED="/dev/sd[a-z]"
WRITE_PROTECTED_IDS+=( fe057b67-8e9f-4b21-a67e-289a83fb3993 )
# grep -v '^#' var/lib/rear/layout/disklayout.conf
disk /dev/sda 12884901888 gpt
disk /dev/sdc 12884901888 gpt
disk /dev/sdd 1073741824 gpt
part /dev/sdd 1072676352 1048576 rear-noname swap /dev/sdd1
raid /dev/md127 level=raid1 raid-devices=2 devices=/dev/sda,/dev/sdc name=raid1sdab metadata=1.0 uuid=8d05eb84:2de831d1:dfed54b2:ad592118
part /dev/md127 10485760 1048576 rear-noname bios_grub /dev/md127p1
part /dev/md127 12739067392 11534336 rear-noname none /dev/md127p2
fs /dev/mapper/cr_root / btrfs uuid=85406026-0559-4b0d-8f67-ec19d3b556f5 label= options=rw,relatime,space_cache,subvolid=256,subvol=/@
...
swap /dev/sdd1 uuid=9c606f48-92cd-4f98-be22-0f8a75358bed label=
crypt /dev/mapper/cr_root /dev/md127p2 type=luks1 cipher=aes-xts-plain64 key_size=512 hash=sha256 uuid=d0446c00-9e79-4872-abaa-2d464fd71c99
Replacement system has same disk sizes available
for my first test so no actual resizing is needed:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk linux_raid_member localhost:raid1sdab 12G
|-/dev/sda1 /dev/sda1 /dev/sda part linux_raid_member localhost:raid1sdab 10M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS localhost:raid1sdab 11.9G
/dev/sdb /dev/sdb ata disk linux_raid_member localhost:raid1sdab 12G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part linux_raid_member localhost:raid1sdab 10M
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS localhost:raid1sdab 11.9G
/dev/sdc /dev/sdc ata disk 1G
`-/dev/sdc1 /dev/sdc1 /dev/sdc part swap 1023M
# rear -D recover
...
Comparing disks
Ambiguous disk layout needs manual configuration (more than one disk with same size used in '/var/lib/rear/layout/disklayout.conf')
Switching to manual disk layout configuration
Using /dev/sda (same name and same size 12884901888) for recreating /dev/sda
Cannot use /dev/sda (same size) for recreating /dev/sdc (/dev/sda already exists as target in /var/lib/rear/layout/disk_mappings)
Using /dev/sdb (same size 12884901888) for recreating /dev/sdc
Using /dev/sdc (same size 1073741824) for recreating /dev/sdd
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdc => /dev/sdb
/dev/sdd => /dev/sdc
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 315
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) n/a
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
...
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/config/df.txt
Applied disk layout mappings to /etc/rear/rescue.conf
Examining gpt disk /dev/sdc to automatically resize its last active partition
Skipping /dev/sdc (size of new disk same as size of old disk)
Examining gpt disk /dev/md127 to automatically resize its last active partition
ERROR: No '/sys/block/md127' directory for /dev/md127
Some latest log messages since the last called script 420_autoresize_last_partitions.sh:
2021-12-09 15:32:21.506947921 Entering debugscript mode via 'set -x'.
2021-12-09 15:32:21.528062250 Examining gpt disk /dev/sdc to automatically resize its last active partition
1073741824
2021-12-09 15:32:21.536517412 Skipping /dev/sdc (size of new disk same as size of old disk)
2021-12-09 15:32:21.555336860 Examining gpt disk /dev/md127 to automatically resize its last active partition
Aborting due to an error, check /var/log/rear/rear-localhost.log for details
jsmeix commented at 2021-12-10 07:57:¶
Now it works at least in my case where the
replacement system has same disk sizes available
for my first test so no actual resizing is needed:
# rear -D recover
...
Comparing disks
Ambiguous disk layout needs manual configuration (more than one disk with same size used in '/var/lib/rear/layout/disklayout.conf')
Switching to manual disk layout configuration
Using /dev/sda (same name and same size 12884901888) for recreating /dev/sda
Cannot use /dev/sda (same size) for recreating /dev/sdc (/dev/sda already exists as target in /var/lib/rear/layout/disk_mappings)
Using /dev/sdb (same size 12884901888) for recreating /dev/sdc
Using /dev/sdc (same size 1073741824) for recreating /dev/sdd
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdc => /dev/sdb
/dev/sdd => /dev/sdc
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 315
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) n/a
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/config/df.txt
Applied disk layout mappings to /etc/rear/rescue.conf
Examining gpt disk /dev/sdc to automatically resize its last active partition
Skipping /dev/sdc (size of new disk same as size of old disk)
Examining gpt disk /dev/md127 to automatically resize its last active partition
Skipping /dev/md127 (size of new disk same as size of old disk)
jsmeix commented at 2021-12-10 09:30:¶
Now it works at least in my case when the replacement system has
one smaller disk size and one same disk size for the RAID1 component
devices
which means the last partition on the RAID device must be shrinked
so that it still fits on the smaller component device disk.
On the recovery system I have now:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk linux_raid_member localhost:raid1sdab 12G
|-/dev/sda1 /dev/sda1 /dev/sda part linux_raid_member localhost:raid1sdab 10M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS localhost:raid1sdab 11.9G
/dev/sdb /dev/sdb ata disk 11G
/dev/sdc /dev/sdc ata disk 1G
`-/dev/sdc1 /dev/sdc1 /dev/sdc part swap 1023M
Because I reduced the /dev/sdb size from 12GiB to 11GiB I need additionally
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=20
Then "rear recover" works well for me:
# rear -D recover
...
Comparing disks
Ambiguous disk layout needs manual configuration (more than one disk with same size used in '/var/lib/rear/layout/disklayout.conf')
Switching to manual disk layout configuration
Using /dev/sda (same name and same size 12884901888) for recreating /dev/sda
Cannot use /dev/sda (same size) for recreating /dev/sdc (/dev/sda already exists as target in /var/lib/rear/layout/disk_mappings)
Could not automap /dev/sdc (no disk with same size 12884901888 found)
Using /dev/sdc (same size 1073741824) for recreating /dev/sdd
Original disk /dev/sdc does not exist (with same size) in the target system
Cannot check write protection by ID for /dev/sdb (no ID found)
Using /dev/sdb (the only available of the disks) for recreating /dev/sdc
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdd => /dev/sdc
/dev/sdc => /dev/sdb
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 315
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) n/a
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/config/df.txt
Applied disk layout mappings to /etc/rear/rescue.conf
Trying to automatically resize last partition when disk size changed
Examining gpt device /dev/sdc to automatically resize its last active partition
Skipping /dev/sdc (size of new device same as size of old device)
Examining gpt device /dev/md127 to automatically resize its last active partition
Checking /dev/md127p1 if it is the last partition on /dev/md127
Checking /dev/md127p2 if it is the last partition on /dev/md127
Found 'rear-noname' partition /dev/md127p2 as last partition on /dev/md127
Determining if last partition /dev/md127p2 is resizeable
Determining new size for last partition /dev/md127p2
Determining if last partition /dev/md127p2 actually needs to be increased or shrinked
New /dev/md127 is 1073741824 bytes smaller than old device
Last partition /dev/md127p2 must be shrinked by 940490240 bytes to still fit on device
Shrinking last partition /dev/md127p2 to end of device (new device at most 20% smaller)
Changed last partition /dev/md127p2 size from 12739067392 to 11798577152 bytes
...
Start system layout restoration.
Disk '/dev/sdc': creating 'gpt' partition table
Disk '/dev/sdc': creating partition number 1 with name ''sdc1''
Creating software RAID /dev/md127
Disk '/dev/md127': creating 'gpt' partition table
Disk '/dev/md127': creating partition number 1 with name ''md127p1''
Disk '/dev/md127': creating partition number 2 with name ''md127p2''
Creating swap on /dev/sdc1
Creating LUKS volume cr_root on /dev/md127p2
Set the password for LUKS volume cr_root (for 'cryptsetup luksFormat' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Enter the password for LUKS volume cr_root (for 'cryptsetup luksOpen' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Creating filesystem of type btrfs with mount point / on /dev/mapper/cr_root.
Mounting filesystem /
Disk layout created.
...
Finished 'recover'. The target system is mounted at '/mnt/local'.
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk linux_raid_member localhost:raid1sdab 12G
|-/dev/sda1 /dev/sda1 /dev/sda part linux_raid_member localhost:raid1sdab 10M
|-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS localhost:raid1sdab 11G
`-/dev/md127 /dev/md127 /dev/sda raid1 11G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11G /mnt/local
/dev/sdb /dev/sdb ata disk linux_raid_member localhost:raid1sdab 11G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part linux_raid_member localhost:raid1sdab 10M
|-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS localhost:raid1sdab 11G
`-/dev/md127 /dev/md127 /dev/sdb raid1 11G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11G /mnt/local
/dev/sdc /dev/sdc ata disk 1G
`-/dev/sdc1 /dev/sdc1 /dev/sdc part swap 1023M
and after reboot (here with size values as bytes):
# lsblk -bipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk linux_raid_member localhost:raid1sdab 12884901888
`-/dev/md127 /dev/md127 /dev/sda raid1 11811028992
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10485760
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11798577152
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11796480000 /
/dev/sdb /dev/sdb ata disk linux_raid_member localhost:raid1sdab 11811160064
`-/dev/md127 /dev/md127 /dev/sdb raid1 11811028992
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10485760
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11798577152
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11796480000 /
/dev/sdc /dev/sdc ata disk 1073741824
`-/dev/sdc1 /dev/sdc1 /dev/sdc part swap 1072676352 [SWAP]
/dev/sr0 /dev/sr0 ata rom iso9660 REAR-ISO 74227712
The relevant change in disklayout.conf regarding the shrinked partition is
-part /dev/md127 12739067392 11534336 rear-noname none /dev/md127p2
+part /dev/md127 11798577152 11534336 rear-noname none /dev/md127p2
jsmeix commented at 2021-12-10 10:38:¶
Another test with
sda shrinked from 12GiB to 10 GiB
sdb shrinked from 12GiB to 11 GiB
sdc enlarged from 1GiB to 2GiB
so I have now on the replacement VM
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 10G
/dev/sdb /dev/sdb ata disk linux_raid_member localhost:raid1sdab 11G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part linux_raid_member localhost:raid1sdab 10M
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS localhost:raid1sdab 11G
/dev/sdc /dev/sdc ata disk 2G
# rear -D recover
...
Comparing disks
Ambiguous disk layout needs manual configuration (more than one disk with same size used in '/var/lib/rear/layout/disklayout.conf')
Switching to manual disk layout configuration
Could not automap /dev/sda (no disk with same size 12884901888 found)
Could not automap /dev/sdc (no disk with same size 12884901888 found)
Could not automap /dev/sdd (no disk with same size 1073741824 found)
Original disk /dev/sda does not exist (with same size) in the target system
Cannot check write protection by ID for /dev/sda (no ID found)
Cannot check write protection by ID for /dev/sdc (no ID found)
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDA needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 278
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) /dev/sdc
4) Do not map /dev/sda
5) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sda'
Using /dev/sda (chosen by user) for recreating /dev/sda
Original disk /dev/sdc does not exist (with same size) in the target system
Cannot check write protection by ID for /dev/sdc (no ID found)
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDC needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 278
Choose an appropriate replacement for /dev/sdc
1) /dev/sdb
2) /dev/sdc
3) Do not map /dev/sdc
4) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sdb'
Using /dev/sdb (chosen by user) for recreating /dev/sdc
Original disk /dev/sdd does not exist (with same size) in the target system
Cannot check write protection by ID for /dev/sdc (no ID found)
Using /dev/sdc (the only available of the disks) for recreating /dev/sdd
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdc => /dev/sdb
/dev/sdd => /dev/sdc
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 315
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) n/a
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/config/df.txt
Applied disk layout mappings to /etc/rear/rescue.conf
Trying to automatically resize last partition when disk size changed
Examining gpt device /dev/sdc to automatically resize its last active partition
Checking /dev/sdc1 if it is the last partition on /dev/sdc
Found 'rear-noname' partition /dev/sdc1 as last partition on /dev/sdc
Determining if last partition /dev/sdc1 is resizeable
Last partition /dev/sdc1 not resizeable (used as swap partition)
Determining new size for last partition /dev/sdc1
Determining if last partition /dev/sdc1 actually needs to be increased or shrinked
New /dev/sdc is 1073741824 bytes bigger than old device
Skip increasing last partition /dev/sdc1 (not resizeable)
Examining gpt device /dev/md127 to automatically resize its last active partition
Checking /dev/md127p1 if it is the last partition on /dev/md127
Checking /dev/md127p2 if it is the last partition on /dev/md127
Found 'rear-noname' partition /dev/md127p2 as last partition on /dev/md127
Determining if last partition /dev/md127p2 is resizeable
Determining new size for last partition /dev/md127p2
Determining if last partition /dev/md127p2 actually needs to be increased or shrinked
New /dev/md127 is 2147483648 bytes smaller than old device
Last partition /dev/md127p2 must be shrinked by 2014232064 bytes to still fit on device
Shrinking last partition /dev/md127p2 to end of device (new device at most 20% smaller)
Changed last partition /dev/md127p2 size from 12739067392 to 10724835328 bytes
A side note about specifying the LUKS password
(i.e. how "rear recover" behaves when one fails to type it in
correctly):
I must have mistyped the LUKS password at the
for 'cryptsetup luksFormat' on /dev/md127p2 password input prompt
because then my password did not work at the subsequent
'cryptsetup luksOpen' on /dev/md127p2 password input prompt.
So I typed [Ctrl]+[C] at the 'cryptsetup luksOpen' input prompt
which aborts 'cryptsetup luksOpen'
which lets diskrestore.sh fail
which results the possibility to rerun diskrestore.sh
which nicely detected what was already created
and resumes at the 'cryptsetup luksFormat' password prompt:
Start system layout restoration.
Disk '/dev/sdc': creating 'gpt' partition table
Disk '/dev/sdc': creating partition number 1 with name ''sdc1''
Creating software RAID /dev/md127
Disk '/dev/md127': creating 'gpt' partition table
Disk '/dev/md127': creating partition number 1 with name ''md127p1''
Disk '/dev/md127': creating partition number 2 with name ''md127p2''
Creating swap on /dev/sdc1
Creating LUKS volume cr_root on /dev/md127p2
Set the password for LUKS volume cr_root (for 'cryptsetup luksFormat' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Enter the password for LUKS volume cr_root (for 'cryptsetup luksOpen' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Enter passphrase for /dev/md127p2:
Enter passphrase for /dev/md127p2: UserInput -I LAYOUT_CODE_RUN needed in /usr/share/rear/layout/recreate/default/200_run_layout_code.sh line 127
The disk layout recreation script failed
1) Rerun disk recreation script (/var/lib/rear/layout/diskrestore.sh)
2) View 'rear recover' log file (/var/log/rear/rear-localhost.log)
3) Edit disk recreation script (/var/lib/rear/layout/diskrestore.sh)
4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Rerun disk recreation script (/var/lib/rear/layout/diskrestore.sh)'
User reruns disk recreation script
Start system layout restoration.
Skipping /dev/sda (disk) as it has already been created.
Skipping /dev/sdb (disk) as it has already been created.
Skipping /dev/sdc (disk) as it has already been created.
Skipping /dev/sdc1 (part) as it has already been created.
Skipping /dev/md127 (raid) as it has already been created.
Skipping /dev/md127p1 (part) as it has already been created.
Skipping /dev/md127p2 (part) as it has already been created.
Skipping swap:/dev/sdc1 (swap) as it has already been created.
Creating LUKS volume cr_root on /dev/md127p2
Set the password for LUKS volume cr_root (for 'cryptsetup luksFormat' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Enter the password for LUKS volume cr_root (for 'cryptsetup luksOpen' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Creating filesystem of type btrfs with mount point / on /dev/mapper/cr_root.
Mounting filesystem /
Disk layout created.
and all went fine then.
After reboot I have now
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk linux_raid_member localhost:raid1sdab 10G
`-/dev/md127 /dev/md127 /dev/sda raid1 10G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 10G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 10G /
/dev/sdb /dev/sdb ata disk linux_raid_member localhost:raid1sdab 11G
`-/dev/md127 /dev/md127 /dev/sdb raid1 10G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 10G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 10G /
/dev/sdc /dev/sdc ata disk 2G
`-/dev/sdc1 /dev/sdc1 /dev/sdc part swap 1023M [SWAP]
/dev/sr0 /dev/sr0 ata rom iso9660 REAR-ISO 70.8M
jsmeix commented at 2021-12-10 14:19:¶
With
https://github.com/rear/rear/pull/2726/commits/8ff57127be0ac673b9ae23c65ed2fdda037b8e2c
I added user info about old and new disks and their sizes in case of
MIGRATION_MODE
to make it easier to answer the questions to which new disks old disk
should be mapped
so now that part of "rear recover" looks like
# rear -D recover
...
Comparing disks
Ambiguous disk layout needs manual configuration (more than one disk with same size used in '/var/lib/rear/layout/disklayout.conf')
Switching to manual disk layout configuration
/dev/sda had size 12884901888 (12 GiB) but is now 10737418240 (10 GiB)
/dev/sdc had size 12884901888 (12 GiB) but is now 2147483648 (2 GiB)
/dev/sdd with size 1073741824 (1 GiB) does no longer exist
/dev/sdb with size 11811160064 (11 GiB) did not exist on the original system
Could not automap /dev/sda (no disk with same size 12884901888 found)
Could not automap /dev/sdc (no disk with same size 12884901888 found)
Could not automap /dev/sdd (no disk with same size 1073741824 found)
Original disk /dev/sda does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDA needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 278
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) /dev/sdc
4) Do not map /dev/sda
5) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sda'
Using /dev/sda (chosen by user) for recreating /dev/sda
Original disk /dev/sdc does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDC needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 278
Choose an appropriate replacement for /dev/sdc
1) /dev/sdb
2) /dev/sdc
3) Do not map /dev/sdc
4) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sdb'
Using /dev/sdb (chosen by user) for recreating /dev/sdc
Original disk /dev/sdd does not exist (with same size) in the target system
Using /dev/sdc (the only available of the disks) for recreating /dev/sdd
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdc => /dev/sdb
/dev/sdd => /dev/sdc
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 315
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) n/a
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
I think I will improve those messages to
/dev/sda had size 12884901888 (12 GiB) but is now 10737418240 (10 GiB)
/dev/sdc had size 12884901888 (12 GiB) but is now 2147483648 (2 GiB)
/dev/sdd had size 1073741824 (1 GiB) but does no longer exist
/dev/sdb was not used on the original system and has now 11811160064 (11 GiB)
to have a consistent odering of the information in each line as
device info what was before info what is now
In particular /dev/sdb ... did not exist on the original system is
wrong
because it existed on the original system (cf. above)
/dev/sdb /dev/sdb ata disk 5G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part 8M
|-/dev/sdb2 /dev/sdb2 /dev/sdb part vfat REAR-EFI 512M
|-/dev/sdb3 /dev/sdb3 /dev/sdb part ext2 MY-BOOT 1G
`-/dev/sdb4 /dev/sdb4 /dev/sdb part ext3 MY-DATA 3.5G
but was not used (mounted) there
excerpt from "rear -D mkbackup"
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Automatically excluding disk /dev/sdb (not used by any mounted filesystem)
Marking component '/dev/sdb' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb1 is a child of component /dev/sdb
Marking component '/dev/sdb1' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb2 is a child of component /dev/sdb
Marking component '/dev/sdb2' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb3 is a child of component /dev/sdb
Marking component '/dev/sdb3' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb4 is a child of component /dev/sdb
Marking component '/dev/sdb4' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdb' in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdb' in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
jsmeix commented at 2021-12-13 11:51:¶
With latest commit
https://github.com/rear/rear/pull/2726/commits/0b7666096dc7865e45611ef299d2ed6041840322
it looks like this
# rear -D recover
...
Comparing disks
Ambiguous disk layout needs manual configuration (more than one disk with same size used in '/var/lib/rear/layout/disklayout.conf')
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 12884901888 (12 GiB) but is now 10737418240 (10 GiB)
/dev/sdc had size 12884901888 (12 GiB) but is now 2147483648 (2 GiB)
/dev/sdd had size 1073741824 (1 GiB) but it does no longer exist
/dev/sdb had size 5368709120 (5 GiB) but is now 11811160064 (11 GiB)
Could not automap /dev/sda (no disk with same size 12884901888 found)
Could not automap /dev/sdc (no disk with same size 12884901888 found)
Could not automap /dev/sdd (no disk with same size 1073741824 found)
Original disk /dev/sda does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDA needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 278
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) /dev/sdc
4) Do not map /dev/sda
5) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
jsmeix commented at 2021-12-13 13:55:¶
Verified that things still work OK on non-RAID systems:
Original VM:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 10G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part ext4 10G / d4d518c3-48f5-4806-b659-32a33f5bd1b5
/dev/sdb /dev/sdb ata disk 8G
`-/dev/sdb1 /dev/sdb1 /dev/sdb part ext4 8G /home f21e8e44-2f7e-427a-90c8-a0d637e37968
/dev/sr0 /dev/sr0 ata rom 1024M
# grep -v '^#' var/lib/rear/layout/disklayout.conf
disk /dev/sda 10737418240 gpt
part /dev/sda 8388608 1048576 rear-noname bios_grub /dev/sda1
part /dev/sda 10727964160 9437184 rear-noname legacy_boot /dev/sda2
disk /dev/sdb 8589934592 gpt
part /dev/sdb 8588869120 1048576 rear-noname none /dev/sdb1
fs /dev/sda2 / ext4 uuid=d4d518c3-48f5-4806-b659-32a33f5bd1b5 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16369 default_mount_options=user_xattr,acl options=rw,relatime
fs /dev/sdb1 /home ext4 uuid=f21e8e44-2f7e-427a-90c8-a0d637e37968 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16381 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
Replacement VM:
RESCUE localhost:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 7G
/dev/sdb /dev/sdb ata disk 5G
/dev/sr0 /dev/sr0 ata rom iso9660 REAR-ISO 69M 2021-12-13-14-28-21-33
RESCUE localhost:~ # rear -D recover
...
Comparing disks
Cannot check write protection by ID for /dev/sda (no ID found)
Cannot check write protection by ID for /dev/sdb (no ID found)
Device sda has size 7516192768 bytes but 10737418240 bytes is expected (needs manual configuration)
Device sdb has size 5368709120 bytes but 8589934592 bytes is expected (needs manual configuration)
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 10737418240 (10 GiB) but is now 7516192768 (7 GiB)
/dev/sdb had size 8589934592 (8 GiB) but is now 5368709120 (5 GiB)
/dev/sr0 was not used on the original system and has now 72333312 (0 GiB)
Could not automap /dev/sda (no disk with same size 10737418240 found)
Could not automap /dev/sdb (no disk with same size 8589934592 found)
Original disk /dev/sda does not exist (with same size) in the target system
Cannot check write protection by ID for /dev/sda (no ID found)
Cannot check write protection by ID for /dev/sdb (no ID found)
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDA needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 278
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) Do not map /dev/sda
4) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sda'
Using /dev/sda (chosen by user) for recreating /dev/sda
Original disk /dev/sdb does not exist (with same size) in the target system
Cannot check write protection by ID for /dev/sdb (no ID found)
Using /dev/sdb (the only available of the disks) for recreating /dev/sdb
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdb => /dev/sdb
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 315
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Confirm identical disk mapping and proceed without manual configuration
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
"rear recover" works well for me
but of course I needed things like
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=50
AUTORESIZE_PARTITIONS=( /dev/sda2 /dev/sdb1 )
to enforce autoresizing in this case, cf. default.conf about
Resizing partitions in MIGRATION_MODE during "rear recover"
jsmeix commented at 2021-12-13 13:57:¶
@rear/contributors
things work sufficiently well for me
so I would like to merge it tomorrow afternoon
provided there are no objections.
jsmeix commented at 2021-12-14 10:25:¶
With the last three commits things work better:
Now AUTORESIZE_EXCLUDE_PARTITIONS=() can be set by the user:
# grep -v '^#' etc/rear/local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://192.168.122.1/nfs
SSH_ROOT_PASSWORD='rear'
USE_DHCLIENT="yes"
FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )
PROGRESS_MODE="plain"
PROGRESS_WAIT_SECONDS="3"
DISKS_TO_BE_WIPED="/dev/sd[a-z]"
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=50
AUTORESIZE_EXCLUDE_PARTITIONS=()
And things look better during "rear recover":
Now CDROM devices are no longer listed as disks:
RESCUE localhost:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 7G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part ext4 7G d4d518c3-48f5-4806-b659-32a33f5bd1b5
/dev/sdb /dev/sdb ata disk 5G
`-/dev/sdb1 /dev/sdb1 /dev/sdb part ext4 5G f21e8e44-2f7e-427a-90c8-a0d637e37968
/dev/sr0 /dev/sr0 ata rom iso9660 REAR-ISO 69M 2021-12-14-10-35-14-96
RESCUE localhost:~ # rear -D recover
...
Comparing disks
Device sda has size 7516192768 bytes but 10737418240 bytes is expected (needs manual configuration)
Device sdb has size 5368709120 bytes but 8589934592 bytes is expected (needs manual configuration)
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 10737418240 (10 GiB) but is now 7516192768 (7 GiB)
/dev/sdb had size 8589934592 (8 GiB) but is now 5368709120 (5 GiB)
Could not automap /dev/sda (no disk with same size 10737418240 found)
Could not automap /dev/sdb (no disk with same size 8589934592 found)
Original disk /dev/sda does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDA needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 281
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) Do not map /dev/sda
4) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sda'
Using /dev/sda (chosen by user) for recreating /dev/sda
Original disk /dev/sdb does not exist (with same size) in the target system
Using /dev/sdb (the only available of the disks) for recreating /dev/sdb
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdb => /dev/sdb
jsmeix commented at 2021-12-14 13:17:¶
For completeness how it looks on my RAID1 VMs from above:
On the original system I have a USB disk that I do not use
because I use OUTPUT=ISO
so on the replacement system I have a CDROM drive.
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk linux_raid_member any:raid1sdab 12G
`-/dev/md127 /dev/md127 /dev/sda raid1 12G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11.9G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11.9G /
/dev/sdb /dev/sdb ata disk 5G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part 8M
|-/dev/sdb2 /dev/sdb2 /dev/sdb part vfat REAR-EFI 512M
|-/dev/sdb3 /dev/sdb3 /dev/sdb part ext2 MY-BOOT 1G
`-/dev/sdb4 /dev/sdb4 /dev/sdb part ext3 MY-DATA 3.5G
/dev/sdc /dev/sdc ata disk linux_raid_member any:raid1sdab 12G
`-/dev/md127 /dev/md127 /dev/sdc raid1 12G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 11.9G
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 11.9G /
/dev/sdd /dev/sdd ata disk 1G
`-/dev/sdd1 /dev/sdd1 /dev/sdd part swap 1023M [SWAP]
# grep -v '^#' etc/rear/local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://192.168.122.1/nfs
REQUIRED_PROGS+=( snapper chattr )
PROGS+=( lsattr )
COPY_AS_IS+=( /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
BACKUP_PROG_INCLUDE=( /boot/grub2/x86_64-efi /home /boot/grub2/i386-pc /root /srv /opt /tmp /usr/local /var )
POST_RECOVERY_SCRIPT=( 'if snapper --no-dbus -r $TARGET_FS_ROOT get-config | grep -q "^QGROUP.*[0-9]/[0-9]" ; then snapper --no-dbus -r $TARGET_FS_ROOT set-config QGROUP= ; snapper --no-dbus -r $TARGET_FS_ROOT setup-quota && echo snapper setup-quota done || echo snapper setup-quota failed ; else echo snapper setup-quota not used ; fi' )
SSH_ROOT_PASSWORD='rear'
USE_DHCLIENT="yes"
FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )
PROGRESS_MODE="plain"
PROGRESS_WAIT_SECONDS="3"
LUKS_CRYPTSETUP_OPTIONS+=" --force-password"
GRUB2_INSTALL_DEVICES="/dev/sda /dev/sdc"
DISKS_TO_BE_WIPED="/dev/sd[a-z]"
WRITE_PROTECTED_IDS+=( fe057b67-8e9f-4b21-a67e-289a83fb3993 )
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=20
# usr/sbin/rear -D mkbackup
...
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Automatically excluding disk /dev/sdb (not used by any mounted filesystem)
Marking component '/dev/sdb' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb1 is a child of component /dev/sdb
Marking component '/dev/sdb1' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb2 is a child of component /dev/sdb
Marking component '/dev/sdb2' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb3 is a child of component /dev/sdb
Marking component '/dev/sdb3' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Dependant component /dev/sdb4 is a child of component /dev/sdb
Marking component '/dev/sdb4' as done in /root/rear.github.master.issue2696/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdb' in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdb' in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Verifying that the entries in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master.issue2696/var/lib/rear/layout/disklayout.conf)
...
# grep -v '^#' var/lib/rear/layout/disklayout.conf
disk /dev/sda 12884901888 gpt
disk /dev/sdc 12884901888 gpt
disk /dev/sdd 1073741824 gpt
part /dev/sdd 1072676352 1048576 rear-noname swap /dev/sdd1
raid /dev/md127 level=raid1 raid-devices=2 devices=/dev/sda,/dev/sdc name=raid1sdab metadata=1.0 uuid=8d05eb84:2de831d1:dfed54b2:ad592118
part /dev/md127 10485760 1048576 rear-noname bios_grub /dev/md127p1
part /dev/md127 12739067392 11534336 rear-noname none /dev/md127p2
fs /dev/mapper/cr_root / btrfs uuid=85406026-0559-4b0d-8f67-ec19d3b556f5 label= options=rw,relatime,space_cache,subvolid=256,subvol=/@
...
swap /dev/sdd1 uuid=9c606f48-92cd-4f98-be22-0f8a75358bed label=
crypt /dev/mapper/cr_root /dev/md127p2 type=luks1 cipher=aes-xts-plain64 key_size=512 hash=sha256 uuid=d0446c00-9e79-4872-abaa-2d464fd71c99
RESCUE localhost:~ # rear -D recover
...
Comparing disks
Ambiguous disk layout needs manual configuration (more than one disk with same size used in '/var/lib/rear/layout/disklayout.conf')
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 12884901888 (12 GiB) but is now 10737418240 (10 GiB)
/dev/sdc had size 12884901888 (12 GiB) but is now 2147483648 (2 GiB)
/dev/sdd had size 1073741824 (1 GiB) but it does no longer exist
/dev/sdb had size 5368709120 (5 GiB) but is now 11811160064 (11 GiB)
Could not automap /dev/sda (no disk with same size 12884901888 found)
Could not automap /dev/sdc (no disk with same size 12884901888 found)
Could not automap /dev/sdd (no disk with same size 1073741824 found)
Original disk /dev/sda does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDA needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 285
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) /dev/sdc
4) Do not map /dev/sda
5) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sda'
Using /dev/sda (chosen by user) for recreating /dev/sda
Original disk /dev/sdc does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDC needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 285
Choose an appropriate replacement for /dev/sdc
1) /dev/sdb
2) /dev/sdc
3) Do not map /dev/sdc
4) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result '/dev/sdb'
Using /dev/sdb (chosen by user) for recreating /dev/sdc
Original disk /dev/sdd does not exist (with same size) in the target system
Using /dev/sdc (the only available of the disks) for recreating /dev/sdd
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdc => /dev/sdb
/dev/sdd => /dev/sdc
and after rebbot of the recreated system
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk linux_raid_memb localhost:raid1sdab 10G 8d05eb84-2de8-31d1-dfed-54b2ad592118
`-/dev/md127 /dev/md127 /dev/sda raid1 10G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 10G d0446c00-9e79-4872-abaa-2d464fd71c99
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 10G / 85406026-0559-4b0d-8f67-ec19d3b556f5
/dev/sdb /dev/sdb ata disk linux_raid_memb localhost:raid1sdab 11G 8d05eb84-2de8-31d1-dfed-54b2ad592118
`-/dev/md127 /dev/md127 /dev/sdb raid1 10G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part 10M
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 10G d0446c00-9e79-4872-abaa-2d464fd71c99
`-/dev/mapper/cr_root /dev/dm-0 /dev/md127p2 crypt btrfs 10G / 85406026-0559-4b0d-8f67-ec19d3b556f5
/dev/sdc /dev/sdc ata disk 2G
`-/dev/sdc1 /dev/sdc1 /dev/sdc part swap 1023M [SWAP] 9c606f48-92cd-4f98-be22-0f8a75358bed
/dev/sr0 /dev/sr0 ata rom iso9660 REAR-ISO 70.8M 2021-12-14-13-54-00-22
jsmeix commented at 2021-12-17 13:55:¶
The code here only works for automatically resizing
on RAID1 devices that consist only of whole disks.
It does not work for RAID1 devices that consist of partitions.
Code to make it (hopefully) work for RAID1 devices that consist of
partitions
is provided in
https://github.com/rear/rear/pull/2730/files
which is like the code for RAID0 that consist of whole disks and
partitions
but nothing wast tested with RAID1 up to now.
jsmeix commented at 2021-12-21 14:06:¶
A test with the changes in
https://github.com/rear/rear/pull/2730
for a RAID1 device that consists of a partition /dev/sda3
and a whole disk /dev/sdb.
Original VM:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 6G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
|-/dev/sda2 /dev/sda2 /dev/sda part swap 1G [SWAP]
`-/dev/sda3 /dev/sda3 /dev/sda part linux_raid_member 4G
`-/dev/md127 /dev/md127 /dev/sda3 raid1 4G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 3.5G /
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 400M
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 398M /home
/dev/sdb /dev/sdb ata disk linux_raid_member 5G
`-/dev/md127 /dev/md127 /dev/sdb raid1 4G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 3.5G /
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 400M
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 398M /home
/dev/sr0 /dev/sr0 ata rom iso9660 11.4G
# grep -v '^#' etc/rear/local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://192.168.122.1/nfs
SSH_ROOT_PASSWORD='rear'
USE_DHCLIENT="yes"
FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )
PROGRESS_MODE="plain"
PROGRESS_WAIT_SECONDS="5"
LUKS_CRYPTSETUP_OPTIONS+=" --force-password"
DISKS_TO_BE_WIPED="/dev/sd[a-z]"
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=50
AUTORESIZE_EXCLUDE_PARTITIONS=()
# usr/sbin/rear -D mkbackup
...
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master.pull2730raid1/var/lib/rear/layout/disklayout.conf
Disabling excluded components in /root/rear.github.master.pull2730raid1/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Verifying that the entries in /root/rear.github.master.pull2730raid1/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master.pull2730raid1/var/lib/rear/layout/disklayout.conf)
...
# grep -v '^#' var/lib/rear/layout/disklayout.conf
disk /dev/sda 6442450944 gpt
part /dev/sda 8388608 1048576 rear-noname bios_grub /dev/sda1
part /dev/sda 1073741824 9437184 rear-noname swap /dev/sda2
part /dev/sda 4294967296 1083179008 rear-noname raid,legacy_boot /dev/sda3
disk /dev/sdb 5368709120 gpt
raid /dev/md127 level=raid1 raid-devices=2 devices=/dev/sda3,/dev/sdb name=raidsda3sdb metadata=1.0 uuid=a82e1697:4be879aa:6f2071d4:1a58219b
raiddisk /dev/md127 4294901760 gpt
part /dev/md127 3758096384 1048576 rear-noname none /dev/md127p1
part /dev/md127 419430400 3759144960 rear-noname none /dev/md127p2
fs /dev/mapper/cr_home /home ext4 uuid=ffe7a283-6007-4a30-b1ac-aadd65d1f7cc label= blocksize=1024 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=4091 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
fs /dev/md127p1 / ext4 uuid=73553933-21f1-4aeb-848a-1dfe09516f39 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
swap /dev/sda2 uuid=3214aa3f-5ceb-4207-8f3d-08bbaee3014f label=
crypt /dev/mapper/cr_home /dev/md127p2 type=luks1 cipher=aes-xts-plain64 key_size=512 hash=sha256 uuid=6154a400-970e-4d09-8da5-8cb079fa5f20
Replacement VM:
RESCUE localhost:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 5G
/dev/sdb /dev/sdb ata disk 3.8G
/dev/sr0 /dev/sr0 ata rom iso9660 69.6M
RESCUE localhost:~ # rear -D recover
...
Comparing disks
Device sda has size 5368709120 bytes but 6442450944 bytes is expected (needs manual configuration)
Device sdb has size 4080219136 bytes but 5368709120 bytes is expected (needs manual configuration)
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 6442450944 (6 GiB) but is now 5368709120 (5 GiB)
/dev/sdb had size 5368709120 (5 GiB) but is now 4080219136 (3 GiB)
Could not automap /dev/sda (no disk with same size 6442450944 found)
Using /dev/sda (same size 5368709120) for recreating /dev/sdb
Original disk /dev/sda does not exist (with same size) in the target system
Using /dev/sdb (the only available of the disks) for recreating /dev/sda
Current disk mapping table (source => target):
/dev/sdb => /dev/sda
/dev/sda => /dev/sdb
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 322
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) n/a
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
3
...
/dev/sda /dev/sda
/dev/sdb /dev/sdb
"rear.github.master.pull2730raid1/var/lib/rear/layout/disk_mappings" 2 lines, 36 characters written
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdb => /dev/sdb
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 322
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Confirm identical disk mapping and proceed without manual configuration
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/config/df.txt
Applied disk layout mappings to /etc/rear/rescue.conf
Trying to automatically resize last partition when disk size changed
Examining gpt device /dev/sda to automatically resize its last active partition
New /dev/sda is 1073741824 bytes smaller than old device
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Checking /dev/sda3 if it is the last partition on /dev/sda
Found 'rear-noname' partition /dev/sda3 as last partition on /dev/sda
Determining if last partition /dev/sda3 is resizeable
Determining new size for last partition /dev/sda3
Determining if last partition /dev/sda3 actually needs to be increased or shrinked
Last partition /dev/sda3 must be shrinked by 10485760 bytes to still fit on device
Shrinking last partition /dev/sda3 to end of device (new device at most 50% smaller)
Changed last partition /dev/sda3 size from 4294967296 to 4284481536 bytes
Examining gpt device /dev/md127 to automatically resize its last active partition
New /dev/md127 is 1288489984 bytes smaller than old device
Checking /dev/md127p1 if it is the last partition on /dev/md127
Checking /dev/md127p2 if it is the last partition on /dev/md127
Found 'rear-noname' partition /dev/md127p2 as last partition on /dev/md127
Determining if last partition /dev/md127p2 is resizeable
Determining new size for last partition /dev/md127p2
Determining if last partition /dev/md127p2 actually needs to be increased or shrinked
Last partition /dev/md127p2 must be shrinked by 98566144 bytes to still fit on device
Shrinking last partition /dev/md127p2 to end of device (new device at most 50% smaller)
Changed last partition /dev/md127p2 size from 419430400 to 320864256 bytes
UserInput -I LAYOUT_FILE_CONFIRMATION needed in /usr/share/rear/layout/prepare/default/500_confirm_layout_file.sh line 26
Confirm or edit the disk layout file
1) Confirm disk layout and continue 'rear recover'
2) Edit disk layout (/var/lib/rear/layout/disklayout.conf)
3) View disk layout (/var/lib/rear/layout/disklayout.conf)
4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk layout and continue 'rear recover''
User confirmed disk layout file
...
User confirmed disk recreation script
Determining disks to be wiped ...
Cannot check write protection by ID for /dev/sda (no ID found)
Cannot check write protection by ID for /dev/sdb (no ID found)
UserInput -I WIPE_DISKS_CONFIRMATION needed in /usr/share/rear/layout/recreate/default/120_confirm_wipedisk_disks.sh line 157
Disks to be wiped: /dev/sda /dev/sdb
1) Confirm disks to be completely overwritten and continue 'rear recover'
2) Use Relax-and-Recover shell and return back to here
3) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disks to be completely overwritten and continue 'rear recover''
User confirmed disks to be wiped
Wiping child devices of /dev/sda in reverse ordering: /dev/sda
Wiped first 16777216 bytes of /dev/sda
Wiped last 16777216 bytes of /dev/sda
Wiping child devices of /dev/sdb in reverse ordering: /dev/sdb
Wiped first 16777216 bytes of /dev/sdb
Wiped last 16777216 bytes of /dev/sdb
Start system layout restoration.
Disk '/dev/sda': creating 'gpt' partition table
Disk '/dev/sda': creating partition number 1 with name ''sda1''
Disk '/dev/sda': creating partition number 2 with name ''sda2''
Disk '/dev/sda': creating partition number 3 with name ''sda3''
Creating software RAID /dev/md127
Disk '/dev/md127': creating 'gpt' partition table
Disk '/dev/md127': creating partition number 1 with name ''md127p1''
Disk '/dev/md127': creating partition number 2 with name ''md127p2''
Creating filesystem of type ext4 with mount point / on /dev/md127p1.
Mounting filesystem /
Creating swap on /dev/sda2
Creating LUKS volume cr_home on /dev/md127p2
Set the password for LUKS volume cr_home (for 'cryptsetup luksFormat' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Enter the password for LUKS volume cr_home (for 'cryptsetup luksOpen' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Creating filesystem of type ext4 with mount point /home on /dev/mapper/cr_home.
Mounting filesystem /home
Disk layout created.
...
User confirmed restored files
Running mkinitrd...
Recreated initrd (/sbin/mkinitrd).
Installing GRUB2 boot loader...
Determining where to install GRUB2 (no GRUB2_INSTALL_DEVICES specified)
Found possible boot disk /dev/sda - installing GRUB2 there
Found possible boot disk /dev/sdb
/dev/sda - installing GRUB2 there
Failed to install GRUB2 on possible boot disk /dev/sdb
/dev/sda
Running 'wrapup' stage ======================
Finished 'recover'. The target system is mounted at '/mnt/local'.
RESCUE localhost:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 5G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
|-/dev/sda2 /dev/sda2 /dev/sda part swap 1G
`-/dev/sda3 /dev/sda3 /dev/sda part linux_raid_member 4G
`-/dev/md127 /dev/md127 /dev/sda3 raid1 3.8G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 3.5G /mnt/local
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 306M
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 304M /mnt/local/home
/dev/sdb /dev/sdb ata disk linux_raid_member 3.8G
`-/dev/md127 /dev/md127 /dev/sdb raid1 3.8G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 3.5G /mnt/local
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 306M
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 304M /mnt/local/home
/dev/sr0 /dev/sr0 ata rom iso9660 69.6M
Regardless of the "Failed to install GRUB2 on possible boot disk
/dev/sdb..." message
the recreated system boots well (from /dev/sda) and seems to work OK
so automated resizing or RAID1 that consist of whole disks and
partitions
should now also work when
https://github.com/rear/rear/pull/2730
is merged.
Note what the automated disk mapping did (excerpt from above)
Comparing disks
Device sda has size 5368709120 bytes but 6442450944 bytes is expected (needs manual configuration)
Device sdb has size 4080219136 bytes but 5368709120 bytes is expected (needs manual configuration)
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 6442450944 (6 GiB) but is now 5368709120 (5 GiB)
/dev/sdb had size 5368709120 (5 GiB) but is now 4080219136 (3 GiB)
Could not automap /dev/sda (no disk with same size 6442450944 found)
Using /dev/sda (same size 5368709120) for recreating /dev/sdb
Original disk /dev/sda does not exist (with same size) in the target system
Using /dev/sdb (the only available of the disks) for recreating /dev/sda
Current disk mapping table (source => target):
/dev/sdb => /dev/sda
/dev/sda => /dev/sdb
Because sda changed from 6 GiB to 5 GiB
and sdb changed from 5 GiB to 4 GiB
the new sda size is same as the old sdb size
so the new sda was automatically chosen as sdb replacement
and then only the new sdb was left as replacement for the old sda
so that automated disk mapping results that
the old 5 GiB sdb is replaced by the new 5 GiB sda and
the old 6 GiB sda is replaced by the new 4 GiB sdb
which is not what I wanted so I selected
"3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)"
and specified my intended mapping that
the old 6 GiB sda is replaced by the new 5 GiB sda and
the old 5 GiB sda is replaced by the new 4 GiB sdb.
"rear recover" fails when I accept the automated disk mapping that
the old 5 GiB sdb is replaced by the new 5 GiB sda and
the old 6 GiB sda is replaced by the new 4 GiB sdb:
RESCUE localhost:~ # rear -D recover
...
Comparing disks
Device sda has size 5368709120 bytes but 6442450944 bytes is expected (needs manual configuration)
Device sdb has size 4080219136 bytes but 5368709120 bytes is expected (needs manual configuration)
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 6442450944 (6 GiB) but is now 5368709120 (5 GiB)
/dev/sdb had size 5368709120 (5 GiB) but is now 4080219136 (3 GiB)
Could not automap /dev/sda (no disk with same size 6442450944 found)
Using /dev/sda (same size 5368709120) for recreating /dev/sdb
Original disk /dev/sda does not exist (with same size) in the target system
Using /dev/sdb (the only available of the disks) for recreating /dev/sda
Current disk mapping table (source => target):
/dev/sdb => /dev/sda
/dev/sda => /dev/sdb
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 322
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) n/a
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/config/df.txt
Applied disk layout mappings to /etc/rear/rescue.conf
Trying to automatically resize last partition when disk size changed
Examining gpt device /dev/sdb to automatically resize its last active partition
New /dev/sdb is 2362231808 bytes smaller than old device
Checking /dev/sdb1 if it is the last partition on /dev/sdb
Checking /dev/sdb2 if it is the last partition on /dev/sdb
Checking /dev/sdb3 if it is the last partition on /dev/sdb
Found 'rear-noname' partition /dev/sdb3 as last partition on /dev/sdb
Determining if last partition /dev/sdb3 is resizeable
Determining new size for last partition /dev/sdb3
Determining if last partition /dev/sdb3 actually needs to be increased or shrinked
Last partition /dev/sdb3 must be shrinked by 1298137088 bytes to still fit on device
Shrinking last partition /dev/sdb3 to end of device (new device at most 50% smaller)
Changed last partition /dev/sdb3 size from 4294967296 to 2996830208 bytes
Examining gpt device /dev/md127 to automatically resize its last active partition
New /dev/md127 is 1288489984 bytes smaller than old device
Checking /dev/md127p1 if it is the last partition on /dev/md127
Checking /dev/md127p2 if it is the last partition on /dev/md127
Found 'rear-noname' partition /dev/md127p2 as last partition on /dev/md127
Determining if last partition /dev/md127p2 is resizeable
Determining new size for last partition /dev/md127p2
Determining if last partition /dev/md127p2 actually needs to be increased or shrinked
Last partition /dev/md127p2 must be shrinked by 98566144 bytes to still fit on device
Shrinking last partition /dev/md127p2 to end of device (new device at most 50% smaller)
Changed last partition /dev/md127p2 size from 419430400 to 320864256 bytes
UserInput -I LAYOUT_FILE_CONFIRMATION needed in /usr/share/rear/layout/prepare/default/500_confirm_layout_file.sh line 26
Confirm or edit the disk layout file
1) Confirm disk layout and continue 'rear recover'
2) Edit disk layout (/var/lib/rear/layout/disklayout.conf)
3) View disk layout (/var/lib/rear/layout/disklayout.conf)
4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk layout and continue 'rear recover''
User confirmed disk layout file
Marking component '/dev/sdb' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sdb1' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sdb2' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sdb3' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/sda' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/md127' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/md127p1' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/md127p2' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'fs:/' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'swap:/dev/sdb2' as done in /var/lib/rear/layout/disktodo.conf
Marking component '/dev/mapper/cr_home' as done in /var/lib/rear/layout/disktodo.conf
Marking component 'fs:/home' as done in /var/lib/rear/layout/disktodo.conf
Running 'layout/recreate' stage ======================
UserInput -I LAYOUT_CODE_CONFIRMATION needed in /usr/share/rear/layout/recreate/default/100_confirm_layout_code.sh line 26
Confirm or edit the disk recreation script
1) Confirm disk recreation script and continue 'rear recover'
2) Edit disk recreation script (/var/lib/rear/layout/diskrestore.sh)
3) View disk recreation script (/var/lib/rear/layout/diskrestore.sh)
4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk recreation script and continue 'rear recover''
User confirmed disk recreation script
Determining disks to be wiped ...
UserInput -I WIPE_DISKS_CONFIRMATION needed in /usr/share/rear/layout/recreate/default/120_confirm_wipedisk_disks.sh line 157
Disks to be wiped: /dev/sda /dev/sdb
1) Confirm disks to be completely overwritten and continue 'rear recover'
2) Use Relax-and-Recover shell and return back to here
3) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disks to be completely overwritten and continue 'rear recover''
User confirmed disks to be wiped
Wiping child devices of /dev/sda in reverse ordering: /dev/sda3 /dev/sda2 /dev/sda1 /dev/sda
Wiped first 16777216 bytes of /dev/sda3
Wiped last 16777216 bytes of /dev/sda3
Wiped first 16777216 bytes of /dev/sda2
Wiped last 16777216 bytes of /dev/sda2
Wiped first 8388608 bytes of /dev/sda1
Skip wiping at the end of /dev/sda1 (dvice size 8388608 not greater than the bytes that were wiped)
Wiped first 16777216 bytes of /dev/sda
Wiped last 16777216 bytes of /dev/sda
Wiping child devices of /dev/sdb in reverse ordering: /dev/sdb2 /dev/sdb1 /dev/sdb
Wiped first 16777216 bytes of /dev/sdb2
Wiped last 16777216 bytes of /dev/sdb2
Wiped first 16777216 bytes of /dev/sdb1
Wiped last 16777216 bytes of /dev/sdb1
Wiped first 16777216 bytes of /dev/sdb
Wiped last 16777216 bytes of /dev/sdb
Start system layout restoration.
Disk '/dev/sdb': creating 'gpt' partition table
Disk '/dev/sdb': creating partition number 1 with name ''sdb1''
Disk '/dev/sdb': creating partition number 2 with name ''sdb2''
Disk '/dev/sdb': creating partition number 3 with name ''sdb3''
Creating software RAID /dev/md127
Disk '/dev/md127': creating 'gpt' partition table
Disk '/dev/md127': creating partition number 1 with name ''md127p1''
UserInput -I LAYOUT_CODE_RUN needed in /usr/share/rear/layout/recreate/default/200_run_layout_code.sh line 127
The disk layout recreation script failed
1) Rerun disk recreation script (/var/lib/rear/layout/diskrestore.sh)
2) View 'rear recover' log file (/var/log/rear/rear-localhost.log)
3) Edit disk recreation script (/var/lib/rear/layout/diskrestore.sh)
4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
6
UserInput: Valid choice number result 'Abort 'rear recover''
ERROR: User chose to abort 'rear recover' in /usr/share/rear/layout/recreate/default/200_run_layout_code.sh
Some latest log messages since the last called script 200_run_layout_code.sh:
2021-12-22 12:58:02.059379910 4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
2021-12-22 12:58:02.065484158 5) Use Relax-and-Recover shell and return back to here
2021-12-22 12:58:02.070543219 6) Abort 'rear recover'
2021-12-22 12:58:02.076606002 (default '1' timeout 300 seconds)
2021-12-22 12:58:10.928296948 UserInput: 'read' got as user input '6'
2021-12-22 12:58:10.936017614 UserInput: Valid choice number result 'Abort 'rear recover''
2021-12-22 12:58:10.942074559 Error detected during restore.
2021-12-22 12:58:10.947084880 Restoring saved original /var/lib/rear/layout/disklayout.conf
Aborting due to an error, check /var/log/rear/rear-localhost.log for details
...
RESCUE localhost:~ # less /var/log/rear/rear-localhost.log
...
+++ Print 'Disk '\''/dev/md127'\'': creating partition number 1 with name '\'''\''md127p1'\'''\'''
+++ [[ ! -n 3759144959 ]]
+++ parted -s /dev/md127 mkpart ''\''md127p1'\''' 1048576B 3759144959B
Error: The location 3759144959B is outside of the device /dev/md127.
RESCUE localhost:~ # parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 4080219136B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1048576B 9437183B 8388608B sdb1 bios_grub
2 9437184B 1083179007B 1073741824B sdb2 swap
3 1083179008B 4080009215B 2996830208B sdb3 raid, legacy_boot
RESCUE localhost:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 5G
`-/dev/md127 /dev/md127 /dev/sda raid1 2.8G
/dev/sdb /dev/sdb ata disk 3.8G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part 8M
|-/dev/sdb2 /dev/sdb2 /dev/sdb part 1G
`-/dev/sdb3 /dev/sdb3 /dev/sdb part linux_raid_member 2.8G
`-/dev/md127 /dev/md127 /dev/sdb3 raid1 2.8G
/dev/sr0 /dev/sr0 ata rom iso9660 69.6M
RESCUE localhost:~ # lsblk -bipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 5368709120
`-/dev/md127 /dev/md127 /dev/sda raid1 2996764672
/dev/sdb /dev/sdb ata disk 4080219136
|-/dev/sdb1 /dev/sdb1 /dev/sdb part 8388608
|-/dev/sdb2 /dev/sdb2 /dev/sdb part 1073741824
`-/dev/sdb3 /dev/sdb3 /dev/sdb part linux_raid_member 2996830208
`-/dev/md127 /dev/md127 /dev/sdb3 raid1 2996764672
/dev/sr0 /dev/sr0 ata rom iso9660 72943616
So something is still not right in the autoresizing code in
layout/prepare/default/420_autoresize_last_partitions.sh
because it should not happen that diskrestore.sh contains
impossible commands that let diskrestore.sh fail.
Instead the autoresizing code in 420_autoresize_last_partitions.sh
should error out when resizing is impossible.
jsmeix commented at 2021-12-22 12:37:¶
As far as I see currently
the autoresizing code in 420_autoresize_last_partitions.sh
fails in this extreme (impossible) case where before the original RAID1
was
`-/dev/md127 /dev/md127 /dev/sdb raid1 4G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 3.5G /
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 400M
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 398M /home
which should be automatically shrinked to
`-/dev/md127 /dev/md127 /dev/sdb3 raid1 2.8G
so it must be shrinked by 1.2 GiB
but the original last partition /dev/md127p2 is only 400 MiB big
so it is impossible to shrink this last partition by 1.2 GiB.
But the "rear -D recover" output contains
Last partition /dev/md127p2 must be shrinked by 98566144 bytes to still fit on device
...
Changed last partition /dev/md127p2 size from 419430400 to 320864256 bytes
which is shrinking by 94 MiB from 400 MiB to 306 MiB.
This is wrong because actually it woud need to be shrinked by 1.2 GiB
because the "rear -D recover" output contains
New /dev/md127 is 1288489984 bytes smaller than old device
which is 1.1999998 GiB smaller so this value looks right.
[Export of Github issue for rear/rear.]