#2730 PR merged: Automatically resize active last partition on RAID0¶
Labels: enhancement, fixed / solved / done
jsmeix opened issue at 2021-12-17 13:29:¶
-
Type: Enhancement
-
Impact: High
-
Reference to related issue (URL):
https://github.com/rear/rear/issues/2696
and follow-up of https://github.com/rear/rear/pull/2726 -
How was this pull request tested?
Works for me with my not-so-simple RAID0 setup,
see my results in the comments below.
jsmeix commented at 2021-12-17 13:48:¶
My test:
Original VM:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 10G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
|-/dev/sda2 /dev/sda2 /dev/sda part ext2 1016M /boot
`-/dev/sda3 /dev/sda3 /dev/sda part linux_raid_member any:raid0sda3b2c 9G
`-/dev/md127 /dev/md127 /dev/sda3 raid0 22G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 11G /
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 9G
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 9G /home
/dev/sdb /dev/sdb ata disk 8G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part swap 1G [SWAP]
`-/dev/sdb2 /dev/sdb2 /dev/sdb part linux_raid_member any:raid0sda3b2c 7G
`-/dev/md127 /dev/md127 /dev/sdb2 raid0 22G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 11G /
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 9G
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 9G /home
/dev/sdc /dev/sdc ata disk linux_raid_member any:raid0sda3b2c 6G
`-/dev/md127 /dev/md127 /dev/sdc raid0 22G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 11G /
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 9G
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 9G /home
/dev/sr0 /dev/sr0 ata rom iso9660 SLE-15-SP3-Full-x86_64187.11.001 11.4G
# grep -v '^#' etc/rear/local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://192.168.122.1/nfs
SSH_ROOT_PASSWORD='rear'
USE_DHCLIENT="yes"
FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )
PROGRESS_MODE="plain"
PROGRESS_WAIT_SECONDS="5"
LUKS_CRYPTSETUP_OPTIONS+=" --force-password"
DISKS_TO_BE_WIPED="/dev/sd[a-z]"
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=50
# usr/sbin/rear -D mkbackup
...
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling excluded components in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Verifying that the entries in /root/rear.github.master/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master/var/lib/rear/layout/disklayout.conf)
...
# grep -v '^#' var/lib/rear/layout/disklayout.conf
disk /dev/sda 10737418240 gpt
part /dev/sda 8388608 1048576 rear-noname bios_grub /dev/sda1
part /dev/sda 1065353216 9437184 rear-noname legacy_boot /dev/sda2
part /dev/sda 9652142080 1074790400 rear-noname raid /dev/sda3
disk /dev/sdb 8589934592 gpt
part /dev/sdb 1073741824 1048576 rear-noname swap /dev/sdb1
part /dev/sdb 7504658432 1074790400 rear-noname raid /dev/sdb2
disk /dev/sdc 6442450944 unknown
raid /dev/md127 level=raid0 raid-devices=3 devices=/dev/sda3,/dev/sdb2,/dev/sdc name=raid0sda3b2c metadata=1.0 uuid=31ed7f77:5625b09a:3c64193e:763ef947 layout=original chunk=64
raiddisk /dev/md127 23599054848 gpt
part /dev/md127 11810635776 1572864 rear-noname none /dev/md127p1
part /dev/md127 9663676416 11812208640 rear-noname none /dev/md127p2
fs /dev/mapper/cr_home /home ext4 uuid=14c2516d-33eb-40ad-a50b-45c48f8b24da label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16380 default_mount_options=user_xattr,acl options=rw,relatime,stripe=48,data=ordered
fs /dev/md127p1 / ext4 uuid=9699406d-66d0-436f-921a-99dbd9b33a96 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16383 default_mount_options=user_xattr,acl options=rw,relatime,stripe=48
fs /dev/sda2 /boot ext2 uuid=70aefc4b-38eb-47c3-a476-66b9753c67c2 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
swap /dev/sdb1 uuid=ce08cc50-9544-45bc-bbed-68d4eb36dca0 label=
crypt /dev/mapper/cr_home /dev/md127p2 type=luks1 cipher=aes-xts-plain64 key_size=512 hash=sha256 uuid=415da632-e2f0-4894-b8f7-a9f7d717c5f2
Replacement VM:
RESCUE localhost:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 9G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
|-/dev/sda2 /dev/sda2 /dev/sda part ext2 1016M
`-/dev/sda3 /dev/sda3 /dev/sda part linux_raid_member localhost:raid0sda3b2c 8G
/dev/sdb /dev/sdb ata disk 7G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part swap 1G
`-/dev/sdb2 /dev/sdb2 /dev/sdb part linux_raid_member localhost:raid0sda3b2c 6G
/dev/sdc /dev/sdc ata disk linux_raid_member localhost:raid0sda3b2c 5G
/dev/sr0 /dev/sr0 ata rom iso9660 REAR-ISO 69.6M
RESCUE localhost:~ # rear -D recover
...
Comparing disks
Device sda has size 9663676416 bytes but 10737418240 bytes is expected (needs manual configuration)
Device sdb has size 7516192768 bytes but 8589934592 bytes is expected (needs manual configuration)
Device sdc has size 5368709120 bytes but 6442450944 bytes is expected (needs manual configuration)
Switching to manual disk layout configuration (GiB sizes rounded down to integer)
/dev/sda had size 10737418240 (10 GiB) but is now 9663676416 (9 GiB)
/dev/sdb had size 8589934592 (8 GiB) but is now 7516192768 (7 GiB)
/dev/sdc had size 6442450944 (6 GiB) but is now 5368709120 (5 GiB)
Could not automap /dev/sda (no disk with same size 10737418240 found)
Could not automap /dev/sdb (no disk with same size 8589934592 found)
Could not automap /dev/sdc (no disk with same size 6442450944 found)
Original disk /dev/sda does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDA needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 285
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) /dev/sdc
4) Do not map /dev/sda
5) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result '/dev/sda'
Using /dev/sda (chosen by user) for recreating /dev/sda
Original disk /dev/sdb does not exist (with same size) in the target system
UserInput -I LAYOUT_MIGRATION_REPLACEMENT_SDB needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 285
Choose an appropriate replacement for /dev/sdb
1) /dev/sdb
2) /dev/sdc
3) Do not map /dev/sdb
4) Use Relax-and-Recover shell and return back to here
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result '/dev/sdb'
Using /dev/sdb (chosen by user) for recreating /dev/sdb
Original disk /dev/sdc does not exist (with same size) in the target system
Using /dev/sdc (the only available of the disks) for recreating /dev/sdc
Current disk mapping table (source => target):
/dev/sda => /dev/sda
/dev/sdb => /dev/sdb
/dev/sdc => /dev/sdc
UserInput -I LAYOUT_MIGRATION_CONFIRM_MAPPINGS needed in /usr/share/rear/layout/prepare/default/300_map_disks.sh line 322
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Confirm identical disk mapping and proceed without manual configuration
3) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
4) Use Relax-and-Recover shell and return back to here
5) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk mapping and continue 'rear recover''
User confirmed disk mapping
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/disklayout.conf
Applied disk layout mappings to /var/lib/rear/layout/config/df.txt
Applied disk layout mappings to /etc/rear/rescue.conf
Trying to automatically resize last partition when disk size changed
Examining gpt device /dev/sda to automatically resize its last active partition
New /dev/sda is 1073741824 bytes smaller than old device
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Checking /dev/sda3 if it is the last partition on /dev/sda
Found 'rear-noname' partition /dev/sda3 as last partition on /dev/sda
Determining if last partition /dev/sda3 is resizeable
Determining new size for last partition /dev/sda3
Determining if last partition /dev/sda3 actually needs to be increased or shrinked
Last partition /dev/sda3 must be shrinked by 1064304640 bytes to still fit on device
Shrinking last partition /dev/sda3 to end of device (new device at most 50% smaller)
Changed last partition /dev/sda3 size from 9652142080 to 8587837440 bytes
Examining gpt device /dev/sdb to automatically resize its last active partition
New /dev/sdb is 1073741824 bytes smaller than old device
Checking /dev/sdb1 if it is the last partition on /dev/sdb
Checking /dev/sdb2 if it is the last partition on /dev/sdb
Found 'rear-noname' partition /dev/sdb2 as last partition on /dev/sdb
Determining if last partition /dev/sdb2 is resizeable
Determining new size for last partition /dev/sdb2
Determining if last partition /dev/sdb2 actually needs to be increased or shrinked
Last partition /dev/sdb2 must be shrinked by 1064304640 bytes to still fit on device
Shrinking last partition /dev/sdb2 to end of device (new device at most 50% smaller)
Changed last partition /dev/sdb2 size from 7504658432 to 6440353792 bytes
Examining gpt device /dev/md127 to automatically resize its last active partition
New /dev/md127 is 3221225472 bytes smaller than old device
Checking /dev/md127p1 if it is the last partition on /dev/md127
Checking /dev/md127p2 if it is the last partition on /dev/md127
Found 'rear-noname' partition /dev/md127p2 as last partition on /dev/md127
Determining if last partition /dev/md127p2 is resizeable
Determining new size for last partition /dev/md127p2
Determining if last partition /dev/md127p2 actually needs to be increased or shrinked
Last partition /dev/md127p2 must be shrinked by 1098907648 bytes to still fit on device
Shrinking last partition /dev/md127p2 to end of device (new device at most 50% smaller)
Changed last partition /dev/md127p2 size from 9663676416 to 8564768768 bytes
UserInput -I LAYOUT_FILE_CONFIRMATION needed in /usr/share/rear/layout/prepare/default/500_confirm_layout_file.sh line 26
Confirm or edit the disk layout file
1) Confirm disk layout and continue 'rear recover'
2) Edit disk layout (/var/lib/rear/layout/disklayout.conf)
3) View disk layout (/var/lib/rear/layout/disklayout.conf)
4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk layout and continue 'rear recover''
User confirmed disk layout file
...
UserInput -I LAYOUT_CODE_CONFIRMATION needed in /usr/share/rear/layout/recreate/default/100_confirm_layout_code.sh line 26
Confirm or edit the disk recreation script
1) Confirm disk recreation script and continue 'rear recover'
2) Edit disk recreation script (/var/lib/rear/layout/diskrestore.sh)
3) View disk recreation script (/var/lib/rear/layout/diskrestore.sh)
4) View original disk space usage (/var/lib/rear/layout/config/df.txt)
5) Use Relax-and-Recover shell and return back to here
6) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disk recreation script and continue 'rear recover''
User confirmed disk recreation script
Determining disks to be wiped ...
UserInput -I WIPE_DISKS_CONFIRMATION needed in /usr/share/rear/layout/recreate/default/120_confirm_wipedisk_disks.sh line 157
Disks to be wiped: /dev/sda /dev/sdb /dev/sdc
1) Confirm disks to be completely overwritten and continue 'rear recover'
2) Use Relax-and-Recover shell and return back to here
3) Abort 'rear recover'
(default '1' timeout 300 seconds)
UserInput: No real user input (empty or only spaces) - using default input
UserInput: Valid choice number result 'Confirm disks to be completely overwritten and continue 'rear recover''
User confirmed disks to be wiped
Wiping child devices of /dev/sda in reverse ordering: /dev/sda3 /dev/sda2 /dev/sda1 /dev/sda
Wiped first 16777216 bytes of /dev/sda3
Wiped last 16777216 bytes of /dev/sda3
Wiped first 16777216 bytes of /dev/sda2
Wiped last 16777216 bytes of /dev/sda2
Wiped first 8388608 bytes of /dev/sda1
Skip wiping at the end of /dev/sda1 (dvice size 8388608 not greater than the bytes that were wiped)
Wiped first 16777216 bytes of /dev/sda
Wiped last 16777216 bytes of /dev/sda
Wiping child devices of /dev/sdb in reverse ordering: /dev/sdb2 /dev/sdb1 /dev/sdb
Wiped first 16777216 bytes of /dev/sdb2
Wiped last 16777216 bytes of /dev/sdb2
Wiped first 16777216 bytes of /dev/sdb1
Wiped last 16777216 bytes of /dev/sdb1
Wiped first 16777216 bytes of /dev/sdb
Wiped last 16777216 bytes of /dev/sdb
Wiping child devices of /dev/sdc in reverse ordering: /dev/sdc
Wiped first 16777216 bytes of /dev/sdc
Wiped last 16777216 bytes of /dev/sdc
Start system layout restoration.
Disk '/dev/sda': creating 'gpt' partition table
Disk '/dev/sda': creating partition number 1 with name ''sda1''
Disk '/dev/sda': creating partition number 2 with name ''sda2''
Disk '/dev/sda': creating partition number 3 with name ''sda3''
Disk '/dev/sdb': creating 'gpt' partition table
Disk '/dev/sdb': creating partition number 1 with name ''sdb1''
Disk '/dev/sdb': creating partition number 2 with name ''sdb2''
Creating software RAID /dev/md127
Disk '/dev/md127': creating 'gpt' partition table
Disk '/dev/md127': creating partition number 1 with name ''md127p1''
Disk '/dev/md127': creating partition number 2 with name ''md127p2''
Creating filesystem of type ext4 with mount point / on /dev/md127p1.
Mounting filesystem /
Creating filesystem of type ext2 with mount point /boot on /dev/sda2.
Mounting filesystem /boot
Creating swap on /dev/sdb1
Creating LUKS volume cr_home on /dev/md127p2
Set the password for LUKS volume cr_home (for 'cryptsetup luksFormat' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Enter the password for LUKS volume cr_home (for 'cryptsetup luksOpen' on /dev/md127p2):
Enter passphrase for /dev/md127p2:
Creating filesystem of type ext4 with mount point /home on /dev/mapper/cr_home.
Mounting filesystem /home
Disk layout created.
...
RESCUE localhost:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT
/dev/sda /dev/sda ata disk 9G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
|-/dev/sda2 /dev/sda2 /dev/sda part ext2 1016M /mnt/local/boot
`-/dev/sda3 /dev/sda3 /dev/sda part linux_raid_member localhost:raid0sda3b2c 8G
`-/dev/md127 /dev/md127 /dev/sda3 raid0 19G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 11G /mnt/local
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 8G
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 8G /mnt/local/home
/dev/sdb /dev/sdb ata disk 7G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part swap 1G
`-/dev/sdb2 /dev/sdb2 /dev/sdb part linux_raid_member localhost:raid0sda3b2c 6G
`-/dev/md127 /dev/md127 /dev/sdb2 raid0 19G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 11G /mnt/local
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 8G
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 8G /mnt/local/home
/dev/sdc /dev/sdc ata disk linux_raid_member localhost:raid0sda3b2c 5G
`-/dev/md127 /dev/md127 /dev/sdc raid0 19G
|-/dev/md127p1 /dev/md127p1 /dev/md127 part ext4 11G /mnt/local
`-/dev/md127p2 /dev/md127p2 /dev/md127 part crypto_LUKS 8G
`-/dev/mapper/cr_home /dev/dm-0 /dev/md127p2 crypt ext4 8G /mnt/local/home
/dev/sr0 /dev/sr0 ata rom iso9660 REAR-ISO 69.6M
jsmeix commented at 2021-12-21 14:11:¶
With the changes in this pull request here it should now also work
to automatically resize RAID1 that consist of whole disks and
partitions, see
https://github.com/rear/rear/pull/2726#issuecomment-998807463
jsmeix commented at 2021-12-21 14:12:¶
@rear/contributors
I would like to merge it tomorrow afternoon
unless there are objections.
jsmeix commented at 2021-12-22 12:48:¶
Regardless of an issue with RAID1 when resizing is impossible
https://github.com/rear/rear/pull/2726#issuecomment-999546292
I will merge this one (for RAID0) now.
[Export of Github issue for rear/rear.]