#1795 Issue closed: Restoring to a bigger HD fails, due to over increasing the 2nd. last (data-)partition, when the following one is a non-data-partion¶
Labels: support / question, fixed / solved / done
chymian opened issue at 2018-05-03 16:15:¶
- ReaR version:
Relax-and-Recover 2.3 / Gitfrom debian buster 2.3+dfsg-1 - OS version: debian stretch
- System architecture (x86 compatible): amd64 bare metal
- Are you using BIOS or UEFI: BIOS
- Brief description of the issue: if the disk to restore to is bigger
than the original,
diskrestore.shfails due to overexpanding the last data-partition, which is followed by a non-data partition. It then has no space left. - Work-around, if any: manual edit
diskrestore.sh
There a two points here:
-
ReaR grows – without consent – the partitions to max. of disk (100%) so that the last part. – in this case
bios_grubwith a size of only 860K – does not have any space left.
There seems to be an bug in the size-calculation/does not handle the case, when a non-data partition is the last one. -
May I suggest to set the default behavior to create the partitions in the same size, if the disk is bigger than the original.
For autoresizing: it could ask whether it should increase all/some partitions by an percental/absolut value. Not every data-partiton needs an increase: i.e. efi/root/boot
I know, there is the possibility to manual edit the diskrestore.sh and
work with Bytes-values, which is very uncomfortable.
The point here is, that a simple restore (which can be done by any user), as it is with restoring to a same-size disk is not possible and therefore needs an admin onsite.
Also it would help a lot if an ssh-server (with the same keys) would be started on the recovery-image and even an openvpn connection (and/or the whole Network-Manager Config w/ plugins) if existent on the original host – to make offsite-restore/help possible.
For debugging purposes it would also help if there are more consoles (gettys) availlable.
Original Disklayout:
parted /dev/sdb p free
Model: ATA OCZ-VECTOR150 (scsi)
Disk /dev/sdb: 480GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
17,4kB 1049kB 1031kB Free Space
1 1049kB 98,6MB 97,5MB fat32 efi boot, esp
2 98,6MB 50,0GB 49,9GB btrfs root
3 50,0GB 66,0GB 16,0GB linux-swap(v1) swap
4 66,0GB 480GB 414GB lvm1 lvm
5 480GB 480GB 860kB bios_grub bios_grub
Created Disklayout:
Model: ATA Samsung SSD 860 (scsi)
Disk /dev/sdb: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
17.4kB 2097kB 2080kB Free Space
1 2097kB 209MB 207MB efi boot, esp
209MB 209MB 1024B Free Space
2 209MB 106GB 106GB root
106GB 106GB 1536B Free Space
3 106GB 122GB 16.0GB swap
122GB 122GB 4096B Free Space
4 122GB 1000GB 878GB lvm1 lvm
the same layout with units set to Bytes
Model: ATA Samsung SSD 860 (scsi)
Disk /dev/sdb: 1000204886016B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
17408B 2097151B 2079744B Free Space
1 2097152B 208899071B 206801920B efi boot, esp
208899072B 208900095B 1024B Free Space
2 208900096B 106033576447B 105824676352B root
106033576448B 106033577983B 1536B Free Space
3 106033577984B 122033799167B 16000221184B swap
122033799168B 122033803263B 4096B Free Space
4 122033803264B 1000204869119B 878171065856B lvm1 lvm
rear-hansa.log
...
+++ create_component /dev/sdb disk
+++ local device=/dev/sdb
+++ local type=disk
+++ local touchfile=disk--dev-sdb
+++ '[' -e /tmp/rear.uDRcanrrPOzMLnF/tmp/touch/disk--dev-sdb ']'
+++ return 0
+++ Log 'Stop mdadm'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2018-05-03 15:52:48.374346194 '
+++ test 1 -gt 0
+++ echo '2018-05-03 15:52:48.374346194 Stop mdadm'
2018-05-03 15:52:48.374346194 Stop mdadm
+++ grep -q md /proc/mdstat
+++ Log 'Erasing MBR of disk /dev/sdb'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2018-05-03 15:52:48.378571303 '
+++ test 1 -gt 0
+++ echo '2018-05-03 15:52:48.378571303 Erasing MBR of disk /dev/sdb'
2018-05-03 15:52:48.378571303 Erasing MBR of disk /dev/sdb
+++ dd if=/dev/zero of=/dev/sdb bs=512 count=1
1+0 records in
1+0 records out
512 bytes copied, 0.000307959 s, 1.7 MB/s
+++ sync
+++ LogPrint 'Creating partitions for disk /dev/sdb (gpt)'
+++ Log 'Creating partitions for disk /dev/sdb (gpt)'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2018-05-03 15:52:48.383776293 '
+++ test 1 -gt 0
+++ echo '2018-05-03 15:52:48.383776293 Creating partitions for disk /dev/sdb (gpt)'
2018-05-03 15:52:48.383776293 Creating partitions for disk /dev/sdb (gpt)
+++ Print 'Creating partitions for disk /dev/sdb (gpt)'
+++ test 1
+++ echo -e 'Creating partitions for disk /dev/sdb (gpt)'
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb mklabel gpt
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb mkpart efi 2097152B 208898892B
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb set 1 boot on
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb set 1 esp on
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb name 1 efi
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb mkpart root 208900096B 106033576083B
Warning: The resulting partition is not properly aligned for best performance.
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb name 2 root
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb mkpart swap 106033577984B 122033799167B
Warning: The resulting partition is not properly aligned for best performance.
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb name 3 swap
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb mkpart lvm1 122033803264B 1000204869119B
Warning: The resulting partition is not properly aligned for best performance.
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb set 4 lvm on
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb name 4 lvm1
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in $@
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/sdb mkpart bios_grub 1000205168640B 1000204869119B
Error: The location 1000205168640B is outside of the device /dev/sdb.
diskrestore.sh
#!/bin/bash
LogPrint "Start system layout restoration."
mkdir -p /mnt/local
if create_component "vgchange" "rear" ; then
lvm vgchange -a n >/dev/null
component_created "vgchange" "rear"
fi
set -e
set -x
if create_component "/dev/sdb" "disk" ; then
# Create /dev/sdb (disk)
Log "Stop mdadm"
if grep -q md /proc/mdstat &>/dev/null; then
mdadm --stop -s >&2 || echo "stop mdadm failed"
# Prevent udev waking up mdadm later.
# Reasoning: At least on RHEL6 when parted created a raid partition on disk,
# udev (via /lib/udev/rules.d/65-md-incremental.rules) wakes up mdadm which locks the disk,
# so further parted commands with the disk will fail since the disk is busy now.
# The /lib/udev/rules.d/65-md-incremental.rules detects anaconda (the Red Hat installer),
# and if it find itself running under anaconda, it will not run.
# Accordingly also for other installers (in particular the ReaR installer)
# this rule should not be there (and other Linux distros probably do not have it)
# which means removing it is the right solution to make ReaR work also for RHEL6:
if [ -e /lib/udev/rules.d/65-md-incremental.rules ] ; then
rm -f /lib/udev/rules.d/65-md-incremental.rules || echo "rm 65-md-incremental.rules failed"
fi
fi
Log "Erasing MBR of disk /dev/sdb"
dd if=/dev/zero of=/dev/sdb bs=512 count=1
sync
LogPrint "Creating partitions for disk /dev/sdb (gpt)"
my_udevsettle
parted -s /dev/sdb mklabel gpt >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb mkpart 'efi' 2097152B 208898892B >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb set 1 boot on >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb set 1 esp on >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb name 1 'efi' >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb mkpart 'root' 208900096B 106033576083B >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb name 2 'root' >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb mkpart 'swap' 106033577984B 122033799167B >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb name 3 'swap' >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb mkpart 'lvm1' 122033803264B 1000204869119B >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb set 4 lvm on >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb name 4 'lvm1' >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb mkpart 'bios_grub' 1000205168640B 1000204869119B >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb set 5 bios_grub on >&2
my_udevsettle
my_udevsettle
parted -s /dev/sdb name 5 'bios_grub' >&2
my_udevsettle
sleep 1
...
jsmeix commented at 2018-05-04 07:45:¶
As far as I see on first glance this issue proves that what is called
Relax-and-Recover 2.3 / Git from debian buster 2.3+dfsg-1
is not a recent ReaR upstream GitHub master code
(but probably the 2.3 release plus whatever GitHub stuff),
cf.
https://github.com/rear/rear/issues/1793#issuecomment-386309765
Revovery on a bigger HD means you are in MIGRATION_MODE and
when in MIGRATION_MODE any non-last partition (e.g. the 2nd. last
partition)
gets changed if you do not use the current ReaR upstream GitHub master
code.
The current ReaR upstream GitHub master code changes
by default - if at all - at most the last partition.
Without reading all your detailed information (many thanks for that!)
I assume you are hit by one of the issues that are described at
https://github.com/rear/rear/issues/1731
and
https://github.com/rear/rear/pull/1733
or by a related issue like
https://github.com/rear/rear/issues/102
I recommend to try out our current ReaR upstream GitHub master code
because that is the only place where we at ReaR upstream fix bugs.
Bugs in released ReaR versions are not fixed by us (by ReaR upstream).
Bugs in released ReaR versions that got fixed in current ReaR upstream
GitHub master code might be fixed (if the fix can be backported with
reasonable effort) by the Linux distributor wherefrom you got ReaR.
To use our current ReaR upstream GitHub master code
do the following:
Basically "git clone" it into a separated directory and then
configure and run ReaR from within that directory like:
# git clone https://github.com/rear/rear.git # mv rear rear.github.master # cd rear.github.master # vi etc/rear/local.conf # usr/sbin/rear -D mkbackup
Note the relative paths "etc/rear/" and "usr/sbin/".
If the issue also happens with current ReaR upstream GitHub master
code
please provide us a complete ReaR debug log file of "rear -D recover"
plus your initial/original disklayout.conf file from your original
system
so that we can have a look how it behaves in your particular
environment
cf. "Debugging issues with Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery
If it works with current ReaR upstream GitHub master code
we would really appreciate an explicit positive feedback.
jsmeix commented at 2018-05-04 08:52:¶
@chymian
I think my current code in
usr/share/rear/layout/prepare/default/420_autoresize_last_partitions.sh
also handles the case when the last partition is not a mounted
partition
on the original system while "rear mkrescue" runs because
I assume also non-mounted partitions are stored during "rear mkrescue"
in disklayout.conf as active (i.e. non-commented) part ... entries
and
my current code in 420_autoresize_last_partitions.sh works on
what is stored in disklayout.conf as non-commented part ... entries.
But there is the assumption that the last partition
is the one with biggest partition start(bytes) value
and not the last entry in the partition table on the disk.
E.g. if you have (with MBR partitioning table)
/dev/sda2 from 300MiB to 400MiB and
/dev/sda4 from 100MiB to 200MiB
then /dev/sda2 is "the last partition" (and not /dev/sda4).
jsmeix commented at 2018-05-04 09:12:¶
@chymian
regarding
ssh-server (with the same keys) would be started in the recovery system
please read the description about the SSH_* config variables
SSH_FILES
SSH_ROOT_PASSWORD
SSH_UNPROTECTED_PRIVATE_KEYS
in default.conf (the new SSH_* config variables exist since the ReaR
2.3 release).
jsmeix commented at 2018-07-19 12:20:¶
Because there are no further comments I assume
it works sufficiently with our ReaR upstream GitHub master code
so that this isssue was sufficiently answered and handled
so that I can close it hereby.
[Export of Github issue for rear/rear.]