#1747 Issue `closed`: rear restore to smaller disk aborts silently¶

Labels: bug, fixed / solved / done

Lukey3332 opened issue at 2018-03-06 18:59:¶

Hello Everyone,
I have an small Problem related to the latest Changes in the Disk Layout Code.

rear version (/usr/sbin/rear -V):
Relax-and-Recover 2.3 / Git
Brief description of the issue:
If I restore to an smaller Disk, rear recover aborts after the disk-layout Promt without any Error message:

Relax-and-Recover 2.3 / Git
Using log file: /var/log/rear/rear-luklap.log
Running workflow recover within the ReaR rescue/recovery system
Will do driver migration (recreating initramfs/initrd)
Comparing disks
Device sda has size 250059350016 but 256060514304 is expected (needs manual configuration)
Switching to manual disk layout configuration
Original disk /dev/sda does not exist (with same size) in the target system
Using /dev/sda (the only appropriate) for recreating /dev/sda
Current disk mapping table (source -> target):
    /dev/sda /dev/sda
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default '1' timeout 300 seconds)
1
User confirmed disk mapping
Exiting rear recover (PID 993) and its descendant processes

Work-around, if any:
Adding the following to the Config file makes the restore work

AUTORESIZE_PARTITIONS=true

I think the user should be warned or asked in this case instead of silently aborting.

jsmeix commented at 2018-03-07 07:53:¶

@Lukey3332
I need debug logging information to be able to work on it,
i.e. the ReaR log file in the recovery system after "rear -D recover",
see "Debugging issues with Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery

jsmeix commented at 2018-03-07 07:59:¶

I guess this "sudden death" is a bash error exit in
usr/share/rear/layout/prepare/default/420_autoresize_last_partitions.sh
because of

set -e -u -o pipefail

therein.

@Lukey3332
try out if it works when you comment the set -e -u -o pipefail line in
usr/share/rear/layout/prepare/default/420_autoresize_last_partitions.sh

I noticed such a "sudden death" once while implementing
usr/share/rear/layout/prepare/default/420_autoresize_last_partitions.sh
regardless that the intended behaviour should be an error message
according to my code in
usr/share/rear/lib/_input-output-functions.sh

# Verbose exit in case of errors which is in particular needed when 'set -e' is active because
# otherwise a 'set -e' error exit would happen silently which could look as if all was o.k.
# cf. https://github.com/rear/rear/issues/700#issuecomment-327755633
# The separated EXIT_FAIL_MESSAGE variable is used to denote a failure exit.
# One cannot use EXIT_CODE for that because there are cases where a non-zero exit code
# is the intended outcome (e.g. in the 'checklayout' workflow, cf. usr/sbin/rear):
QuietAddExitTask "(( EXIT_FAIL_MESSAGE )) && echo '${MESSAGE_PREFIX}$PROGRAM $WORKFLOW failed, check $RUNTIME_LOGFILE for details' 1>&8"

which had worked at the time when I had implemented that
but somehow it seems this does no longer work.

jsmeix commented at 2018-03-07 10:28:¶

https://github.com/rear/rear/pull/1748
fixes only that no longer any error message was shown
when ReaR did a "sudden death" bash error exit
when "set -e -u -o pipefail" was set in a script.

jsmeix commented at 2018-03-07 10:29:¶

@Lukey3332
I still need your "rear -D recover" debug logging information to be able to find out
what the actual root cause was why ReaR did a "sudden death" bash error exit
from usr/share/rear/layout/prepare/default/420_autoresize_last_partitions.sh
in your particular case - for a possible reasoning behind have a look at
https://github.com/rear/rear/issues/700#issuecomment-297954640

Lukey3332 commented at 2018-03-07 18:58:¶

2018-03-06 18:47:52.174328379 Confirm or edit the disk mapping
2018-03-06 18:47:52.176800411 1) Confirm disk mapping and continue 'rear recover'
2018-03-06 18:47:52.179231377 2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
2018-03-06 18:47:52.181650918 3) Use Relax-and-Recover shell and return back to here
2018-03-06 18:47:52.184030975 4) Abort 'rear recover'
2018-03-06 18:47:52.186380358 (default '1' timeout 300 seconds)
2018-03-06 18:48:03.889806392 UserInput: 'read' got as user input '1'
2018-03-06 18:48:03.896916288 User confirmed disk mapping
2018-03-06 18:48:03.907456999 Including layout/prepare/default/310_remove_exclusions.sh
2018-03-06 18:48:03.918008309 Including layout/prepare/default/320_apply_mappings.sh
2018-03-06 18:48:03.942803692 Including layout/prepare/default/400_autoresize_disks.sh
/usr/share/rear/layout/prepare/default/400_autoresize_disks.sh: line 10: backup_file: command not found
2018-03-06 18:48:03.960985855 Total resize of -6001164288B
2018-03-06 18:48:03.963323201 Searching for resizeable partitions on disk /dev/sda (250059350016B)
2018-03-06 18:48:03.973949625 Will not resize partition /dev/sda1.
2018-03-06 18:48:03.982307494 Will resize partition /dev/sda2.
2018-03-06 18:48:03.993812196 Resized partition /dev/sda2 from 255804309504B to 249804546048B.
2018-03-06 18:48:04.001879035 Including layout/prepare/default/420_autoresize_last_partitions.sh
250059350016
255852544
2018-03-06 18:48:04.032209019 Exiting rear recover (PID 993) and its descendant processes
/usr/share/rear/lib/_input-output-functions.sh: line 100: pstree: command not found
2018-03-06 18:48:05.037861561 
/usr/share/rear/lib/_input-output-functions.sh: line 104: COLUMNS: unbound variable

This is from my last try, I will post an Log with Debug-Mode later.

Lukey3332 commented at 2018-03-07 19:52:¶

I can't upload files here :/
https://paste.ubuntu.com/p/PXxBrsBvgG/

Lukey3332 commented at 2018-03-07 20:03:¶

Ok, with set -e -u -o pipefail Commented out it Errors out with:

User confirmed disk mapping
Last partition /dev/sda2 must be shrinked by 1236992 bytes to still fit on disk
ERROR: Last partition /dev/sda2 cannot be shrinked (new disk more than 2% smaller)
Aborting due to an error, check /var/log/rear/rear-luklap.log for details
Exiting rear recover (PID 3720) and its descendant processes
Running exit tasks

I'm using LVM on an luks-Encrypted Partition (sda2) btw.

jsmeix commented at 2018-03-08 08:49:¶

The

/usr/share/rear/lib/_input-output-functions.sh: line 104: COLUMNS: unbound variable

issue is https://github.com/rear/rear/pull/1748 and fixed thereby.

The

ERROR: Last partition /dev/sda2 cannot be shrinked (new disk more than 2% smaller)

works as intended, see in default.conf the section about
Resizing partitions in MIGRATION_MODE during "rear recover"
therein in particular the AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE
config variable that reads in particular:

# When the new disk is smaller than AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE percent it errors out.
...
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=2

In your case the new sda has a size of 250059350016 bytes
but the old sda was 256060514304 bytes
and 2% of 256060514304 bytes is 5121210286 bytes
but 256060514304 - 250059350016 = 6001164288
which is bigger than 5121210286 so that it errors out.

If you want automated shrinking in your case you could set
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=3
but read in default.conf the whole section about
Resizing partitions in MIGRATION_MODE during "rear recover"
in particular because you use higher-level strorage objects
on top of the plain partitions but automated shrinking
by ReaR only shrinks plain partitions but not any higher-level
strorage objects like LVM stuff on top of the plain partitions:

# When AUTORESIZE_PARTITIONS is true, all active partitions on all active disks
# get resized by the 430_autoresize_all_partitions.sh script
# (except boot and swap partitions via some special hardcoded rules in that script)
# if the disk size had changed (i.e. only in migration mode).
# This does not resize volumes on top of the affected partitions.
...
# When the first value in AUTORESIZE_PARTITIONS is neither true nor false
# only the last active partition on each active disk gets resized
# by the 420_autoresize_last_partitions.sh script.
...
# In particular this does not resize volumes on top of the affected partitions.
# To migrate volumes on disk where the disk size had changed the user must in advance
# manually adapt his disklayout.conf file before he runs "rear recover".

When you use AUTORESIZE_PARTITIONS=true
you will be affected by https://github.com/rear/rear/issues/1731
and the related issue https://github.com/rear/rear/issues/102

jsmeix commented at 2018-03-08 09:05:¶

Your https://paste.ubuntu.com/p/PXxBrsBvgG/ contains (excerpts):

2018-03-07 19:39:38.704224394 Examining /dev/sda to automatically resize its last active partition
...
2018-03-07 19:39:38.724577861 Checking /dev/sda1 if it is the last partition on /dev/sda
...
2018-03-07 19:39:38.732599450 Checking /dev/sda2 if it is the last partition on /dev/sda
...
2018-03-07 19:39:38.742238645 Found 'primary' partition /dev/sda2 as last partition on /dev/sda
...
2018-03-07 19:39:38.747556350 Determining if last partition /dev/sda2 is resizeable
...
++ last_part_is_resizeable=
++ IsInArray /dev/sda2
++ local needle=/dev/sda2
++ test -z /dev/sda2
++ shift
++ test /dev/sda2 == ''
++ shift
++ return 1
++ last_part_filesystem_entry=($( grep "^fs $last_part_dev " "$LAYOUT_FILE" ))
+++ grep '^fs /dev/sda2 ' /var/lib/rear/layout/disklayout.conf
+ DoExitTasks

so it seems you have a last partition /dev/sda2
but there is no fs /dev/sda2 entry for it in your disklayout.conf
which causes this code in 420_autoresize_last_partitions.sh

        # Example filesystem 'fs' entry in disklayout.conf (excerpt):
        #  # Format: fs    ...
        #  fs /dev/sda3 /boot/efi vfat ...
        last_part_filesystem_entry=( $( grep "^fs $last_part_dev " "$LAYOUT_FILE" ) )

to fail (the grep therein results non-zero exit code).

jsmeix commented at 2018-03-08 10:03:¶

Via
https://github.com/rear/rear/commit/32eafc491f793e5b8a510ff8f4219ff9be2a7edf
I removed "set -e -u -o pipefail" from 420_autoresize_last_partitions.sh
because currently there are too many places in its code that work right
but are not implemented to work even with "set -e -u -o pipefail"
cf. https://github.com/rear/rear/issues/700#issuecomment-297944196
and the subsequent comments therein.

I assume with that commit this issue here should be fixed.

For me 420_autoresize_last_partitions.sh without "set -e -u -o pipefail"
works with a last partition that does not have a matching fs entry
in disklayout.conf.

@Lukey3332
many thanks for testing our newest GitHub master code and for your issue report.
It helped me a lot to make 420_autoresize_last_partitions.sh working as intended.

jsmeix commented at 2018-03-08 10:40:¶

@Lukey3332
regardless that I already closed this issue please provide feedback
whether or not the latest GitHub master code also works for you
with appropriate settings for your particular case like
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=3

Lukey3332 commented at 2018-03-09 10:26:¶

Yep, works as Expected now. Thanks for your Support!

jsmeix commented at 2018-03-09 11:41:¶

You are welcome.
Thanks for your prompt feedback!

[Export of Github issue for rear/rear.]

#1747 Issue closed: rear restore to smaller disk aborts silently¶