#1733 PR `merged`: Replace 400_autoresize_disks by 420_autoresize_last_partitions and 430_autoresize_all_partitions¶

Labels: enhancement, bug, cleanup, fixed / solved / done

jsmeix opened issue at 2018-02-16 13:34:¶

Type: Bug Fix Enhancement
Impact: High
Reference to related issue (URL):
https://github.com/rear/rear/issues/1731
How was this pull request tested?
Currently this is in a "submit early" state
where subsequent "submit often" commits will follow
that is currently mainly intended for others to see what I have up to now
which is currently only a little bit tested by me
Brief description of the changes in this pull request:

Renamed 400_autoresize_disks.sh into 430_autoresize_all_partitions.sh
to keep full backward compatible old weird behaviour but now that
needs to be explicity wanted by the user via the new config variable
AUTORESIZE_PARTITIONS=true
and
implemented new default 420_autoresize_last_partitions.sh
that only resizes the last partition on each disk (if at all).

Longer description of the changes in this pull request:

The new default behaviour is to only resize the last partition on each disk
which means to only change the end vaule of the last partitions.

This way ReaR should behave now fail save.

I think ReaR is meant to recreate a system as much as possible
exactly as it was before and not to do a complete repartitioning.

I talked to colleagues in general about resizing partitions during disaster recovery
and got consensus that the general expectation is that by default the recreated
system should be "the same" regardless if a replacement disk is bigger.
In particular nobody wants changed partitions when the replacement disk
is only a bit bigger (e.g. some small disk model specific differences).

The new automated resizing implements a "minimal changes" approach
in the new 420_autoresize_last_partitions.sh script.

The old 430_autoresize_all_partitions.sh and
the new 420_autoresize_last_partitions.sh
are mutually excluding each other via the AUTORESIZE_PARTITIONS value.

Because only the end value of the last partition may get changed,
the partitioning alignment of the original system is not changed,
cf. https://github.com/rear/rear/issues/102
so that with the new default behaviour "rear recover" can no longer result
a partitioning that is worse than what it was on the original system.

Additionally there are limits via the new config variables
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE and
AUTOINCREASE_DISK_SIZE_THRESHOLD_PERCENTAGE
when the last partition will be changed at all to avoid changes
when the new disk is only a little bit bigger and to avoid that
the backup cannot be restored when the new disk is much smaller.

When the new disk is a bit smaller
(at most AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE percent),
the last (active - i.e. not commented in disklayout.conf) partition
gets shrinked but all other partitions are not changed.
When the new disk is smaller than
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE percent it errors out.
To migrate onto a substantially smaller new disk the user must in advance
manually adapt his disklayout.conf file before he runs "rear recover".

When the new disk is only a bit bigger
(less than AUTOINCREASE_DISK_SIZE_THRESHOLD_PERCENTAGE percent),
no partition gets increased (which leaves the bigger disk space
at the end of the disk unused).
When the new disk is substantially bigger
(at least AUTOINCREASE_DISK_SIZE_THRESHOLD_PERCENTAGE percent),
the last (active) partition gets increased but all other partitions are not changed.
To migrate various partitions onto a substantially bigger new disk the user must
in advance manually adapt his disklayout.conf file before he runs "rear recover".

Furthermore the user has the final power because via the new config variables
AUTORESIZE_EXCLUDE_PARTITIONS and AUTORESIZE_PARTITIONS
the user can specify what partitions should be explicitly excluded or inculed
to be resized.

The new config variables apply only to the new 420_autoresize_last_partitions.sh
because currently the old 430_autoresize_all_partitions.sh is there
to keep full backward compatible old weird behaviour.

Perhaps later - if there is really no use-case for the old weird behaviour - the
430_autoresize_all_partitions.sh could be enhanced to resize more than
only the last partition in a fail-safe way, but currently this is not at all
my scope or my intention of this pull request.

I tried two times to enhance the old 400_autoresize_disks.sh into something
that works both backward compatible and reasonably fail-safe
but I failed two times so that I will no longer try to enhance that script
in any way - of course would I appreciate it if others could do it.

jsmeix commented at 2018-02-16 13:50:¶

I forgot to mention that the new 420_autoresize_last_partitions.sh
provides much broader support to exclude 'boot' 'swap' and now also 'efi'
partitions from being resized via AUTORESIZE_EXCLUDE_PARTITIONS:
The special values 'boot', 'swap', and 'efi' specify that

partitions where its filesystem mountpoint contains 'boot' or 'bios' or 'grub'
or where its GPT name or flags contain 'boot' or 'bios' or 'grub' (anywhere case insensitive)

partitions for which an active 'swap' entry exists in disklayout.conf
or where its GPT name or flags contain 'swap' (anywhere case insensitive)

partitions where its filesystem mountpoint contains 'efi' or 'esp'
or where its GPT name or flags contains 'efi' or 'esp' (anywhere case insensitive)

are excluded from being resized via
AUTORESIZE_EXCLUDE_PARTITIONS=( boot swap efi )

This exludes more often boot and swap partitions
as the old code in 430_autoresize_all_partitions.sh does.

jsmeix commented at 2018-02-17 11:49:¶

It is explicitly intended to also work reasonably well when the disk size shrinked.

Shrinking can only work to some reasonable extent because I think
no automatism can work if the disk is too small so that it has less space
than the backup restore would need.
I think this cannot be checked at partitioning time.
Or do we have a stored value what the uncompressed backup needs?
Don't we have some disk usage values from the original system stored?
If yes I will use those vaules to check if shrinking should be o.k.
I will have a closer look next week...

FYI
What is currently missing is:

a)
Resizing when the last partition is a logical partition
i.e. when the last partition is "inside" a "container partition" (an extended partition).
Then I also need to adjust the end value of the "container partition"
to have all values right in disklayout.conf.

I want to always have all values perfectly right in disklayout.conf because
in migration mode the user is asked to confirm that disklayout.conf is right.
But what should the user do when just before a ReaR script had messed up
his original disklayout.conf with inconsistent values that look plain wrong?
The user must then assume ReaR is broken.

b)
Shrinking when the last partitions ends before the end of disk on the original system
i.e. when there was unused disk space at the end of the disk on the original system.
Then on a smaller replacement disk the last partition may not need to be shrinked.
This case is an important use-case because things should "just work" when the
user had intentionally left some unused space at the end of the original disk
to be prepared to recover without any partitioning changes on a bit smaller
replacement disk (i.e. small disk size differences between different disk models
that are all labeled as e.g. "2 TB" sized disks).

jsmeix commented at 2018-02-19 11:37:¶

FYI:
In case of b) in
https://github.com/rear/rear/pull/1733#issuecomment-366436113
i.e. when the last partitions ends before the end of disk on the original system
it currently blindly proceeds and luckily fails by accident as follows:

On the original system I have now the last partition smaller than to end of disk:

# parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2147483648B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start        End          Size        File system     Name   Flags
 1      8388608B     847249407B   838860800B  ext2            data1
 2      847249408B   1266679807B  419430400B  linux-swap(v1)  swap1
 3      1266679808B  1476395007B  209715200B  ext2            data2
 4      1476395008B  1790967807B  314572800B  linux-swap(v1)  swap2

and with that "rear recover" fails with

Last partition /dev/sdb4 must be shrinked by -248512512 bytes to still fit on disk
ERROR: Last partition /dev/sdb4 cannot be shrinked (new disk more than 3% smaller)

The wrong negative shrink value happens because when the new disk is smaller
it calculates the shrink value from the original end of partition to the end of new disk
which results a wrong negative shrink value if the original end of partition is
less than end of new disk which means there is no need to do any shrinking
but my current insufficient code blindly proceeds and luckily stops at that error.

jsmeix commented at 2018-02-20 13:28:¶

With the above last three commits
case of b) in
https://github.com/rear/rear/pull/1733#issuecomment-366436113
should be solved
plus (via the third of the three commits)
the user gets a warning (one of the rare cases where a "WARNING" is justified)
when the new size of last partition will be smaller than its disk usage was
which needs 510_current_disk_usage.sh to be enhaced so that now
the values in var/lib/rear/layout/config/df.txt can be used to calculate
in 420_autoresize_last_partitions.sh if the current disk usage still fits
on a smaller disk (when the last partition must be shrinked).

On my original system I have now (my sdb is 2 GiB):

# df -Pl -BM -x encfs -x tmpfs -x devtmpfs
Filesystem     1048576-blocks  Used Available Capacity Mounted on
/dev/sda2              18684M 2699M    15036M      16% /
/dev/sdb4                591M  551M       10M      99% /data3

# ls -lh /data3/
total 551M
drwx------ 2 root root  16K Feb 20 13:22 lost+found
-rw-r--r-- 1 root root 550M Feb 20 13:29 udandom550MiBs.data

How it looks during "rear recover" with a new sdb of only 1.9 GiB (excerpts):

# rear -D recover
...
WARNING: New size of last partition /dev/sdb4 will be smaller than its disk usage was
Last partition /dev/sdb4 must be shrinked by 66060288 bytes to still fit on disk
Shrinking last partition /dev/sdb4 to end of disk (new disk at most 5% smaller)
Changed last partition /dev/sdb4 size from 629145600 to 563085312 bytes
...
Restoring data3/udandom550MiBs.data 
...
Restored 2608 MiB [avg. 19353 KiB/sec] 
OK
Backup restore program 'tar' failed with exit code '2'. Check '/var/log/rear/rear-d57.log' and the restored system.
...

# grep ^tar: /var/log/rear/rear-d57.log
tar: data3/udandom550MiBs.data: Wrote only 3584 of 10240 bytes
tar: Exiting with failure status due to previous errors

# df -Pl -BM -x encfs -x tmpfs -x devtmpfs
Filesystem     1048576-blocks  Used Available Capacity Mounted on
/dev/sda2              18684M 2473M    15452M      14% /mnt/local
/dev/sdb4                529M  529M        0M     100% /mnt/local/data3

# ls -lh /mnt/local/data3
total 529M
drwx------ 2 root root  16K Feb 20 13:22 lost+found
-rw-r--r-- 1 root root 528M Feb 20 13:29 udandom550MiBs.data

('udandom' should have been 'urandom' but that doesn't matter)

jsmeix commented at 2018-02-21 12:56:¶

I will now work on case a) in
https://github.com/rear/rear/pull/1733#issuecomment-366436113

Now I have this sdb:

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2048MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system     Flags
 1      8.00MiB  808MiB   800MiB   primary   ext2            type=83
 2      808MiB   1208MiB  400MiB   primary   linux-swap(v1)  type=82
 3      1208MiB  1708MiB  500MiB   extended                  lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical   ext2            type=83
 6      1400MiB  1600MiB  200MiB   logical   ext2            type=83

Note the gaps around the second logical partition /dev/sdb6 and
the gap from the end of the extended partition to the end of disk:

1208MiB start extended
1209MiB start logical1
1300MiB end logical1
gap
1400MiB start logical2
1600MiB end logical2
gap
1708MiB end extended
gap
2048MiB end disk

What must work is to shrink the disk to those values

1708MiB (or more):
no changes of the partitioning

less than 1708MiB but not less than 1600MiB:
end extended is adjusted to new end of disk
but logical2 is not changed

less than 1600MiB but still more than 1400MiB:
end extended and end logical2 are adjusted to new end of disk
plus test if new logical2 size is smaller than logical2 disk usage

1400MiB (or less):
error out with some "no space for logical2 on disk" message
regardless that start logical2 could be moved down to end logical1
but such things will not be implemented now (perhaps later, perhaps never).

jsmeix commented at 2018-02-22 13:14:¶

While working on it I try out various stuff in the recovery system and
I got confused how original config files (in particular disklayout.conf)
were saved and restored and improved that with my last commit here.

jsmeix commented at 2018-02-22 13:29:¶

There is a severe bug in current ReaR master code
at least regarding the size of an extended partition:

With this sdb
( same as in https://github.com/rear/rear/pull/1733#issuecomment-367317079 )

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2048MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system     Flags
 1      8.00MiB  808MiB   800MiB   primary   ext2            type=83
 2      808MiB   1208MiB  400MiB   primary   linux-swap(v1)  type=82
 3      1208MiB  1708MiB  500MiB   extended                  lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical   ext2            type=83
 6      1400MiB  1600MiB  200MiB   logical   ext2            type=83

# parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2147483648B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start        End          Size        Type      File system     Flags
 1      8388608B     847249407B   838860800B  primary   ext2            type=83
 2      847249408B   1266679807B  419430400B  primary   linux-swap(v1)  type=82
 3      1266679808B  1790967807B  524288000B  extended                  lba, type=0f
 5      1267728384B  1363148799B  95420416B   logical   ext2            type=83
 6      1468006400B  1677721599B  209715200B  logical   ext2            type=83

I get after "rear mkrescue" this
var/lib/rear/layout/disklayout.conf entries for sdb

# Disk /dev/sdb
# Format: disk <devname> <size(bytes)> <partition label type>
disk /dev/sdb 2147483648 msdos
# Partitions on /dev/sdb
# Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
part /dev/sdb 838860800 8388608 primary none /dev/sdb1
part /dev/sdb 419430400 847249408 primary none /dev/sdb2
part /dev/sdb 1024 1266679808 extended lba /dev/sdb3
part /dev/sdb 95420416 1267728384 logical none /dev/sdb5
part /dev/sdb 209715200 1468006400 logical none /dev/sdb6

where the size of the extended partition is only 1024 bytes
in contrast to its actual size shown by parted of 524288000 bytes = 500 MiB.

And that is what I get with current ReaR master code after "rear recover"

RESCUE d57:~ # rear -D recover
...
User confirmed disk layout file
Partition primary on /dev/sda: size reduced to fit on disk.
Partition logical on /dev/sdb: size reduced to fit on disk.
...
User confirmed disk recreation script
Start system layout restoration.
Creating partitions for disk /dev/sda (msdos)
Creating partitions for disk /dev/sdb (msdos)
Creating filesystem of type ext3 with mount point / on /dev/sda2.
Mounting filesystem /
Creating filesystem of type ext2 with mount point /data.logical2 on /dev/sdb6.
Mounting filesystem /data.logical2
Creating swap on /dev/sda1
Disk layout created.
Restoring from '/tmp/rear.EShvaipVvXXIREv/outputfs/d57/backup.tar.gz'...
...
Restoring finished.
...
Finished recovering your system. You can explore it under '/mnt/local'.
Exiting rear recover (PID 842) and its descendant processes
Running exit tasks

RESCUE d57:~ # parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1843MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size    Type      File system  Flagsfinal commandment 
 1      2.00MiB  991MiB   989MiB  primary                type=83
 2      991MiB   1485MiB  494MiB  primary                type=83
 3      1485MiB  1843MiB  358MiB  extended               lba, type=0f
 5      1485MiB  1598MiB  112MiB  logical                type=83
 6      1598MiB  1843MiB  245MiB  logical   ext2         type=83

RESCUE d57:~ # parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1932735488B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start        End          Size         Type      File system  Flags
 1      2097152B     1039111167B  1037014016B  primary                type=83
 2      1039114240B  1557621247B  518507008B   primary                type=83
 3      1557622784B  1932735487B  375112704B   extended               lba, type=0f
 5      1557626880B  1675587583B  117960704B   logical                type=83
 6      1675591680B  1932735487B  257143808B   logical   ext2         type=83

What a mess: I got a very different partitioning.

That insane

Partition primary on /dev/sda: size reduced to fit on disk.
Partition logical on /dev/sdb: size reduced to fit on disk.

comes from layout/prepare/GNU/Linux/100_include_partition_code.sh
which arbitrarily changes my disklayout.conf after

User confirmed disk layout file

so that with current ReaR master code "rear recover" clearly works
blindly against what the user has confirmed that should be set up
in migration mode.

WHAT A MESS!

In migration mode what the user has confirmed that should be set up
is sacrosanct: It is the user's final commandment that must be obeyed.

If the user's confirmed disklayout.conf partitioning cannot be set up,
100_include_partition_code.sh has to error out (at least in migration mode)
so that the user can adapt disklayout.conf but 100_include_partition_code.sh
must not arbitrarily change the user's confirmed partitioning to whatever
that script "thinks" is best.

jsmeix commented at 2018-02-23 14:45:¶

And now what I get with my current code of this pull request
after "rear recover" (with same original system)

RESCUE d57:~ # rear -D recover
...
User confirmed disk mapping
Last partition /dev/sda2 cannot be resized (new disk less than 1% bigger)
Last partition /dev/sdb6 will not be resized (original last partition still fits on the new smaller disk)
...
User confirmed disk recreation script
Start system layout restoration.
...
Finished recovering your system. You can explore it under '/mnt/local'.
Exiting rear recover (PID 850) and its descendant processes
Running exit tasks

RESCUE d57:~ # parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1843MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system  Flags
 1      2.00MiB  802MiB   800MiB   primary                type=83
 2      802MiB   1202MiB  400MiB   primary                type=83
 3      1202MiB  1843MiB  641MiB   extended               lba, type=0f
 5      1202MiB  1293MiB  91.0MiB  logical                type=83
 6      1293MiB  1493MiB  200MiB   logical   ext2         type=83

RESCUE d57:~ # parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1932735488B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start        End          Size        Type      File system  Flags
 1      2097152B     840957951B   838860800B  primary                type=83
 2      840962048B   1260392447B  419430400B  primary                type=83
 3      1260396544B  1932735487B  672338944B  extended               lba, type=0f
 5      1260400640B  1355821055B  95420416B   logical                type=83
 6      1355825152B  1565540351B  209715200B  logical   ext2         type=83

The MiB values look o.k. but the byte values reveal bad alignment after "rear recover"

RESCUE d57:~ # echo "bytes = MiB" ; for start in 2097152 840962048 1260396544 1260400640 1355825152 ; do echo -n "$start = " ; echo "$start / 1024 /1024" | bc -l ; done

bytes = MiB
2097152 = 2.00000000000000000000
840962048 = 802.00390625000000000000
1260396544 = 1202.00781250000000000000
1260400640 = 1202.01171875000000000000
1355825152 = 1293.01562500000000000000

in contrast to the good alignemt on the original system:

# echo "bytes = MiB" ; for start in 8388608 847249408 1266679808 1267728384 1468006400 ; do echo -n "$start = " ; echo "$start / 1024 /1024" | bc -l ; done

bytes = MiB
8388608 = 8.00000000000000000000
847249408 = 808.00000000000000000000
1266679808 = 1208.00000000000000000000
1267728384 = 1209.00000000000000000000
1468006400 = 1400.00000000000000000000

I.e. regardless that disklayout.conf was not changed during "rear recover"

RESCUE d57:~ # diff -us /var/lib/rear/layout/disklayout.conf.20180223152350.recover.850.orig /var/lib/rear/layout/disklayout.conf
Files /var/lib/rear/layout/disklayout.conf.20180223152350.recover.850.orig and /var/lib/rear/layout/disklayout.conf are identical

I do not get same partitioning after "rear recover"
so something must still silently change the partitioning.

jsmeix commented at 2018-02-23 14:59:¶

I think my next step (for next week) will be to check the functions in
layout/prepare/GNU/Linux/100_include_partition_code.sh
to what extent those functions may change the partitioning
to something different than what is specified in disklayout.conf.

jsmeix commented at 2018-02-23 16:01:¶

I found out why in my disklayout.conf the
size of the extended partition is only 1024 bytes:

This happens in layout/save/GNU/Linux/200_partition_layout.sh
at this call

        size=$(get_disk_size ${path#/sys/block/})

that multiplies the values of
/sys/block/sdb/sdb*/size and /sys/block/sdb/queue/logical_block_size
which are in my case:

# cat /sys/block/sdb/sdb*/size
1638400
819200
2
186368
409600

# cat /sys/block/sdb/queue/logical_block_size
512

i.e. the /sys/block/sdb/sdb3/size value is 2 which is not right
but the other values are:

# for s in $( cat /sys/block/sdb/sdb*/size ) ; do echo "$s * 512" | bc -l ; done
838860800
419430400
1024
95420416
209715200

This fixes it for me:

--- a/usr/share/rear/layout/save/GNU/Linux/200_partition_layout.sh
+++ b/usr/share/rear/layout/save/GNU/Linux/200_partition_layout.sh
@@ -241,6 +239,23 @@ extract_partitions() {
                     sed -i /^$partition_nr\ /s/\ primary\ /\ extended\ / $TMP_DIR/partitions
                 fi
             fi
+
+            # Replace currently possibly wrong extended partition size value
+            # by the value that parted reports if those values differ, cf.
+            # https://github.com/rear/rear/pull/1733#issuecomment-368051895
+            # In SLE10 there is GNU Parted 1.6.25.1 which supports 'unit B'
+            # that is documented in 'info parted' (but not yet in 'man parted').
+            # Example of a parted_extended_partition_line:
+            #   # parted -s /dev/sdb unit B print | grep -w '3' | grep -w 'extended'
+            #    3      1266679808B  1790967807B  524288000B  extended                  lba, type=0f
+            # where the size is 524288000B i.e. parted_extended_partition_line[3]
+            parted_extended_partition_line=( $( parted -s $device unit B print | grep -w "$partition_nr" | grep -w 'extended' ) )
+            parted_extended_partition_size="${parted_extended_partition_line[3]%%B*}"
+            if test $size -ne $parted_extended_partition_size ; then
+                 Log "Replacing probably wrong extended partition size $size by what parted reports $parted_extended_partition_size"
+                 sed -i /^$partition_nr\ /s/\ $size\ /\ $parted_extended_partition_size\ / $TMP_DIR/partitions
+            fi
+
         done < $TMP_DIR/partitions-data
     fi

gdha commented at 2018-02-23 17:42:¶

@jsmeix Not sure you can see the content of https://access.redhat.com/solutions/2943451 ?

Issue

ReaR uses a value for disk size that does not match physical disk info.
The disk size determined by rear is 9592809455616B bytes
according to the disklayout.conf generated
The correct size is visible in the fdisk output below:

<snip>
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sda: 1199.1 GB, 1199101181952 bytes, 292749312 sectors
Units = sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
<snip>

Resolution
Permanent Solution

A permanent solution is currently being pursued via the following public bug report:
Workaround

So as to avoid the overage, the following alteration can be used:

1 - In the /usr/share/rear/lib/layout-functions.sh file, alter the following from:

# Get the block size of a disk.
get_block_size() {
    # Only newer kernels have an interface to get the block size
    if [ -r /sys/block/$1/queue/logical_block_size ] ; then
    echo $( < /sys/block/$1/queue/logical_block_size)
    else
    echo "512"
    fi
}

To:

# Get the block size of a disk.
get_block_size() {
    echo "512"
}

2 - Build a new rescue image.
3 - verify the resulting disklayout.conf and restoration succeeds with the correct size.
Root Cause

The reason for the failure is specifically that the kernel-provided /sys/block//queue/logical_block_size value is reported as 4096 bytes by the kernel instead of the traditional 512 bytes. This value is correct, due to the use of AF hard disks, but does not apply to the /sys/block//size interface that rear uses to determine the number of blocks. The latter is not a discovered value, but a block count measured in 512 bytes.

Disk /dev/sda: 1199.1 GB, 1199101181952 bytes, 292749312 sectors
Units = sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt

In the output above, the logical sector size is 4096 bytes. This causes the following to occur within the rear backup solution;

# Get the size in bytes of a disk/partition.
# For partitions, use "sda/sda1" as argument.
get_disk_size() {
    local disk_name=$1

    local block_size=$(get_block_size ${disk_name%/*})

    [ -r /sys/block/$disk_name/size ]
    BugIfError "Could not determine size of disk $disk_name, please file a bug."

    local nr_blocks=$( < /sys/block/$disk_name/size)
    local disk_size=$(( nr_blocks * block_size ))

    ### Make sure we always return a number
    echo $(( disk_size ))
}

# Get the block size of a disk.
get_block_size() {
    # Only newer kernels have an interface to get the block size
    if [ -r /sys/block/$1/queue/logical_block_size ] ; then
    echo $( < /sys/block/$1/queue/logical_block_size)
    else
    echo "512"
    fi
}

Note the disk_size member is evaluated as the nr_blocks (size) multiplied by the block_size (logical_block_size). This evaluation should be hard-coded to (nr_block * 512).

jsmeix commented at 2018-02-26 12:42:¶

My last commit here https://github.com/rear/rear/pull/1733/commits/6efb681d8b4c6a4d9f20b2900bbea79548c624a8
fixes only https://github.com/rear/rear/pull/1733#issuecomment-368051895 for me.

In contrast https://github.com/rear/rear/pull/1733#issuecomment-368083755
is a different issue with a different root cause (as far as I understand it).

Currently I do not understand why in
usr/share/rear/layout/save/GNU/Linux/200_partition_layout.sh
the 'part' data for disklayout.conf is not simply what parted reports
(because later also parted will be used to recreate those partitions)
but instead the partitioning data is read from /sys/block/ files?
Why is there that RFC 1925 item 6a "another level of indirection"?
Why not simply parted output data => parted input data?

Because I do not understand how layout/save/GNU/Linux/200_partition_layout.sh
works as a whole I can currently not fix or clean up generic things therein
or in the functions that belong to the disk layout code.

jsmeix commented at 2018-02-26 13:51:¶

Only FYI:

With extended partition size fixed
I get now those results:

Original system (same as above):

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2048MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system     Flags
 1      8.00MiB  808MiB   800MiB   primary   ext2            type=83
 2      808MiB   1208MiB  400MiB   primary   linux-swap(v1)  type=82
 3      1208MiB  1708MiB  500MiB   extended                  lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical   ext2            type=83
 6      1400MiB  1600MiB  200MiB   logical   ext2            type=83

d57:~/rear.issue1733 # parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2147483648B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start        End          Size        Type      File system     Flags
 1      8388608B     847249407B   838860800B  primary   ext2            type=83
 2      847249408B   1266679807B  419430400B  primary   linux-swap(v1)  type=82
 3      1266679808B  1790967807B  524288000B  extended                  lba, type=0f
 5      1267728384B  1363148799B  95420416B   logical   ext2            type=83
 6      1468006400B  1677721599B  209715200B  logical   ext2            type=83

On the recreated system

RESCUE d57:~ # rear -D recover
...
Last partition /dev/sdb6 will not be resized (original last partition still fits on the new smaller disk)
...
Finished recovering your system. You can explore it under '/mnt/local'.

RESCUE d57:~ # diff -us /var/lib/rear/layout/disklayout.conf.20180226142731.recover.786.orig /var/lib/rear/layout/disklayout.conf
Files /var/lib/rear/layout/disklayout.conf.20180226142731.recover.786.orig and /var/lib/rear/layout/disklayout.conf are identical

RESCUE d57:~ # grep sdb /var/lib/rear/layout/disklayout.conf
# Disk /dev/sdb
disk /dev/sdb 2147483648 msdos
# Partitions on /dev/sdb
part /dev/sdb 838860800 8388608 primary none /dev/sdb1
part /dev/sdb 419430400 847249408 primary none /dev/sdb2Last partition /dev/sdb6 will not be resized (original last partition still fits on the new smaller disk)
part /dev/sdb 524288000 1266679808 extended lba /dev/sdb3
part /dev/sdb 95420416 1267728384 logical none /dev/sdb5
part /dev/sdb 209715200 1468006400 logical none /dev/sdb6

RESCUE d57:~ # grep 'parted .*sdb' /var/lib/rear/layout/diskrestore.sh
parted -s /dev/sdb mklabel msdos >&2
parted -s /dev/sdb mkpart "'primary'" 2097152B 840957951B >&2
parted -s /dev/sdb mkpart "'primary'" 840962048B 1260392447B >&2
parted -s /dev/sdb mkpart "'extended'" 1260396544B 1932735487B >&2
parted -s /dev/sdb set 3 lba on >&2
parted -s /dev/sdb mkpart "'logical'" 1784688640B 1880109055B >&2
parted -s /dev/sdb mkpart "'logical'" 1880113152B 1932735487B >&2

RESCUE d57:~ # parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1843MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system  Flags
 1      2.00MiB  802MiB   800MiB   primary                type=83
 2      802MiB   1202MiB  400MiB   primary                type=83
 3      1202MiB  1843MiB  641MiB   extended               lba, type=0f
 5      1702MiB  1793MiB  91.0MiB  logical                type=83
 6      1793MiB  1843MiB  50.2MiB  logical   ext2         type=83

RESCUE d57:~ # parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1932735488B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start        End          Size        Type      File system  Flags
 1      2097152B     840957951B   838860800B  primary                type=83
 2      840962048B   1260392447B  419430400B  primary                type=83
 3      1260396544B  1932735487B  672338944B  extended               lba, type=0f
 5      1784688640B  1880109055B  95420416B   logical                type=83
 6      1880113152B  1932735487B  52622336B   logical   ext2         type=83

Regardless that my code in 420_autoresize_last_partitions.sh
does not change disklayout.conf in any way because
Last partition /dev/sdb6 will not be resized (original last partition still fits on the new smaller disk)
the actual parted commands in diskrestore.sh do not match
the partition values in disklayout.conf so that the resulting MiB values look o.k.
but the byte values reveal bad alignment after "rear recover" as in
https://github.com/rear/rear/pull/1733#issuecomment-368028494

Now I will try to fix things in 100_include_partition_code.sh ...

jsmeix commented at 2018-02-26 15:06:¶

With my latest commit here
https://github.com/rear/rear/pull/1733/commits/f4273dc500be87263297f88827508b64d521ccf1
things work o.k. for me for the first time:

Now I get:

RESCUE d57:~ # rear -D recover
...
Last partition /dev/sdb6 will not be resized (original last partition still fits on the new smaller disk)
...
Finished recovering your system. You can explore it under '/mnt/local'.

RESCUE d57:~ # diff -us /var/lib/rear/layout/disklayout.conf.20180226152840.recover.830.orig /var/lib/rear/layout/disklayout.conf
Files /var/lib/rear/layout/disklayout.conf.20180226152840.recover.830.orig and /var/lib/rear/layout/disklayout.conf are identical

RESCUE d57:~ # grep sdb /var/lib/rear/layout/disklayout.conf
# Disk /dev/sdb
disk /dev/sdb 2147483648 msdos
# Partitions on /dev/sdb
part /dev/sdb 838860800 8388608 primary none /dev/sdb1
part /dev/sdb 419430400 847249408 primary none /dev/sdb2
part /dev/sdb 524288000 1266679808 extended lba /dev/sdb3
part /dev/sdb 95420416 1267728384 logical none /dev/sdb5
part /dev/sdb 209715200 1468006400 logical none /dev/sdb6

RESCUE d57:~ # parted -s /dev/sdb unit B print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1932735488B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start        End          Size        Type      File system  Flags
 1      8388608B     847249407B   838860800B  primary                type=83
 2      847249408B   1266679807B  419430400B  primary                type=83
 3      1266679808B  1790967807B  524288000B  extended               lba, type=0f
 5      1267728384B  1363148799B  95420416B   logical                type=83
 6      1468006400B  1677721599B  209715200B  logical   ext2         type=83

RESCUE d57:~ # parted -s /dev/sdb unit B print | grep '^ [1-9]' | tr -s ' ' | cut -d ' ' -f3,5 | tr -d 'B'
8388608 838860800
847249408 419430400
1266679808 524288000
1267728384 95420416
1468006400 209715200

RESCUE d57:~ # grep '^part .* /dev/sdb[1-9]' /var/lib/rear/layout/disklayout.conf | cut -d ' ' -f3,4 | awk '{print $2, $1}'
8388608 838860800
847249408 419430400
1266679808 524288000
1267728384 95420416
1468006400 209715200

HOORAY!
Now there is byte-by-byte identical re-partitioning of the recreated sdb.

Now the first (and simplest) case in
https://github.com/rear/rear/pull/1733#issuecomment-367317079

1708MiB (or more):
no changes of the partitioning

seems to work for me as intended.

jsmeix commented at 2018-02-26 15:08:¶

Now I need to enhance step by step
layout/prepare/default/420_autoresize_last_partitions.sh
to make it also work for the other cases in
https://github.com/rear/rear/pull/1733#issuecomment-367317079

jsmeix commented at 2018-02-28 15:01:¶

With unchanged sdb as above I get now with

AUTORESIZE_PARTITIONS=( /dev/sda2 /dev/sdb6 /dev/sdb3 )

during "rear -D recover"

Examining /dev/sda to automatically resize its last active partition
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Found 'primary' partition /dev/sda2 as last partition on /dev/sda
Determining if last partition /dev/sda2 is resizeable
Last partition should be resized (/dev/sda2 in AUTORESIZE_PARTITIONS)
Determining new size for last partition /dev/sda2
Determining if last partition /dev/sda2 actually needs to be increased or shrinked
New /dev/sda is 107374592 bigger than old disk
Last partition /dev/sda2 cannot be resized (new disk less than 1% bigger)
Examining /dev/sdb to automatically resize its last active partition
Checking /dev/sdb1 if it is the last partition on /dev/sdb
Checking /dev/sdb2 if it is the last partition on /dev/sdb
Checking /dev/sdb3 if it is the last partition on /dev/sdb
Found extended partition /dev/sdb3 on /dev/sdb
Checking /dev/sdb5 if it is the last partition on /dev/sdb
Checking /dev/sdb6 if it is the last partition on /dev/sdb
Found 'logical' partition /dev/sdb6 as last partition on /dev/sdb
Determining if last partition /dev/sdb6 is resizeable
Last partition should be resized (/dev/sdb6 in AUTORESIZE_PARTITIONS)
Extended partition should be increased (/dev/sdb3 in AUTORESIZE_PARTITIONS)
Determining new size for last partition /dev/sdb6
Increasing extended partition /dev/sdb3 to end of disk
Increased extended partition /dev/sdb3 size from 524288000 to 665845760 bytes
Determining if last partition /dev/sdb6 actually needs to be increased or shrinked
New /dev/sdb is 214748160 smaller than old disk
Last partition /dev/sdb6 will not be resized (original last partition still fits on the new smaller disk)

which results only this change in disklayout.conf

-part /dev/sdb 524288000 1266679808 extended lba /dev/sdb3
+part /dev/sdb 665845760 1266679808 extended lba /dev/sdb3

and this recreated sdb:

RESCUE d57:~ # parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1843MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system  Flags
 1      8.00MiB  808MiB   800MiB   primary                type=83
 2      808MiB   1208MiB  400MiB   primary                type=83
 3      1208MiB  1843MiB  635MiB   extended               lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical                type=83
 6      1400MiB  1600MiB  200MiB   logical   ext2         type=83

all perfectly as I expect it to be.

Now the user can specify that only the extended partition is increased up to the end of disk
when there is no other partition after the end of the extended partition
i.e. when the last partition is a logical partition
regardless if also the last logical partition gets resized.
This way the user can migrate onto a bigger disk and gets only an increased extended partition
where he can later add more logical partitions as he likes.

jsmeix commented at 2018-03-01 08:31:¶

I will do a few more tests today (but I cannot test everything with reasonable effort)
and then I would like to merge it today (if it looks to work o.k. for me)
so that also others who use the current ReaR GitHub master code
will test it on their systems because I do need real user feedback
for such major changes while we are in development phase
to avoid bugs in the ReaR 2.4 release.

jsmeix commented at 2018-03-01 10:30:¶

The next case in
https://github.com/rear/rear/pull/1733#issuecomment-367317079

less than 1708MiB but not less than 1600MiB:
end extended is adjusted to new end of disk
but logical2 is not changed

works o.k. for me.

I have now in local.conf

AUTORESIZE_EXCLUDE_PARTITIONS=( boot swap efi )
AUTOINCREASE_DISK_SIZE_THRESHOLD_PERCENTAGE=1
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=40
BACKUP_PROG_INCLUDE=( /data.logical2 )

With same 2GiB original sdb as above
I get with a 1638MiB replacement sdb
during "rear recover"

Examining /dev/sda to automatically resize its last active partition
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Found 'primary' partition /dev/sda2 as last partition on /dev/sda
Determining if last partition /dev/sda2 is resizeable
Last partition /dev/sda2 not resizeable (used during boot)
Determining new size for last partition /dev/sda2
Determining if last partition /dev/sda2 actually needs to be increased or shrinked
New /dev/sda is 107374592 bigger than old disk
Skip increasing last partition /dev/sda2 (new disk less than 1% bigger)
Examining /dev/sdb to automatically resize its last active partition
Checking /dev/sdb1 if it is the last partition on /dev/sdb
Checking /dev/sdb2 if it is the last partition on /dev/sdb
Checking /dev/sdb3 if it is the last partition on /dev/sdb
Found extended partition /dev/sdb3 on /dev/sdb
Checking /dev/sdb5 if it is the last partition on /dev/sdb
Checking /dev/sdb6 if it is the last partition on /dev/sdb
Found 'logical' partition /dev/sdb6 as last partition on /dev/sdb
Determining if last partition /dev/sdb6 is resizeable
Determining new size for last partition /dev/sdb6
Shrinking extended partition /dev/sdb3 to end of disk
Shrinked extended partition /dev/sdb3 size from 524288000 to 450887680 bytes
Determining if last partition /dev/sdb6 actually needs to be increased or shrinked
New /dev/sdb is 429496320 smaller than old disk
Skip shrinking last partition /dev/sdb6 (original last partition still fits on the new smaller disk)

which results that recreated sdb

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1638MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system  Flags
 1      8.00MiB  808MiB   800MiB   primary                type=83
 2      808MiB   1208MiB  400MiB   primary                type=83
 3      1208MiB  1638MiB  430MiB   extended               lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical                type=83
 6      1400MiB  1600MiB  200MiB   logical   ext2         type=83

jsmeix commented at 2018-03-01 10:58:¶

The next case in
https://github.com/rear/rear/pull/1733#issuecomment-367317079

less than 1600MiB but still more than 1400MiB:
end extended and end logical2 are adjusted to new end of disk
plus test if new logical2 size is smaller than logical2 disk usage

works o.k for me.

With same 2GiB original sdb as above
I get with a 1536MiB replacement sdb
during "rear recover" (excerpts):

Examining /dev/sda to automatically resize its last active partition
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Found 'primary' partition /dev/sda2 as last partition on /dev/sda
Determining if last partition /dev/sda2 is resizeable
Last partition /dev/sda2 not resizeable (used during boot)
Determining new size for last partition /dev/sda2
Determining if last partition /dev/sda2 actually needs to be increased or shrinked
New /dev/sda is 107374592 bigger than old disk
Skip increasing last partition /dev/sda2 (new disk less than 1% bigger)
Examining /dev/sdb to automatically resize its last active partition
Checking /dev/sdb1 if it is the last partition on /dev/sdb
Checking /dev/sdb2 if it is the last partition on /dev/sdb
Checking /dev/sdb3 if it is the last partition on /dev/sdb
Found extended partition /dev/sdb3 on /dev/sdb
Checking /dev/sdb5 if it is the last partition on /dev/sdb
Checking /dev/sdb6 if it is the last partition on /dev/sdb
Found 'logical' partition /dev/sdb6 as last partition on /dev/sdb
Determining if last partition /dev/sdb6 is resizeable
Determining new size for last partition /dev/sdb6
WARNING: New size of last partition /dev/sdb6 will be smaller than its disk usage was
Shrinking extended partition /dev/sdb3 to end of disk
Shrinked extended partition /dev/sdb3 size from 524288000 to 343932928 bytes
Determining if last partition /dev/sdb6 actually needs to be increased or shrinked
New /dev/sdb is 536870912 smaller than old disk
Last partition /dev/sdb6 must be shrinked by 67108864 bytes to still fit on disk
Shrinking last partition /dev/sdb6 to end of disk (new disk at most 40% smaller)
Changed last partition /dev/sdb6 size from 209715200 to 142606336 bytes
...
Backup restore program 'tar' started in subshell (PID=3826)
Restored 0 MiB [avg. 0 KiB/sec] 
Restoring data.logical2/urandom150MiBs.data 
Restored 177 MiB [avg. 12114 KiB/sec] 
...
Backup restore program 'tar' failed with exit code '2'. Check '/var/log/rear/rear-d57.log' and the restored system.

which results that recreated sdb

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 1536MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system  Flags
 1      8.00MiB  808MiB   800MiB   primary                type=83
 2      808MiB   1208MiB  400MiB   primary                type=83
 3      1208MiB  1536MiB  328MiB   extended               lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical                type=83
 6      1400MiB  1536MiB  136MiB   logical   ext2         type=83

and those expected 'tar' error messages in the ReaR log file

# grep 'tar:' /var/log/rear/rear-d57.log
tar: data.logical2/urandom150MiBs.data: Wrote only 3584 of 10240 bytes
tar: Exiting with failure status due to previous errors

jsmeix commented at 2018-03-01 11:05:¶

The last case in
https://github.com/rear/rear/pull/1733#issuecomment-367317079

1400MiB (or less):
error out with some "no space for logical2 on disk" message

works o.k for me.

With same 2GiB original sdb as above
I get with a 1331MiB replacement sdb
during "rear recover" (excerpt):

Examining /dev/sda to automatically resize its last active partition
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Found 'primary' partition /dev/sda2 as last partition on /dev/sda
Determining if last partition /dev/sda2 is resizeable
Last partition /dev/sda2 not resizeable (used during boot)
Determining new size for last partition /dev/sda2
Determining if last partition /dev/sda2 actually needs to be increased or shrinked
New /dev/sda is 107374592 bigger than old disk
Skip increasing last partition /dev/sda2 (new disk less than 1% bigger)
Examining /dev/sdb to automatically resize its last active partition
Checking /dev/sdb1 if it is the last partition on /dev/sdb
Checking /dev/sdb2 if it is the last partition on /dev/sdb
Checking /dev/sdb3 if it is the last partition on /dev/sdb
Found extended partition /dev/sdb3 on /dev/sdb
Checking /dev/sdb5 if it is the last partition on /dev/sdb
Checking /dev/sdb6 if it is the last partition on /dev/sdb
Found 'logical' partition /dev/sdb6 as last partition on /dev/sdb
Determining if last partition /dev/sdb6 is resizeable
Determining new size for last partition /dev/sdb6
ERROR: No space for last partition /dev/sdb6 on new disk (new last partition size would be less than 1 MiB)
Aborting due to an error, check /var/log/rear/rear-d57.log for details

jsmeix commented at 2018-03-01 11:10:¶

With same 2GiB original sdb as above
I get with an also 2GiB sized replacement sdb
(but sda is still a bit bigger)
during "rear recover" (excerpt):

Examining /dev/sda to automatically resize its last active partition
Checking /dev/sda1 if it is the last partition on /dev/sda
Checking /dev/sda2 if it is the last partition on /dev/sda
Found 'primary' partition /dev/sda2 as last partition on /dev/sda
Determining if last partition /dev/sda2 is resizeable
Last partition /dev/sda2 not resizeable (used during boot)
Determining new size for last partition /dev/sda2
Determining if last partition /dev/sda2 actually needs to be increased or shrinked
New /dev/sda is 107374592 bigger than old disk
Skip increasing last partition /dev/sda2 (new disk less than 1% bigger)
Examining /dev/sdb to automatically resize its last active partition
Skipping /dev/sdb (size of new disk same as size of old disk)

jsmeix commented at 2018-03-01 11:16:¶

With same sized replacement sda and sdb
all works for me as before.

Of course not really as before with current master code! :-)

Now I get byte-by-byte identical partitioning on sdb,
in particular with exactly all the gaps as on the original system.

jsmeix commented at 2018-03-01 11:30:¶

With same sized replacement sda
but replacement sdb is more than
AUTOINCREASE_DISK_SIZE_THRESHOLD_PERCENTAGE=1
bigger I get during "rear recover" (excerpt):

Examining /dev/sda to automatically resize its last active partition
Skipping /dev/sda (size of new disk same as size of old disk)
Examining /dev/sdb to automatically resize its last active partition
Checking /dev/sdb1 if it is the last partition on /dev/sdb
Checking /dev/sdb2 if it is the last partition on /dev/sdb
Checking /dev/sdb3 if it is the last partition on /dev/sdb
Found extended partition /dev/sdb3 on /dev/sdb
Checking /dev/sdb5 if it is the last partition on /dev/sdb
Checking /dev/sdb6 if it is the last partition on /dev/sdb
Found 'logical' partition /dev/sdb6 as last partition on /dev/sdb
Determining if last partition /dev/sdb6 is resizeable
Determining new size for last partition /dev/sdb6
Determining if last partition /dev/sdb6 actually needs to be increased or shrinked
New /dev/sdb is 214749184 bigger than old disk
Increasing last partition /dev/sdb6 up to end of disk (new disk at least 1% bigger)
Increasing extended partition /dev/sdb3 up to end of disk
Changed last partition /dev/sdb6 size from 209715200 to 893386752 bytes
Increased extended partition /dev/sdb3 size from 524288000 to 1094713344 bytes

and this partitioning on sdb

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2253MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system  Flags
 1      8.00MiB  808MiB   800MiB   primary                type=83
 2      808MiB   1208MiB  400MiB   primary                type=83
 3      1208MiB  2252MiB  1044MiB  extended               lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical                type=83
 6      1400MiB  2252MiB  852MiB   logical   ext2         type=83

Note that only partition end values (and therefore partition sizes) got changed.
No partition start value was changed so no change in any partition alignment.
For easier comparison the original sdb (still same as above) here again:

# parted -s /dev/sdb unit MiB print
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdb: 2048MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number  Start    End      Size     Type      File system     Flags
 1      8.00MiB  808MiB   800MiB   primary   ext2            type=83
 2      808MiB   1208MiB  400MiB   primary   linux-swap(v1)  type=82
 3      1208MiB  1708MiB  500MiB   extended                  lba, type=0f
 5      1209MiB  1300MiB  91.0MiB  logical   ext2            type=83
 6      1400MiB  1600MiB  200MiB   logical   ext2            type=83

Only /dev/sdb6 is mounted on /data.logical2 which is the reason
why no other filesystems were re-created during "rear recover".

jsmeix commented at 2018-03-01 11:33:¶

Unless there are furious objections right now
I will merge it after lunch (i.e. in about one hour).

Of course I will fix bugs that appear when things get tested
also by other users on other systems "out there".

gdha commented at 2018-03-01 11:41:¶

@jsmeix because you were so brave to work on this damn complicated code you have my blessing 👍

jsmeix commented at 2018-03-01 13:13:¶

I used the code of my last commit here
https://github.com/rear/rear/pull/1733/commits/c09bae8c230b55ae5c9310af0c2bb040bd97c712
on my local test system for my tests starting at
https://github.com/rear/rear/pull/1733#issuecomment-369548894
i.e. the code of my last commit was tested by me.

jsmeix commented at 2018-03-01 13:15:¶

@gdha
thank you very much for your faith in me,
it is very much appreciated!

jsmeix commented at 2018-03-01 13:42:¶

Phew!

[Export of Github issue for rear/rear.]

#1733 PR merged: Replace 400_autoresize_disks by 420_autoresize_last_partitions and 430_autoresize_all_partitions¶