#1434 PR merged: Using UserInput in 300_map_disks.sh and general overhaul of that script¶
Labels: enhancement, cleanup, fixed / solved / done
jsmeix opened issue at 2017-07-28 12:12:¶
Using UserInput in 300_map_disks.sh and
a general overhaul of that script and
made UserInput less verbose unless in debug mode, cf.
https://github.com/rear/rear/issues/1431
Do not yet merge it - it is currently work-in-progress.
jsmeix commented at 2017-07-28 12:44:¶
FYI
how that currently looks for me during "rear recover"
RESCUE e205:~ # rear recover
...
Comparing disks.
Device sda has size 22548578304, 21474836480 expected
Switching to manual disk layout configuration.
Original disk /dev/sda does not exist in the target system.
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) Do not map /dev/sda
4) Use Relax-and-Recover shell and return back to here
(default 1 timeout 300 seconds)
1
Disk /dev/sda will be used as replacement for /dev/sda
This is the disk mapping table (source -> target):
/dev/sda /dev/sda
Please confirm that '/var/lib/rear/layout/disklayout.conf' is as you expect.
...
In my case I have a bigger /dev/sda and an additional /dev/sdb.
The messeges are not yet perfect, e.g.
Original disk /dev/sda does not exist in the target system.
should actually be
Original disk /dev/sda does not exist in the target system with same size.
otherwise the whole stuff looks as if ReaR was a bit stupid.
I added the "Use Relax-and-Recover shell and return back to here"
choice so that one can switch to a shell to find out what that
rather meaningless disk choices "/dev/sda" and "/dev/sdb"
actually are i.e. which real disks are meant with that.
schlomo commented at 2017-07-28 13:54:¶
I have a long standing wish to get rid of the pointless questions, especially the ones that ask me to choose the one single replacement option. Thanks a lot, we are getting there!
jsmeix commented at 2017-07-31 12:32:¶
With latest commit
https://github.com/rear/rear/pull/1434/commits/c059626865363fb1810d7d0a328969a9cee63f3d
I got rid of the pointless question that asks
to choose the one single option, cf.
https://github.com/rear/rear/issues/1366
But in migration mode we need a confirmation from the user
to be safe against wrong automated replacements
so that I added a general overall disk mapping confirmation
and disk mapping edit user dialog.
Because the UserInput functions supports user-defined timeout
settings (not yet officially documented but there for testing)
it now looks as follows
for a single replacement disk with different size
RESCUE e205:~ # export USER_INPUT_TIMEOUT=10
RESCUE e205:~ # rear recover
...
Comparing disks.
Device sda has size 22548578304, 21474836480 expected
Switching to manual disk layout configuration.
Original disk /dev/sda does not exist (with same size) in the target system
Using /dev/sda (the only appropriate) as replacement for /dev/sda
Current disk mapping table (source -> target):
/dev/sda /dev/sda
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default 1 timeout 10 seconds)
where it automatically proceeds after 10 seconds.
For two replacement disks with different size
RESCUE e205:~ # export USER_INPUT_TIMEOUT=10
RESCUE e205:~ # rear recover
...
Comparing disks.
Device sda has size 22548578304, 21474836480 expected
Switching to manual disk layout configuration.
Original disk /dev/sda does not exist (with same size) in the target system
Choose an appropriate replacement for /dev/sda
1) /dev/sda
2) /dev/sdb
3) Do not map /dev/sda
4) Use Relax-and-Recover shell and return back to here
(default 1 timeout 10 seconds)
Using /dev/sda (chosen by user) as replacement for /dev/sda
Current disk mapping table (source -> target):
/dev/sda /dev/sda
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default 1 timeout 10 seconds)
where it automatically proceeds after 10 seconds.
jsmeix commented at 2017-07-31 13:11:¶
@schabrolles
I would like your review here because recently you used
and tested the migration mode with multipath devices.
I would like to get a verification from you that
my completely overhauled 300_map_disks.sh
still works also for your use-cases.
I need this mainly because in my test scenarios
I have only simple disk device names like '/dev/sda'
where no actual device name translation via the
get_device_name function happens.
I would liie to know if the overhauled 300_map_disks.sh
still works when device name translations actually happen.
schabrolles commented at 2017-07-31 15:00:¶
Ok, it's in my todo list. I'll give it a try tomorrow.
schabrolles commented at 2017-08-03 08:09:¶
@jsmeix, I made a migration test with your 1434.patch
- OS tested: SLE12-SP2
- Source server: KVM guest with "virtual disk" vda
- Target server: PowerVM LPAR with SAN disk multipathed (mpatha)
I just use the latest version of ReaR from github (git pull / master
branch) and apply your 1434.patch with git am
Everything looks good, at least with SLE12-SP2 (I may be have to test also RHEL)
Welcome to Relax-and-Recover. Run "rear recover" to restore your system !
RESCUE rear-sles12:~ # rear -v recover
Relax-and-Recover 2.2-git.2362.fd303a1c.test1434 / 2017-08-01
Using log file: /var/log/rear/rear-rear-sles12.log
Running workflow recover within the ReaR rescue/recovery system
Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
Started RPC portmapper 'rpcbind'.
RPC portmapper 'rpcbind' available.
Started rpc.statd.
RPC status rpc.statd available.
Starting rpc.idmapd failed.
Using backup archive '/tmp/rear.PawliCVp3lKeRMt/outputfs/rear-sles12/backup.tar.gz'
Will do driver migration (recreating initramfs/initrd)
Calculating backup archive size
Backup archive size is 1.2G /tmp/rear.PawliCVp3lKeRMt/outputfs/rear-sles12/backup.tar.gz (compressed)
Setting up multipathing
mpathconf not found... creating default multipath.conf file with friendly_names
Activating multipath
multipath activated
mpatha (254, 0)
mpatha (3600507680c82004cf8000000000000d8) dm-0 IBM,2145
size=50G features='1 retain_attached_hw_handler' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| |- 1:0:0:0 sda 8:0 active undef running
| |- 1:0:1:0 sdb 8:16 active undef running
| |- 2:0:0:0 sde 8:64 active undef running
| `- 2:0:1:0 sdf 8:80 active undef running
`-+- policy='service-time 0' prio=0 status=enabled
|- 1:0:2:0 sdc 8:32 active undef running
|- 1:0:3:0 sdd 8:48 active undef running
|- 2:0:2:0 sdg 8:96 active undef running
`- 2:0:3:0 sdh 8:112 active undef running
Comparing disks.
Device vda does not exist.
Switching to manual disk layout configuration.
Using /dev/mapper/mpatha (same size) for recreating /dev/vda
Current disk mapping table (source -> target):
/dev/vda /dev/mapper/mpatha
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default 1 timeout 300 seconds)
1
User confirmed disk mapping
Please confirm that '/var/lib/rear/layout/disklayout.conf' is as you expect.
1) View disk layout (disklayout.conf) 4) Go to Relax-and-Recover shell
2) Edit disk layout (disklayout.conf) 5) Continue recovery
3) View original disk space usage 6) Abort Relax-and-Recover
#? 5
Partition primary on /dev/mapper/mpatha: size reduced to fit on disk.
Doing SLES12 special btrfs subvolumes setup because the default subvolume path contains '@/.snapshots/'
Please confirm that '/var/lib/rear/layout/diskrestore.sh' is as you expect.
1) View restore script (diskrestore.sh) 4) Go to Relax-and-Recover shell
2) Edit restore script (diskrestore.sh) 5) Continue recovery
3) View original disk space usage 6) Abort Relax-and-Recover
#? 5
Start system layout restoration.
Creating partitions for disk /dev/mapper/mpatha (msdos)
Creating LVM PV /dev/mapper/mpatha-part2
0 logical volume(s) in volume group "system" now active
Creating LVM VG system
Creating LVM volume system/root
Wiping btrfs signature on /dev/system/root.
Logical volume "root" created.
Creating LVM volume system/swap
Wiping swap signature on /dev/system/swap.
Logical volume "swap" created.
Creating filesystem of type btrfs with mount point / on /dev/mapper/system-root.
btrfs-progs v4.5.3+20160729
See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID: 6949ef9a-b935-40a8-ab9f-722724484d7b
Node size: 65536
Sector size: 65536
Filesystem size: 40.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: DUP 1.01GiB
System: DUP 12.00MiB
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 40.00GiB /dev/mapper/system-root
Mounting filesystem /
Create subvolume '/mnt/local/@'
Create subvolume '/mnt/local//@/boot/grub2/powerpc-ieee1275'
Create subvolume '/mnt/local//@/home'
Create subvolume '/mnt/local//@/opt'
Create subvolume '/mnt/local//@/srv'
Create subvolume '/mnt/local//@/tmp'
Create subvolume '/mnt/local//@/usr/local'
Create subvolume '/mnt/local//@/var/cache'
Create subvolume '/mnt/local//@/var/crash'
Create subvolume '/mnt/local//@/var/lib/libvirt/images'
Create subvolume '/mnt/local//@/var/lib/machines'
Create subvolume '/mnt/local//@/var/lib/mailman'
Create subvolume '/mnt/local//@/var/lib/mariadb'
Create subvolume '/mnt/local//@/var/lib/mysql'
Create subvolume '/mnt/local//@/var/lib/named'
Create subvolume '/mnt/local//@/var/lib/pgsql'
Create subvolume '/mnt/local//@/var/log'
Create subvolume '/mnt/local//@/var/opt'
Create subvolume '/mnt/local//@/var/spool'
Create subvolume '/mnt/local//@/var/tmp'
Running snapper/installation-helper:
step 1 device:/dev/mapper/system-root
temporarily mounting device
copying/modifying config-file
creating filesystem config
creating snapshot
setting default subvolume
done
Creating swap on /dev/mapper/system-swap
Disk layout created.
Restoring from '/tmp/rear.PawliCVp3lKeRMt/outputfs/rear-sles12/backup.tar.gz'...
Restored 3466 MiB [avg. 78884 KiB/sec] OK
Restored 3466 MiB in 46 seconds [avg. 77169 KiB/sec]
Restoring finished.
Restore the Mountpoints (with permissions) from /var/lib/rear/recovery/mountpoint_permissions
Patching file 'boot/grub2/grub.cfg'
Patching file 'boot/grub2/device.map'
Patching file 'etc/sysconfig/bootloader'
Patching file 'etc/fstab'
Patching file 'etc/smartd.conf'
Patching file 'etc/sysconfig/smartmontools'
Patching file 'etc/security/pam_mount.conf.xml'
Updating udev configuration (70-persistent-net.rules)
/etc/multipath.conf not available in target, creating it...
/etc/multipath/bindings copied to /mnt/local
successfully reset wwids
Running mkinitrd...
Recreated initrd (/sbin/mkinitrd).
Installing GRUB2 boot loader
Boot partition found: /dev/mapper/mpatha-part1
GRUB2 installed on /dev/mapper/mpatha-part1
PPC PReP Boot partition found: /dev/mapper/mpatha-part1
Boot device disk is /dev/mapper/mpatha.
Limiting bootlist to 5 entries...
Set LPAR bootlist to /dev/sdh /dev/sdg /dev/sdd /dev/sdc /dev/sdf
snapper setup-quota done
Finished recovering your system. You can explore it under '/mnt/local'.
jsmeix commented at 2017-08-03 09:24:¶
@schabrolles
many thanks for your testing!
I think it is sufficient to merge this one.
If issues appear there is time to fix them for ReaR v 2.3
that is currently planned for end of October 2017.
jsmeix commented at 2017-08-03 09:31:¶
Next steps are the
Please confirm that '/var/lib/rear/layout/disklayout.conf' is as you expect.
and
Please confirm that '/var/lib/rear/layout/diskrestore.sh' is as you expect.
user dialogs to use UserInput also there
but that would happen after my summer vacation...
jsmeix commented at 2017-08-03 09:54:¶
@schabrolles
in your 'rear -v recover' output in
https://github.com/rear/rear/pull/1434#issuecomment-319899757
I noticed messages that look contradicting
Using /dev/mapper/mpatha (same size) for recreating /dev/vda
versus later
Partition primary on /dev/mapper/mpatha: size reduced to fit on disk.
because when the replacement disk /dev/mapper/mpatha
has same size as the original disk /dev/vda
I would not expect that a partition size needs
to be reduced to fit on disk.
The 'size reduced to fit on disk' message comes from the
create_partitions() function in 100_include_partition_code.sh
A probably related issue could be
https://github.com/rear/rear/issues/102
where
https://github.com/rear/rear/issues/102#issuecomment-149550235
might even explain what happens here.
Accordingly the reason for the "size reduced to fit on disk" here
could be a (very) small size reduction (e.g. only a few MBs)
to get the recreated partition aligned on certain "boundaries"
(such alignment happens only in migration mode).
@schabrolles
to verify how much it was reduced in your case,
you need to inspect your log file because
the create_partitions() function code is
LogPrint "Partition $name on $device: size reduced to fit on disk." Log "End changed from $end to $device_size."
I do not understand why the actual reduction value
is only in the log but not also directly shown to the user?
schabrolles commented at 2017-08-04 10:52:¶
@jsmeix I'm gonna have a look on it.
thanks
[Export of Github issue for rear/rear.]