#2506 PR merged: Make recreating LUKS volumes work with optional cryptsetup options¶
Labels: enhancement, cleanup, fixed / solved / done
jsmeix opened issue at 2020-11-02 13:36:¶
-
Type: Enhancement
-
Impact: Normal
-
Reference to related issue (URL):
https://github.com/rear/rear/pull/2504 -
How was this pull request tested?
Not yet tested by me. -
Brief description of the changes in this pull request:
The "cryptseup luksFormat" command does not require
any of the cipher, key-size, hash option values
because if omitted a cryptseup default value is used, cf.
https://github.com/rear/rear/pull/2504#issuecomment-720341023
jsmeix commented at 2020-11-02 14:17:¶
@vcrhonek
if you like to test my current changes here you could 'git clone' them
into a separated directory and run ReaR from inside that directory as
follows:
git clone --single-branch --branch recreate_LUKS_with_cryptsetup_optional_values https://github.com/rear/rear.git
mv rear rear.recreate_LUKS_with_cryptsetup_optional_values
cd rear.recreate_LUKS_with_cryptsetup_optional_values
vi etc/rear/local.conf
usr/sbin/rear -D mkrescue
For me my current changes result the same
var/lib/rear/layout/disklayout.conf
as before but I still need to test whether or not also "rear recover"
still works.
jsmeix commented at 2020-11-03 13:08:¶
For me all works well with "rear recover" as it did before
so it seems I did not introduce regressions.
But I did not yet test how things behave during "rear recover"
with missing optional cryptsetup option values or in real error cases.
jsmeix commented at 2020-11-03 13:34:¶
@vcrhonek
could you have a look at
https://github.com/rear/rear/pull/2504#issuecomment-721117530
and tell me what you think about it?
vcrhonek commented at 2020-11-04 07:50:¶
Changes look good and rear recover works as expected (rhel8, f32,
f25).
jsmeix commented at 2020-11-04 14:17:¶
I tested "rear recover" without optional cryptsetup option values.
Plain "rear recover" worked nevertheless
BUT
the recreated system failed to boot in dracut and/or systemd startup
when a uuid value was missing for a LUKS volume
that contains the root filesystem or
that contains a filesystem that is mountend via /etc/fstab.
The GRUB2 boot part worked without a uuid value
but when there was no LUKS uuid value for the root filesystem
further booting procedure fails in dracut and
when there was no LUKS uuid value for a filesystem in /etc/fstab
booting fails at some subsequent systemd service.
So uuid values are mantatory for LUKS volumes
that will be mounted during startup of the recreated system.
But this does not mean "rear recover" should error out in such cases
because I think it is possible to correct things to use the new UUIDs
via dracut's or systemd's emergency shell.
jsmeix commented at 2020-11-05 11:20:¶
I know now why the system (at least my openSUSE Leap 15.1 test system)
fails to boot when there is no uuid option set for the crypt entries
in disklayout.conf
where the uuid value must be "the right UUID" from the original
system.
Without uuid=<UUID_from_the_original_system> (or when there is no
uuid option set)
"rear recover" recreates LUKS volumes with UUIDs that are different than
they have been
on the original system.
For example on my original system I have:
# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT,UUID
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT UUID
/dev/sda /dev/sda ata disk 20G
|-/dev/sda1 /dev/sda1 /dev/sda part 8M
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 20G 0c58676a-bcb6-42be-8e1c-46a24d954ca7
`-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2
/dev/dm-0 /dev/sda2 crypt LVM2_member 20G zJalOt-2mjE-OrW4-MRxO-ErID-beeL-rRwzOp
|-/dev/mapper/system-swap /dev/dm-1 /dev/dm-0 lvm swap 2G [SWAP] a88670c6-43c7-4024-822f-f0fa0d00cfbc
|-/dev/mapper/system-root /dev/dm-2 /dev/dm-0 lvm btrfs 12.6G / bd326c18-0806-47d7-a740-97d5047d7de4
`-/dev/mapper/system-home /dev/dm-3 /dev/dm-0 lvm xfs 5.4G /home c9a5ebc9-3eac-4aa8-a768-731295af64a8
/dev/sdb /dev/sdb ata disk 1G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 307M fb79b19e-0e6d-4570-aa96-fa968d6e0795
| `-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1
| /dev/dm-4 /dev/sdb1 crypt ext4 305M /luks1test 745a0d13-2b73-4a81-a81c-96906c45ef5a
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS 409M 30376f43-60fd-4fc7-af0c-fad8063d5a1a
/dev/sr0 /dev/sr0 ata rom iso9660 657M 2020-01-08-15-17-54-22
/dev/sr1 /dev/sr1 ata rom iso9660 8.2G 2020-01-08-15-53-34-34
and that results by "rear mkrescue" in disklayout.conf those crypt
lines:
crypt /dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1 /dev/sdb1 type=luks1 cipher=aes-xts-plain64 key_size=256 hash=sha256 uuid=fb79b19e-0e6d-4570-aa96-fa968d6e0795
crypt /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2 /dev/sda2 type=luks1 cipher=aes-xts-plain64 key_size=256 hash=sha256 uuid=0c58676a-bcb6-42be-8e1c-46a24d954ca7
crypt /dev/mapper/luks2test /dev/sdb2 type=luks2 cipher=aes-xts-plain64 key_size=256 hash=sha256 uuid=30376f43-60fd-4fc7-af0c-fad8063d5a1a
In the ReaR recovery system I changed those disklayout.conf crypt
lines to
crypt /dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1 /dev/sdb1
crypt /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2 /dev/sda2
crypt /dev/mapper/luks2test /dev/sdb2 type=luks2
to test how "rear recover" behaves when tere are no uuid values.
Then "rear recover" recreated the following (only the relevant LUKS parts):
RESCUE linux-uxxi:~ # lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT,UUID | grep -A1 crypto_LUKS
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 20G c8b21495-e469-46a4-b76f-8bd190722364
`-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2 /dev/dm-1 /dev/sda2 crypt LVM2_member 20G zJalOt-2mjE-OrW4-MRxO-ErID-beeL-rRwzOp
--
|-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 307M fe5a41be-9dc5-4142-a366-a8282f202176
| `-/dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1 /dev/dm-0 /dev/sdb1 crypt ext4 305M /mnt/local/luks1test 745a0d13-2b73-4a81-a81c-96906c45ef5a
`-/dev/sdb2 /dev/sdb2 /dev/sdb part crypto_LUKS 409M 031c99cf-8878-4fa0-ad66-4f7ee0409830
`-/dev/mapper/luks2test /dev/dm-5 /dev/sdb2 crypt ext4 405M /mnt/local/luks2test 850e0fcc-6739-4190-9940-0b27cb82ee66
So we have now in particular (excerpts):
/dev/sda2 ... c8b21495-e469-46a4-b76f-8bd190722364
/dev/sdb1 ... fe5a41be-9dc5-4142-a366-a8282f202176
BUT
after "rear recover" /mnt/local/etc/crypttab is not adapted to the new
UUIDs
but it still contains the old ones from the original system:
RESCUE linux-uxxi:~ # cat /mnt/local/etc/crypttab
cr_ata-QEMU_HARDDISK_QM00001-part2 UUID=0c58676a-bcb6-42be-8e1c-46a24d954ca7
cr_ata-QEMU_HARDDISK_QM00004-part1 UUID=fb79b19e-0e6d-4570-aa96-fa968d6e0795
Manually adapting /mnt/local/etc/crypttab to the new UUIDs
RESCUE linux-uxxi:~ # cat /mnt/local/etc/crypttab
cr_ata-QEMU_HARDDISK_QM00001-part2 UUID=c8b21495-e469-46a4-b76f-8bd190722364
cr_ata-QEMU_HARDDISK_QM00004-part1 UUID=fe5a41be-9dc5-4142-a366-a8282f202176
and afterwards manually recreating the initrd and
finally manually reinstalling the bootloader
makes the recreated system boot.
If you wonder why my LUKS version 2 voulme 'luks2test' is not in
etc/crypttab:
This is because my LUKS version 2 test voulme is not mounted
automatically.
I do manually
# cryptsetup luksOpen /dev/sdb2 luks2test
# mount /dev/mapper/luks2test /luks2test
to use it (which still "just works" in the recreated system).
jsmeix commented at 2020-11-05 11:23:¶
So what is missing is a usr/share/rear/finalize/ script that
adapts /mnt/local/etc/crypttab to new UUIDs if needed
before the initrd is recreated and the bootloader is (re)-installed.
jsmeix commented at 2020-11-05 12:05:¶
I submitted
https://github.com/rear/rear/issues/2509
which is about the above
https://github.com/rear/rear/pull/2506#issuecomment-722316640
jsmeix commented at 2020-11-05 15:20:¶
Argh! How delicate the ReaR code is!
Via
https://github.com/rear/rear/pull/2506/commits/281f88b360f0a13d8d21c42929d7bee75ae73d30
I fixed a selfmade critical bug that does not show up
as long as one tests again and again on the same test system
(where the right disk partitions still existed from the tests before).
But it shows up when one tests on a new test system with new empty
disk
and then "rear -D recover" shows on the terminal (excerpts):
Skip recreating LUKS volume /dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1 on device '/dev/sdb1' that is no block device (see the 'crypt /dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1' entry in /var/lib/rear/layout/disklayout.conf)
...
Skip recreating LUKS volume /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2 on device '/dev/sda2' that is no block device (see the 'crypt /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2' entry in /var/lib/rear/layout/disklayout.conf)
...
Skip recreating LUKS volume /dev/mapper/luks2test on device '/dev/sdb2' that is no block device (see the 'crypt /dev/mapper/luks2test' entry in /var/lib/rear/layout/disklayout.conf)
...
Start system layout restoration.
Disk '/dev/sda': creating 'gpt' partition table
Disk '/dev/sda': creating partition number 1 with name ''sda1''
Disk '/dev/sda': creating partition number 2 with name ''sda2''
Disk '/dev/sdb': creating 'gpt' partition table
Disk '/dev/sdb': creating partition number 1 with name ''sdb1''
Disk '/dev/sdb': creating partition number 2 with name ''sdb2''
Creating LVM PV /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2
UserInput -I LAYOUT_CODE_RUN needed in /usr/share/rear/layout/recreate/default/200_run_layout_code.sh line 127
The disk layout recreation script failed
Of course it cannot
create LVM PV /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2
because it had falsely skipped
to recreate LUKS volume
/dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2
because its source device /dev/sda2 did not yet exist at that point in
time.
With the fix "rear -D recover" shows on the terminal (excerpts):
Start system layout restoration.
Disk '/dev/sda': creating 'gpt' partition table
Disk '/dev/sda': creating partition number 1 with name ''sda1''
Disk '/dev/sda': creating partition number 2 with name ''sda2''
Disk '/dev/sdb': creating 'gpt' partition table
Disk '/dev/sdb': creating partition number 1 with name ''sdb1''
Disk '/dev/sdb': creating partition number 2 with name ''sdb2''
Creating LUKS volume cr_ata-QEMU_HARDDISK_QM00004-part1 on /dev/sdb1
Set the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00004-part1 (for 'cryptsetup luksFormat' on /dev/sdb1):
Enter passphrase for /dev/sdb1:
Enter the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00004-part1 (for 'cryptsetup luksOpen' on /dev/sdb1):
Enter passphrase for /dev/sdb1:
Creating LUKS volume cr_ata-QEMU_HARDDISK_QM00001-part2 on /dev/sda2
Set the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00001-part2 (for 'cryptsetup luksFormat' on /dev/sda2):
Enter passphrase for /dev/sda2:
Enter the password for LUKS volume cr_ata-QEMU_HARDDISK_QM00001-part2 (for 'cryptsetup luksOpen' on /dev/sda2):
Enter passphrase for /dev/sda2:
Creating LVM PV /dev/mapper/cr_ata-QEMU_HARDDISK_QM00001-part2
Restoring LVM VG 'system'
Sleeping 3 seconds to let udev or systemd-udevd create their devices...
Creating filesystem of type btrfs with mount point / on /dev/mapper/system-root.
Mounting filesystem /
Running snapper/installation-helper
Creating filesystem of type ext4 with mount point /luks1test on /dev/mapper/cr_ata-QEMU_HARDDISK_QM00004-part1.
Mounting filesystem /luks1test
Creating filesystem of type xfs with mount point /home on /dev/mapper/system-home.
Mounting filesystem /home
Creating swap on /dev/mapper/system-swap
Creating LUKS volume luks2test on /dev/sdb2
Set the password for LUKS volume luks2test (for 'cryptsetup luksFormat' on /dev/sdb2):
Enter passphrase for /dev/sdb2:
Enter the password for LUKS volume luks2test (for 'cryptsetup luksOpen' on /dev/sdb2):
Enter passphrase for /dev/sdb2:
Creating filesystem of type ext4 with mount point /luks2test on /dev/mapper/luks2test.
Mounting filesystem /luks2test
Disk layout created.
jsmeix commented at 2020-11-05 16:25:¶
Now all works well for me with and without cryptsetup option values.
Without uuid it works as well as one can expect things to work
without
https://github.com/rear/rear/issues/2509
i.e. with the right manual adaptions as described in
https://github.com/rear/rear/issues/2509#issuecomment-722484629
jsmeix commented at 2020-11-05 16:26:¶
@rear/contributors @vcrhonek
if there are no objections I would like to merge it tomorrow afternoon.
vcrhonek commented at 2020-11-10 13:18:¶
Tested, no problem found, thank you!
jsmeix commented at 2020-11-10 13:23:¶
@vcrhonek
thank you for testing it again.
And thanks God that no new problems appeared ;-)
[Export of Github issue for rear/rear.]