#2204 Issue `closed`: Failed to recover LUKS version 2¶

Labels: enhancement, fixed / solved / done

adatum opened issue at 2019-08-11 08:19:¶

ReaR version ("/usr/sbin/rear -V"): 2.4 and 2.5
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"): Fedora 30
ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

PROGS=( "${PROGS[@]}" /home/test/Downloads/borg-linux64 )

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR): Virtualbox VM
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device): x86_64
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot): UEFI
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe): local SSD
Description of the issue (ideally so that others can reproduce it):
The disk layout recreation script fails after printing a message asking for the LUKS password, but never giving a chance to enter it. I tried rerunning it, but each time it skips to exactly the same point asking the options seen in the screenshot.

rear_luks_fail

gozora commented at 2019-08-11 09:30:¶

Can you please test if you have same behavior with ReaR 2.2 ?

V.

adatum commented at 2019-08-11 17:45:¶

Here's the log after it failed with rear-2.2:

rear_luks_error_log

gozora commented at 2019-08-11 18:16:¶

heh, this is even worse ... Never mind it was worth a try ;-).

Could you provide debug log from session executed with your most recent ReaR (guess it was 2.4)

V.

adatum commented at 2019-08-11 18:53:¶

Here is the log file from rear -vD recover with RearR 2.5

rear-localhost.log

This time I noticed a couple of message about UserInput. I'm not sure what they mean:
rear_UserInput
Additional notes:

Is it normal to be asked for a login? When ReaR finishes booting, it stops at the line in the screenshot with "[OK] Reached target Multi-User." Then if I press enter I see "localhost login:", which accepts any input (notice I typed "blah"). Is this expected behavior? It has been consistent between ReaR 2.2, 2.4, and 2.5.
To get rear mkrescue to succeed in creating an ISO, as stated in another issue, I must always remove "multiboot" from I believe line 49 of /usr/share/rear/lib/uefi-functions.sh for ReaR versions 2.2, 2.4, and 2.5, and also "linuxefi" for ReaR 2.2 and 2.4, but not 2.5.

gozora commented at 2019-08-11 19:09:¶

@adatum
Your problem is on line 3750 of your rear-localhost.log.
Executed code from 160_include_luks_code.sh, in your particular case: cryptsetup luksFormat --batch-mode --cipher - --key-size --hash --uuid 974b19f8-021a-46b6-a089-a46e06e6e746 --iter-time 2000 --use-random /dev/sda3 does not have any "safety net", meaning if it fails you will get exactly this behavior (prompt for password will be ignored)

Your command cryptsetup luksFormat --batch-mode --cipher - --key-size --hash --uuid 974b19f8-021a-46b6-a089-a46e06e6e746 --iter-time 2000 --use-random /dev/sda3 failed most probably due: --hash: invalid numeric value. Why you don't get ~~correct~~ hash from disklayout.conf, I really don't know.

According change history of 160_include_luks_code.sh, @OliverO2 was the last one doing changes in this file so he might be the most suitable one to help you ...

V.

adatum commented at 2019-08-11 19:24:¶

Thanks for noticing that @gozora If there's anything you'd like me to try, let me know. Since the system being backed up and recovered is just a test VM, I can try anything even if it's destructive.

Where is a good place to ask questions about ReaR that aren't necessarily issues? Besides the two "additional notes" above, I have a few other questions about ReaR's behavior.

OliverO2 commented at 2019-08-11 19:40:¶

Good evening @adatum, having been notified of this issue I took a short look. As @gozora noted correctly there is a missing hash parameter value. The root cause thus seems to be that the hash value is not properly detected by https://github.com/rear/rear/blob/ef074a6041ebb640b258dd36e4f8dbf3bf0e72fe/usr/share/rear/layout/save/GNU/Linux/260_crypt_layout.sh#L41

That code exists since its initial implementation on 2011, so its not something I was directly involved in. But maybe you could shed some light on it by running this command on the original system:

sudo cryptsetup luksDump /dev/sda3

The output is expected to look similar to this, in particular the line starting with Hash spec::

LUKS header information for /dev/sda3

Version:        1
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha256
Payload offset: 4096
MK bits:        512
[...]

adatum commented at 2019-08-11 19:46:¶

Thank you for the quick reply @OliverO2. Here is the luksDump. The hash spec is sha256 but notice that this is using LUKS version 2, which is now the default on recent Fedora releases. Could that have anything to do with the problem?

$ sudo cryptsetup luksDump /dev/sda3
LUKS header information
Version:        2
Epoch:          3
Metadata area:  16384 [bytes]
Keyslots area:  16744448 [bytes]
UUID:           974b19f8-021a-46b6-a089-a46e06e6e746
Label:          (no label)
Subsystem:      (no subsystem)
Flags:          (no flags)

Data segments:
  0: crypt
    offset: 16777216 [bytes]
    length: (whole device)
    cipher: aes-xts-plain64
    sector: 512 [bytes]

Keyslots:
  0: luks2
    Key:        512 bits
    Priority:   normal
    Cipher:     aes-xts-plain64
    Cipher key: 512 bits
    PBKDF:      argon2i
    Time cost:  4
    Memory:     973984
    Threads:    4
    Salt:       af 33 7e 3b 6c bb 55 dc e3 dc 2b 07 c5 9e c3 6d 
                f2 c9 08 be 2f 1d 8b 78 8a 33 65 90 41 e3 05 10 
    AF stripes: 4000
    AF hash:    sha256
    Area offset:32768 [bytes]
    Area length:258048 [bytes]
    Digest ID:  0
Tokens:
Digests:
  0: pbkdf2
    Hash:       sha256
    Iterations: 100361
    Salt:       d9 30 b6 7f 60 d0 e0 19 39 f6 a2 38 ae 22 88 43 
                1e 5c 74 75 e6 b5 dd db a9 e7 29 1a 74 64 9c 0f 
    Digest:     ae 06 29 5f 71 49 bd c8 75 de 53 e8 95 94 d3 38 
                57 43 5f 0e 1e ac 6d 59 fb 34 a3 97 e4 5a 94 0c

OliverO2 commented at 2019-08-11 20:30:¶

Yes, it seems like LUKS version 2 requires its own port within ReaR: The output format of cryptsetup luksDump differs significantly between the two versions, and that might be just the tip of the iceberg. One would have to analyze the implications of the version changes step by step to make ReaR save and reproduce a LUKS 2 setup correctly.

Maybe you have the option of just staying with LUKS version 1 for now until version 2 is more established and someone would come up with a port for ReaR.

adatum commented at 2019-08-11 21:07:¶

Thanks for confirming.

This complicates the situation. Of two systems I want to use with ReaR, one is using LUKS2.

LUKS2 has been supported on Fedora since 29, and is the default since Fedora 30.

LUKS allows converting between versions 1 and 2 under certain conditions:
https://www.saout.de/pipermail/dm-crypt/2017-December/005771.html

* In-place conversion form LUKS1

  To allow easy testing and transition to the new LUKS2 format, there is a new
  convert command that allows in-place conversion from the LUKS1 format and,
  if there are no incompatible options, also conversion back from LUKS2
  to LUKS1 format.

  Note this command can be used only on some LUKS1 devices (some device header
  sizes are not supported).
  This command is dangerous, never run it without header backup!
  If something fails in the middle of conversion (IO error), the header
  is destroyed. (Note that conversion requires move of keyslot data area to
  a different offset.)

  To convert header in-place to LUKS2 format, use
  $ cryptsetup convert <device> --type luks2

  To convert it back to LUKS1 format, use
  $ cryptsetup convert <device> --type luks1

  You can verify LUKS version with luksDump command.
  $ cryptsetup luksDump <device>

  Note that some LUKS2 features will make header incompatible with LUKS1 and
  conversion will be rejected (for example using new Argon2 PBKDF or integrity
  extensions). Some minor attributes can be lost in conversion.

Unfortunately that last point combined with Fedora's defaults prevents converting back to LUKS1. Here is the output from my conversion attempt on the VM with the luksDump from above:

$ sudo cryptsetup convert /dev/sda3 --type luks1

WARNING!
========
This operation will convert /dev/sda3 to LUKS1 format.


Are you sure? (Type uppercase yes): YES
Cannot convert to LUKS1 format - keyslot 0 is not LUKS1 compatible.

gozora commented at 2019-08-11 21:20:¶

@OliverO2 thanks for finding culprit!

Since fixing this issue would require some additional work, I'm assigning "need sponsorship" label and changing title of this issue.

If anyone wants to sponsor fix for this issue, please drop message to this thread to avoid double work.

V.

jsmeix commented at 2019-08-12 12:53:¶

I think in general missing support for a newer and incompatible version
of some software cannot be a bug in ReaR but is an enhancement.

adatum commented at 2019-08-18 06:31:¶

In the meantime, a workaround is to convert LUKS2 to LUKS1, which is by no means guaranteed to succeed, as "it depends" on many factors such as what features of LUKS2 are being used.

However this worked for me:

Boot from an environment other than the LUKS2 volume that is to be converted
Do cryptsetup luksConvertKey --pbkdf=pbkdf2 [/device/with/luks2] if for example argon2i is used (default on Fedora), which LUKS1 does not support.
Do cryptsetup convert [/device/with/luks2] --type luks1

https://unix.stackexchange.com/questions/535077/convert-luks2-back-to-luks-version-1/535081#535081

jsmeix commented at 2020-04-27 08:37:¶

With https://github.com/rear/rear/pull/2381
we detect at least when gathering crypt information
was not successful because LUKS 2 is used
instead of let the code blindly proceed resulting
wrong (incomplete) entries in disklayout.conf
that let later (when it is too late) "rear recover" fail.

The whole LUKS code needs to be overhauled to be more fail safe.
Currently it blindly proceeds basically everywhere, e.g. also
layout/prepare/GNU/Linux/160_include_luks_code.sh
blindly adds cryptsetup_options independent of whether or not
the resulting cryptsetup command is at least syntactically correct.

In
https://github.com/rear/rear/blob/master/doc/user-guide/06-layout-configuration.adoc
the LUKS Devices syntax description in disklayout.conf
indicates that most parameters are optional but it seems this is not the case.

github-actions commented at 2020-06-27 01:33:¶

Stale issue message

mannp commented at 2020-07-04 11:45:¶

Reading through the numerous updates to the thread, it seems LUKS2 is not supported, but the issue has been closed?

Is LUKS2 now supported?

Thanks for any clarifications 👍

gdha commented at 2020-07-04 12:24:¶

@mannp LUKS2 is not (yet) supported so far. Cold issues are getting automatically closed by GitHub Bot as it makes no sense if nothing happens (after 90 days) to keep them open. I f you feel it is so important you cannot live without it then sponsoring an issue is a valid option to keep it alive. Or, even better preparing a pull request or adding feedback to the issue which could be useful to get it fixed by someone.

mannp commented at 2020-07-04 12:33:¶

@mannp LUKS2 is not (yet) supported so far. Cold issues are getting automatically closed by GitHub Bot as it makes no sense if nothing happens (after 90 days) to keep them open. I f you feel it is so important you cannot live without it then sponsoring an issue is a valid option to keep it alive. Or, even better preparing a pull request or adding feedback to the issue which could be useful to get it fixed by someone.

Thanks @gdha I assumed as much, but as it had been open for almost a year I wondered why the bot closed now....updated 12 days ago....

Anyway, I have not been using rear as all my machines are luks2, just using alternatives :)

All the best.

adatum commented at 2020-07-04 16:14:¶

I have not been using rear as all my machines are luks2, just using alternatives

@mannp What alternatives have you found?

One way is to convert LUKS2 to LUKS1 https://github.com/rear/rear/issues/2204#issuecomment-522295377 to be able to use ReaR.

I find it unfortunate to close valid issues simply because they haven't been addressed in X days.

mannp commented at 2020-07-04 19:54:¶

I have not been using rear as all my machines are luks2, just using alternatives

@mannp What alternatives have you found?

One way is to convert LUKS2 to LUKS1 #2204 (comment) to be able to use ReaR.

I find it unfortunate to close valid issues simply because they haven't been addressed in X days.

Hi @adatum, not wanting to take anything away from ReaR, as I think it is a great idea

I use timeshift [https://github.com/teejee2008/timeshift] which is not perhaps a direct comparison, but is does support LUKS2 and has bailed me out on one occasion. Using it with arch it takes care of snapshots in between pacman updates.

That said I did subscribe to this thread as I would have liked another backup of my backup and I do use borg backup [https://github.com/borgbackup/borg] but only for user files, not bare metal restores.

That's where ReaR would be an great fit being able to use borg.

Hope that helps :)

kalos commented at 2020-09-12 14:06:¶

With the patch https://github.com/rear/rear/pull/2381, I get an error even if I exclude the device with LUKS2.

In this way, if I have mounted an (even external) LUKS2 device, which I have no intention to backing up, the creation of the layout fails:
ERROR: No hash value for LUKS device 'XXX_crypt' at '/dev/sdb1' (only LUKS version 1 is supported)

jsmeix commented at 2020-09-15 08:13:¶

@kalos
please report your case as a new issue via
https://github.com/rear/rear/issues/new
and provide the information we need as asked for there.
In particular I cannot guess what you had specified in
your etc/rear/local.conf to exclude the not needed thing
and how lsblk looks like on your particular system.
Additionally provide your var/lib/rear/layout/disklayout.conf
and var/lib/rear/layout/disktodo.conf files.

jsmeix commented at 2020-09-15 10:24:¶

https://github.com/rear/rear/issues/2204#issuecomment-691494336
is LUKS2 error even if the device is excluded reported via
https://github.com/rear/rear/issues/2491

jsmeix commented at 2020-10-28 11:19:¶

With https://github.com/rear/rear/pull/2504 merged
initial LUKS2 support was added tom ReaR.

@vcrhonek
thank you for your valuable contribution to ReaR!

[Export of Github issue for rear/rear.]

#2204 Issue closed: Failed to recover LUKS version 2¶