#2157 Issue `closed`: Backup tar file 0 bytes if encryption enabled when BACKUP_PROG_CRYPT_KEY contains special characters¶

Labels: enhancement, bug, documentation, fixed / solved / done

chumunga opened issue at 2019-06-07 17:43:¶

Relax-and-Recover (ReaR) Issue Template¶

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.4 / Git
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):
cat /etc/os-release

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

OUTPUT=ISO
BACKUP=NETFS
BACKUP_PROG=tar
BACKUP_TYPE=incremental
FULLBACKUPDAY="Mon"
FULLBACKUP_OUTDATED_DAYS=5
BACKUP_PROG_CRYPT_ENABLED=1
BACKUP_PROG_CRYPT_KEY=PASSWORD
BACKUP_PROG_CRYPT_OPTIONS="/usr/bin/openssl aes-256-cbc -salt -k"
BACKUP_PROG_DECRYPT_OPTIONS="/usr/bin/openssl aes-256-cbc -d -k"
BACKUP_URL=cifs://192.168.0.95/corp_server
BACKUP_OPTIONS="cred=/root/.confid/.cifs,vers=1.0"

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
PC
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
BIOS | GRUB
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
local NAS
Description of the issue (ideally so that others can reproduce it):
My config is pretty basic I would image but i had this issue before it went away and now it came back. When I run rear -v mkbackup or even the mkbackuponly
when I get the out put "preparing archive operationOK" I never see the archiving status it just simple completes the task as if it thinks it is backing up an an empty folder or it can complete archiving 2TB of data in <1 seconds ( dont we all wish)
Workaround, if any: I will report back if I can find one, I can't replicate this issue on my cent
Attachments, as applicable ("rear -D
backup.log
mkrescue/mkbackup/recover" debug log files):

chumunga commented at 2019-06-07 22:00:¶

So did some testing, I commented out the following and was able to start a backup where the TAR File had data. However, i've ran a backup with the setttings below the other day and it worked just fine granted only once. But I am getting consistent results.

I checked my openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017

#BACKUP_PROG_CRYPT_ENABLED="1"
OR
#BACKUP_PROG_CRYPT_ENABLED=1
OR
#BACKUP_PROG_CRYPT_ENABLED="true"
#BACKUP_PROG_CRYPT_KEY=TEST_PASS
OR
#BACKUP_PROG_CRYPT_KEY="TEST_PASS"
#BACKUP_PROG_CRYPT_OPTIONS="/usr/bin/openssl aes-256-cbc -salt -k"
#BACKUP_PROG_DECRYPT_OPTIONS="/usr/bin/openssl aes-256-cbc -d -k"

chumunga commented at 2019-06-11 23:52:¶

Not sure why but I was able to get backups to work by changing my Crypt Option to

BACKUP_PROG_CRYPT_OPTIONS="/usr/bin/openssl aes256 -salt -k"
BACKUP_PROG_DECRYPT_OPTIONS="/usr/bin/openssl aes256 -d -k"

to who ever is reading this, me posting here isn't for me but is more or less just to help if anyone else comes across this issue.

gdha commented at 2019-06-12 08:00:¶

@chumunga Thanks for bringing this up - it could help other users!

chumunga commented at 2019-06-13 23:36:¶

Not a problem. I will close this ticket.

jsmeix commented at 2019-06-17 12:14:¶

In default.conf there is

BACKUP_PROG_CRYPT_OPTIONS="/usr/bin/openssl des3 -salt -k "
BACKUP_PROG_DECRYPT_OPTIONS="/usr/bin/openssl des3 -d -k "

and that (i.e. with the default des3 openssl cipher command)
had "just worked" for me for my tests in
https://github.com/rear/rear/pull/2156

On my openSUSE Leap 15.0 system with OpenSSL 1.1.0
man openssl does not show any aes* cipher command.

chumunga commented at 2019-06-26 23:39:¶

I am replying since I just reopen the ticket for the same issue again. I tried encrypting a random text file
with openssl enc -aes-256-cbc -salt -in somefile.txt -out somefile.txt and I was prompted for a password.

I also tried openssl enc -aes256 -salt -in somefile.txt -out somefile.txt and it also prompted me fore a password for aes-256-cbc. I read somewhere that aes256 = aes-256-cbc.

As far as I can tell this is my version for centos 7

[root@it-cnt7 ~]# openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017

jsmeix commented at 2019-06-27 09:56:¶

@chumunga
you need to simulate what ReaR does
i.e. you need to try it out in the same way as ReaR calls it.

For BACKUP=NETFS with BACKUP_PROG=tar see
usr/share/rear/backup/NETFS/default/500_make_backup.sh

            $BACKUP_PROG $TAR_OPTIONS --sparse --block-number --totals --verbose    \
                --no-wildcards-match-slash --one-file-system                        \
                --ignore-failed-read "${BACKUP_PROG_OPTIONS[@]}"                    \
                $BACKUP_PROG_CREATE_NEWER_OPTIONS                                   \
                ${BACKUP_PROG_BLOCKS:+-b $BACKUP_PROG_BLOCKS}                       \
                "${BACKUP_PROG_COMPRESS_OPTIONS[@]}"                                \
                -X $TMP_DIR/backup-exclude.txt -C / -c -f -                         \
                $(cat $TMP_DIR/backup-include.txt) $RUNTIME_LOGFILE |               \
            { $BACKUP_PROG_CRYPT_OPTIONS $BACKUP_PROG_CRYPT_KEY ; } 2>/dev/null |   \
            $SPLIT_COMMAND

and
usr/share/rear/restore/NETFS/default/400_restore_backup.sh

                    dd if=$restore_input | { $BACKUP_PROG_DECRYPT_OPTIONS $BACKUP_PROG_CRYPT_KEY ; } 2>/dev/null | $BACKUP_PROG --block-number --totals --verbose "${BACKUP_PROG_OPTIONS[@]}" "${BACKUP_PROG_COMPRESS_OPTIONS[@]}" -C $TARGET_FS_ROOT/ -x -f -

i.e. within a pipe and with a specified key.

E.g. in your case do something like

cat somefile.in | openssl aes-256-cbc -salt -k testing_key | cat - 1>somefile.aes-256-cbc

and

cat somefile.aes-256-cbc | openssl aes-256-cbc -d -k testing_key | cat - 1>somefile.out

For comparison how it works on my openSUSE Leap 15.0 system
with OpenSSL 1.1.0 with the des3 openssl cipher command:

# echo Hello >somefile.in

# cat somefile.in | openssl des3 -salt -k testing_key | cat - >somefile.des3

# cat somefile.des3 | openssl des3 -d -k testing_key | cat - >somefile.out

# file somefile.*
somefile.des3: openssl enc'd data with salted password
somefile.in:   ASCII text
somefile.out:  ASCII text

# diff -s somefile.in somefile.out
Files somefile.in and somefile.out are identical

chumunga commented at 2019-06-27 20:23:¶

I think I figured it out. I tried your test jsmeix and it worked fine for me, the problem was my password.

My password was something like this

BACKUP_PROG_CRYPT_KEY="$SOMEPASS"
also tried
BACKUP_PROG_CRYPT_KEY=$SOMEPASS

The problem here was I believe I am passing a script a variable that has no value, openssl won't accept a no password, thur output a 0 byte file.

so ultimately what resolved it for me was either single quote OR don't start a password with $. And I believe if you use either above you will get the same results I was.

For me this worked and repeatable
BACKUP_PROG_CRYPT_KEY='$SOMEPASS'
or
BACKUP_PROG_CRYPT_KEY=SOMEPASS

Sorry if this is very obvious and I waste time, hopefully someone sees this and doesn't fall in the same stupidity i did.

jsmeix commented at 2019-06-28 07:17:¶

@chumunga
thank you for your explanatory information
what the root cause was in your particular case.
Such feedback helps us a lot because otherwise we wonder
if there is perhaps some obscure bug somewhere in ReaR.

For the fun of it:
Regarding special characters in passwords you may have a look at the section
"Use non-ASCII characters in usernames and passwords to lock yourself out" in
https://en.opensuse.org/SDB:Plain_Text_versus_Locale

Perhaps I should add a section about passwords
that contain executable scripting code like

my super $( rm -rf ) password

;-)

jsmeix commented at 2019-06-28 07:25:¶

I think I could add a test to avoid an empty password
according to "Try to care about possible errors" in
https://github.com/rear/rear/wiki/Coding-Style

Of course this does not help against problematic characters
in passwords (i.e. special bash characters) like

# password="foo$bar"
# echo $password
foo

# password='foo'bar'baz'
# echo $password
foobarbaz

chumunga commented at 2019-06-28 17:53:¶

@jsmeix Thank you! Really appreciate you guys!

Suggestion, maybe there is a way to encapsulate the password in single quotes so it becomes literal. However, ultimately I think if the default.conf file should states explicitly about the password's format as well as syntax this should be enough for any conscious admin. There comes a point you can't JUST google it ( I hope this day never comes)

jsmeix commented at 2019-07-02 13:19:¶

@chumunga
there are still some more things wrong with the code in ReaR
that deals with BACKUP_PROG_CRYPT_KEY.
I will fix it as time permits...

For now use a BACKUP_PROG_CRYPT_KEY value without
special characters or spaces to avoid unexpected weird effects...

jsmeix commented at 2019-07-08 12:42:¶

My https://github.com/rear/rear/pull/2178 intends to make
the BACKUP_PROG_CRYPT_KEY handling code more fail safe.

jsmeix commented at 2019-07-09 09:19:¶

With https://github.com/rear/rear/pull/2178 merged
the BACKUP_PROG_CRYPT_KEY handling should be more fail safe.

Special characters in the BACKUP_PROG_CRYPT_KEY
value should now work, cf.
https://github.com/rear/rear/pull/2178#issuecomment-509558118
but to be in general on the safe side it is still not recommended
to use special characters in values.

[Export of Github issue for rear/rear.]

#2157 Issue closed: Backup tar file 0 bytes if encryption enabled when BACKUP_PROG_CRYPT_KEY contains special characters¶