#3036 PR `merged`: Update 998_dump_variables.sh¶

Labels: fixed / solved / done, critical / security / legal

jsmeix opened issue at 2023-08-04 12:18:¶

Type: Bug Fix
Impact: Critical
Reference to related issue (URL):
https://github.com/rear/rear/issues/2967
How was this pull request tested?
see below
Brief description of the changes in this pull request:

Overhauled init/default/998_dump_variables.sh
so that its intended functionality happens
only when explicitly requested by the user
to avoid that possibly confidential values
are output into the log file by accident, see
https://github.com/rear/rear/issues/2967

jsmeix commented at 2023-08-04 12:39:¶

With current master code and
the changes in this pull request
I get:

# usr/sbin/rear help
Usage: rear [-h|--help] [-V|--version] ...
...
Use 'rear -v help' for more advanced commands.

# usr/sbin/rear -e help
Dumped all variable values (including possibly confidential values) into /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log.lockless
Usage: rear [-h|--help] [-V|--version] ...
...
Use 'rear -v help' for more advanced commands.

# less var/log/rear/rear-linux-h9wr.log.lockless
...
2023-08-04 14:22:05.392310858 Runtime Configuration:
                              declare -- ARCH="Linux-i386"
                              declare -ar ARGS=()
                              declare -x AUDIODRIVER="pulseaudio"
                              declare -- AUTOEXCLUDE_AUTOFS=""
                              ...
2023-08-04 14:22:05.435334881 Dumped all variable values (including possibly confidential values) into /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log.lockless
...

# usr/sbin/rear -e -v help
Running workflow help on the normal/original system
Dumped all variable values (including possibly confidential values) into /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log.lockless
Usage: rear [-h|--help] [-V|--version] ...
...
Exiting rear help (PID 1312) and its descendant processes ...
Running exit tasks

The above shows the interesting exception
that the 'help' workflow never shows "Using log file: ..."
which is another reason to show the log file name
to the user in init/default/998_dump_variables.sh

# usr/sbin/rear -d help
Running 'init' stage ======================
Running workflow help on the normal/original system
Usage: rear [-h|--help] [-V|--version]
...
Exiting rear help (PID 4673) and its descendant processes ...
Running exit tasks

# less var/log/rear/rear-linux-h9wr.log.lockless
...
2023-08-04 14:30:46.530563766 Including init/default/998_dump_variables.sh
2023-08-04 14:30:46.539265465 Skipped dumping all variable values into the log file (use 'expose-secrets' for that)
2023-08-04 14:30:46.543946935 Finished running 'init' stage in 0 seconds
2023-08-04 14:30:46.550306004 Running help workflow
...

# usr/sbin/rear -e savelayout
Dumped all variable values (including possibly confidential values) into /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log

# less var/log/rear/rear-linux-h9wr.log
...
2023-08-04 14:34:09.201589237 Runtime Configuration:
                              declare -- ARCH="Linux-i386"
                              declare -ar ARGS=()
                              declare -x AUDIODRIVER="pulseaudio"
                              declare -- AUTOEXCLUDE_AUTOFS=""
                              ...
2023-08-04 14:34:09.243533663 Dumped all variable values (including possibly confidential values) into /root/rear.dump_variables/var/log/rear/rear-linu
x-h9wr.log
...

# usr/sbin/rear -e -v savelayout
Relax-and-Recover 2.7 / Git
Running rear savelayout (PID 9521 date 2023-08-04 14:35:37)
Using log file: /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log
Running workflow savelayout on the normal/original system
Dumped all variable values (including possibly confidential values) into /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log
Creating disk layout
...

# usr/sbin/rear -d savelayout
Relax-and-Recover 2.7 / Git
Running rear savelayout (PID 13700 date 2023-08-04 14:36:53)
Command line options: usr/sbin/rear -d savelayout
Using log file: /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log
Using build area: /var/tmp/rear.XGFaw8EaGl3oh6O
Running 'init' stage ======================
Running workflow savelayout on the normal/original system
Running 'layout/save' stage ======================
Creating disk layout
...

# less var/log/rear/rear-linux-h9wr.log
...
2023-08-04 14:36:55.404213313 Including init/default/998_dump_variables.sh
2023-08-04 14:36:55.411377293 Skipped dumping all variable values into the log file (use 'expose-secrets' for that)
2023-08-04 14:36:55.415276482 Finished running 'init' stage in 1 seconds
...

jsmeix commented at 2023-08-04 12:41:¶

@schlomo @pcahyna @rear/contributors
please have a look (as time permits)
whether or not you like it this way.

If possible I would like to merge it later next week
(Wednesday or Thursday afternoon) unless there are objections.

jsmeix commented at 2023-08-05 05:57:¶

Oops!
I did it again!
One must be super careful to not leak secrets by accident!
Cf. https://github.com/rear/rear/issues/2967#issuecomment-1576722509

With current code:

# usr/sbin/rear -D help
...

# less var/log/rear/rear-linux-h9wr.log.lockless
...
+ source /root/rear.dump_variables/usr/share/rear/init/default/998_dump_variables.sh
+++ declare -p
++ LogSecret 'Runtime Configuration:
declare -- ARCH="Linux-i386"
declare -ar ARGS=()
declare -x AUDIODRIVER="pulseaudio"
declare -- AUTOEXCLUDE_AUTOFS=""
declare -- AUTOEXCLUDE_DISKS="y"
...
declare -- BACKUP_PROG_CRYPT_KEY=""
...
declare -- GALAXY11_PASSWORD=""
...
declare -- OPAL_PBA_TKNKEY="tpm:opalauthtoken:7"
...

jsmeix commented at 2023-08-05 06:45:¶

With my recent commit
https://github.com/rear/rear/pull/3036/commits/1e171478d76cb13c87948c4b13276710ef856f61
things should (hopefully) behave properly:

# usr/sbin/rear -D help
...

# less var/log/rear/rear-linux-h9wr.log.lockless
...
2023-08-05 08:32:06.111090259 Including init/default/998_dump_variables.sh
2023-08-05 08:32:06.114778739 Entering debugscript mode via 'set -x'.
+ source /root/rear.dump_variables/usr/share/rear/init/default/998_dump_variables.sh
++ Debug 'Skipped dumping all variable values into the log file (use '\''expose-secrets'\'' for that)'
++ test 1
++ Log 'Skipped dumping all variable values into the log file (use '\''expose-secrets'\'' for that)'
++ test -w /root/rear.dump_variables/var/log/rear/rear-linux-h9wr.log.lockless
++ echo '2023-08-05 08:32:06.126098024 Skipped dumping all variable values into the log file (use '\''expose-secrets'\'' for that)'
2023-08-05 08:32:06.126098024 Skipped dumping all variable values into the log file (use 'expose-secrets' for that)
+ source_return_code=0
...

# egrep -i 'pass|key|crypt|auth' var/log/rear/rear-linux-h9wr.log.lockless
++ COPY_AS_IS_EXCLUDE+=('/etc/pki/tls/private' '/etc/pki/CA/private' '/etc/pki/nssdb/key*.db' '/usr/lib/ssl/private')
++ has_binary dumpkeys
++ type dumpkeys
++ has_binary loadkeys
++ type loadkeys

jsmeix commented at 2023-08-05 06:50:¶

Now it's (hopefully) relaxed weekend time!

jsmeix commented at 2023-08-08 06:45:¶

@schlomo
thank you for your review!

@rear/contributors
I will merge it today afternoon unless objections appear.

[Export of Github issue for rear/rear.]

#3036 PR merged: Update 998_dump_variables.sh¶