#2633 PR `merged`: Have stdout and stderr also in non-debug modes¶

Labels: enhancement, cleanup, fixed / solved / done

jsmeix opened issue at 2021-06-21 13:41:¶

Type: Enhancement
Impact: High
Reference to related issue (URL):
https://github.com/rear/rear/issues/2623
https://github.com/rear/rear/issues/2416
How was this pull request tested?
Only some "rear mkrescue" tests by me.
Brief description of the changes in this pull request:

In non-debug modes (in particular also in verbose mode)
stdout and stderr are redirected to a temporary file
STDOUT_STDERR_FILE="$TMP_DIR/rear.$WORKFLOW.stdout_stderr
so in non-debug modes stdout and stderr of all programs is still available
for the Error function to extract some latest messages,
cf. at the end of
https://github.com/rear/rear/pull/2627#issuecomment-857594905

Perhaps we should redirect in non-debug modes strderr in general
into a separated normal file where the Error function could extract things
so in non-debug modes the ReaR log cannot contain possibly false alarm messages
but in non-debug modes stderr of all programs is still available for the Error function
to extract some latest stderr messages.

Furthermore the log files and the stdout/stderr file can now only read by root
because they may contain sensitive information
so we do now in usr/sbin/rear

rm -f "$RUNTIME_LOGFILE"
cat /dev/null >"$RUNTIME_LOGFILE"
chmod u=rw,go=- "$RUNTIME_LOGFILE"

rm -f "$STDOUT_STDERR_FILE"
cat /dev/null >"$STDOUT_STDERR_FILE"
chmod u=rw,go=- "$STDOUT_STDERR_FILE"

rm -f "$LOGFILE"
cat /dev/null >"$LOGFILE"
chmod u=rw,go=- "$LOGFILE"

cf. https://github.com/rear/rear/issues/2416#issuecomment-856034599
I hope this makes it sufficiently secure for how ReaR is meant to be used
(at least it should now be more secure than before)
but I am not a security expert.

jsmeix commented at 2021-06-21 14:43:¶

To test how error out behaves
I added at the beginning of
init/default/030_update_recovery_system.sh

ls -l
grep foo /etc/QQQ
grep -Q foo /etc/QQQ || Error "QQQ"

I get this results:

Neither verbose nor debug mode:

# usr/sbin/rear mkrescue
ERROR: QQQ
Some messages from /tmp/rear.C0YmACGf663Ywi8/tmp/rear.mkrescue.stdout_stderr since the last called script 030_update_recovery_system.sh:
  drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
  drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
  grep: /etc/QQQ: No such file or directory
  grep: invalid option -- 'Q'
  Usage: grep [OPTION]... PATTERN [FILE]...
  Try 'grep --help' for more information.
Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details
Terminated

# cat var/log/rear/rear-linux-h9wr.log
2021-06-21 16:17:18.647797544 Relax-and-Recover 2.6 / Git
2021-06-21 16:17:18.649223594 Running rear mkrescue (PID 12269 date 2021-06-21 16:17:18)
2021-06-21 16:17:18.657266023 Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
2021-06-21 16:17:18.659001612 Including /root/rear.github.master/etc/rear/os.conf
2021-06-21 16:17:18.662211126 Including conf/Linux-i386.conf
2021-06-21 16:17:18.663576455 Including conf/GNU/Linux.conf
2021-06-21 16:17:29.902951469 Including conf/SUSE_LINUX.conf
2021-06-21 16:17:29.906149108 Including /root/rear.github.master/etc/rear/local.conf
2021-06-21 16:17:29.907973283 ======================
2021-06-21 16:17:29.909118602 Running 'init' stage
2021-06-21 16:17:29.910225888 ======================
2021-06-21 16:17:29.917067235 Including init/default/005_verify_os_conf.sh
2021-06-21 16:17:29.920970381 Including init/default/010_EFISTUB_check.sh
2021-06-21 16:17:29.924775739 Including init/default/010_set_drlm_env.sh
2021-06-21 16:17:29.928606592 Including init/default/030_update_recovery_system.sh
2021-06-21 16:17:29.945287828 ERROR: QQQ
2021-06-21 16:17:29.952696043 Exiting rear mkrescue (PID 12269) and its descendant processes ...
2021-06-21 16:17:32.982436932 rear,12269 usr/sbin/rear mkrescue
  `-rear,12455 usr/sbin/rear mkrescue
      `-pstree,12456 -Aplau 12269
2021-06-21 16:17:33.003582629 Running exit tasks

Verbose mode:

# usr/sbin/rear -v mkrescue
Relax-and-Recover 2.6 / Git
Running rear mkrescue (PID 12532 date 2021-06-21 16:21:41)
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
ERROR: QQQ
Some messages from /tmp/rear.pBxFQyZ8HaKaMrs/tmp/rear.mkrescue.stdout_stderr since the last called script 030_update_recovery_system.sh:
  drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
  drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
  grep: /etc/QQQ: No such file or directory
  grep: invalid option -- 'Q'
  Usage: grep [OPTION]... PATTERN [FILE]...
  Try 'grep --help' for more information.
Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details
Exiting rear mkrescue (PID 12532) and its descendant processes ...
Running exit tasks
Terminated

# cat var/log/rear/rear-linux-h9wr.log
2021-06-21 16:21:41.731357254 Relax-and-Recover 2.6 / Git
2021-06-21 16:21:41.733007827 Running rear mkrescue (PID 12532 date 2021-06-21 16:21:41)
2021-06-21 16:21:41.734696245 Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
2021-06-21 16:21:41.737243264 Including /root/rear.github.master/etc/rear/os.conf
2021-06-21 16:21:41.741022269 Including conf/Linux-i386.conf
2021-06-21 16:21:41.742469381 Including conf/GNU/Linux.conf
2021-06-21 16:21:52.975399319 Including conf/SUSE_LINUX.conf
2021-06-21 16:21:52.978555950 Including /root/rear.github.master/etc/rear/local.conf
2021-06-21 16:21:52.981752239 ======================
2021-06-21 16:21:52.983232209 Running 'init' stage
2021-06-21 16:21:52.984893301 ======================
2021-06-21 16:21:52.992378971 Including init/default/005_verify_os_conf.sh
2021-06-21 16:21:52.996270758 Including init/default/010_EFISTUB_check.sh
2021-06-21 16:21:53.000276195 Including init/default/010_set_drlm_env.sh
2021-06-21 16:21:53.004594496 Including init/default/030_update_recovery_system.sh
2021-06-21 16:21:53.022177502 ERROR: QQQ
2021-06-21 16:21:53.028851326 Exiting rear mkrescue (PID 12532) and its descendant processes ...
2021-06-21 16:21:56.058817271 rear,12532 usr/sbin/rear -v mkrescue
  `-rear,12722 usr/sbin/rear -v mkrescue
      `-pstree,12723 -Aplau 12532
2021-06-21 16:21:56.078866658 Running exit tasks

Debug mode

# usr/sbin/rear -d mkrescue
Relax-and-Recover 2.6 / Git
Running rear mkrescue (PID 13137 date 2021-06-21 16:33:10)
Command line options: usr/sbin/rear -d mkrescue
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area '/tmp/rear.mGbwF14RslXO0PM'
ERROR: QQQ
Some latest log messages since the last called script 030_update_recovery_system.sh:
  drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
  drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
  grep: /etc/QQQ: No such file or directory
  grep: invalid option -- 'Q'
  Usage: grep [OPTION]... PATTERN [FILE]...
  Try 'grep --help' for more information.
Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details
Exiting rear mkrescue (PID 13137) and its descendant processes ...
Running exit tasks
Terminated

# cat var/log/rear/rear-linux-h9wr.log
2021-06-21 16:33:10.419568152 Relax-and-Recover 2.6 / Git
2021-06-21 16:33:10.420986922 Running rear mkrescue (PID 13137 date 2021-06-21 16:33:10)
2021-06-21 16:33:10.422755464 Command line options: usr/sbin/rear -d mkrescue
2021-06-21 16:33:10.424428319 Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
2021-06-21 16:33:10.426022802 Using build area '/tmp/rear.mGbwF14RslXO0PM'
2021-06-21 16:33:10.427667941 Current set of flags is 'hB'
2021-06-21 16:33:10.429514524 The debugscripts flags are 'x'
2021-06-21 16:33:10.431099319 Combining configuration files
2021-06-21 16:33:10.432720432 Including /root/rear.github.master/etc/rear/os.conf
2021-06-21 16:33:10.436492351 Including conf/Linux-i386.conf
2021-06-21 16:33:10.438143334 Including conf/GNU/Linux.conf
2021-06-21 16:33:21.666554903 Including conf/SUSE_LINUX.conf
2021-06-21 16:33:21.669697162 Including /root/rear.github.master/etc/rear/local.conf
2021-06-21 16:33:21.672648736 ======================
2021-06-21 16:33:21.673779174 Running 'init' stage
2021-06-21 16:33:21.674918251 ======================
2021-06-21 16:33:21.681624941 Including init/default/005_verify_os_conf.sh
2021-06-21 16:33:21.685286704 Including init/default/010_EFISTUB_check.sh
2021-06-21 16:33:21.688975604 Including init/default/010_set_drlm_env.sh
2021-06-21 16:33:21.692631321 Including init/default/030_update_recovery_system.sh
total 88
-rw-r--r-- 1 root root 35147 Jun 21 08:07 COPYING
-rw-r--r-- 1 root root   451 Jun 21 08:07 MAINTAINERS
-rw-r--r-- 1 root root 10762 Jun 21 08:07 Makefile
-rw-r--r-- 1 root root 10967 Jun 21 08:07 README.adoc
drwxr-xr-x 4 root root  4096 Jun 21 08:07 doc
drwxr-xr-x 4 root root  4096 Jun 21 08:07 etc
drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
grep: /etc/QQQ: No such file or directory
grep: invalid option -- 'Q'
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
2021-06-21 16:33:21.705876122 ERROR: QQQ
===== Stack trace =====
Trace 0: usr/sbin/rear:572 main
Trace 1: /root/rear.github.master/usr/share/rear/lib/framework-functions.sh:127 SourceStage
Trace 2: /root/rear.github.master/usr/share/rear/lib/framework-functions.sh:59 Source
Trace 3: /root/rear.github.master/usr/share/rear/init/default/030_update_recovery_system.sh:6 source
=== End stack trace ===
2021-06-21 16:33:21.713680107 Exiting rear mkrescue (PID 13137) and its descendant processes ...
2021-06-21 16:33:24.743243842 rear,13137 usr/sbin/rear -d mkrescue
  `-rear,13319 usr/sbin/rear -d mkrescue
      `-pstree,13320 -Aplau 13137
/root/rear.github.master/usr/share/rear/lib/_input-output-functions.sh: line 151: kill: (13323) - No such process
2021-06-21 16:33:24.763465183 Running exit tasks
2021-06-21 16:33:24.765350420 Exit task '(( EXIT_FAIL_MESSAGE )) && echo 'rear mkrescue failed, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details' 1>&8'
2021-06-21 16:33:24.767604205 Exit task 'exec 8>&-'
2021-06-21 16:33:24.769253004 Exit task 'exec 7>&-'
2021-06-21 16:33:24.770796233 Exit task 'exec 6<&-'
2021-06-21 16:33:24.772304636 Exit task ''

Debugscript mode

# usr/sbin/rear -D mkrescue
Relax-and-Recover 2.6 / Git
Running rear mkrescue (PID 13399 date 2021-06-21 16:34:59)
Command line options: usr/sbin/rear -D mkrescue
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area '/tmp/rear.nOuCSDTwIRbaWQY'
ERROR: QQQ
Some latest log messages since the last called script 030_update_recovery_system.sh:
  drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
  drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
  grep: /etc/QQQ: No such file or directory
  grep: invalid option -- 'Q'
  Usage: grep [OPTION]... PATTERN [FILE]...
  Try 'grep --help' for more information.
Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details
Exiting rear mkrescue (PID 13399) and its descendant processes ...
Running exit tasks
Terminated

# cat var/log/rear/rear-linux-h9wr.log
.
.
.
+ source /root/rear.github.master/usr/share/rear/init/default/030_update_recovery_system.sh
++ ls -l
total 88
-rw-r--r-- 1 root root 35147 Jun 21 08:07 COPYING
-rw-r--r-- 1 root root   451 Jun 21 08:07 MAINTAINERS
-rw-r--r-- 1 root root 10762 Jun 21 08:07 Makefile
-rw-r--r-- 1 root root 10967 Jun 21 08:07 README.adoc
drwxr-xr-x 4 root root  4096 Jun 21 08:07 doc
drwxr-xr-x 4 root root  4096 Jun 21 08:07 etc
drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
++ grep foo /etc/QQQ
grep: /etc/QQQ: No such file or directory
++ grep -Q foo /etc/QQQ
grep: invalid option -- 'Q'
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
++ Error QQQ
++ PrintError 'ERROR: QQQ'
.
.
.

Because there is nothing in the log about the error in non-debug modes
I think in non-debug modes something from STDOUT_STDERR_FILE
should be copied into the log , perhaps all lines in STDOUT_STDERR_FILE
since the last sourced script.
In this case this would be

Including init/default/030_update_recovery_system.sh
total 88
-rw-r--r-- 1 root root 35147 Jun 21 08:07 COPYING
-rw-r--r-- 1 root root   451 Jun 21 08:07 MAINTAINERS
-rw-r--r-- 1 root root 10762 Jun 21 08:07 Makefile
-rw-r--r-- 1 root root 10967 Jun 21 08:07 README.adoc
drwxr-xr-x 4 root root  4096 Jun 21 08:07 doc
drwxr-xr-x 4 root root  4096 Jun 21 08:07 etc
drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
grep: /etc/QQQ: No such file or directory
grep: invalid option -- 'Q'
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
===== Stack trace =====
Trace 0: usr/sbin/rear:572 main
Trace 1: /root/rear.github.master/usr/share/rear/lib/framework-functions.sh:127 SourceStage
Trace 2: /root/rear.github.master/usr/share/rear/lib/framework-functions.sh:59 Source
Trace 3: /root/rear.github.master/usr/share/rear/init/default/030_update_recovery_system.sh:6 source
=== End stack trace ===
/root/rear.github.master/usr/share/rear/lib/_input-output-functions.sh: line 151: kill: (12459) - No such process

jsmeix commented at 2021-06-22 12:16:¶

I decided against copying all lines in STDOUT_STDERR_FILE
since the last sourced script into the log because some scripts
produce tons of stdout and stderr lines and having them all in the log
makes the log look like a debug log which was not requested by the user
because STDOUT_STDERR_FILE is only used in non-debug modes.

So I decided to copy only the extracted lines from STDOUT_STDERR_FILE
since the last sourced script into the log because then what there is in the log
matches what is shown to the user on his terminal.

I implemented that via
https://github.com/rear/rear/pull/2633/commits/43f05b6e75021ee1c1606a8ca2f4cf3ea3848442

With that I get in non-debug modes in the log
(with same artificial Error exit as above):

# usr/sbin/rear mkrescue
ERROR: QQQ
Some messages from /tmp/rear.AZSidfy3WIEsvsv/tmp/rear.mkrescue.stdout_stderr since the last called script 030_update_recovery_system.sh:
  drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
  drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
  grep: /etc/QQQ: No such file or directory
  grep: invalid option -- 'Q'
  Usage: grep [OPTION]... PATTERN [FILE]...
  Try 'grep --help' for more information.
Use debug mode '-d' for some debug messages or debugscript mode '-D' for full debug messages with 'set -x' output
Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details
Terminated

# cat var/log/rear/rear-linux-h9wr.log
2021-06-22 13:42:05.072738751 Relax-and-Recover 2.6 / Git
2021-06-22 13:42:05.073932349 Running rear mkrescue (PID 8254 date 2021-06-22 13:42:04)
2021-06-22 13:42:05.075112542 Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
2021-06-22 13:42:05.076597495 Including /root/rear.github.master/etc/rear/os.conf
2021-06-22 13:42:05.079926352 Including conf/Linux-i386.conf
2021-06-22 13:42:05.081362994 Including conf/GNU/Linux.conf
2021-06-22 13:42:16.271030637 Including conf/SUSE_LINUX.conf
2021-06-22 13:42:16.274296516 Including /root/rear.github.master/etc/rear/local.conf
2021-06-22 13:42:16.276295819 ======================
2021-06-22 13:42:16.277549486 Running 'init' stage
2021-06-22 13:42:16.278792302 ======================
2021-06-22 13:42:16.286907749 Including init/default/005_verify_os_conf.sh
2021-06-22 13:42:16.290938827 Including init/default/010_EFISTUB_check.sh
2021-06-22 13:42:16.295005689 Including init/default/010_set_drlm_env.sh
2021-06-22 13:42:16.299032949 Including init/default/030_update_recovery_system.sh
2021-06-22 13:42:16.320428658 Some messages from /tmp/rear.AZSidfy3WIEsvsv/tmp/rear.mkrescue.stdout_stderr since the last called script 030_update_recovery_system.sh:
  drwxr-xr-x 6 root root  4096 Jun 21 08:07 packaging
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 tests
  drwxr-xr-x 4 root root  4096 Jun 21 08:07 usr
  drwxr-xr-x 4 root root  4096 Jun 21 13:38 var
  grep: /etc/QQQ: No such file or directory
  grep: invalid option -- 'Q'
  Usage: grep [OPTION]... PATTERN [FILE]...
  Try 'grep --help' for more information.
2021-06-22 13:42:16.322149893 ERROR: QQQ
2021-06-22 13:42:16.330061856 Exiting rear mkrescue (PID 8254) and its descendant processes ...
2021-06-22 13:42:19.362841438 rear,8254 usr/sbin/rear mkrescue
  `-rear,8443 usr/sbin/rear mkrescue
      `-pstree,8444 -Aplau 8254
2021-06-22 13:42:19.383989639 Running exit tasks

jsmeix commented at 2021-06-23 09:06:¶

Now things work well for me as far as I currently see.

@rear/contributors
please have a look if time permits
and if there are no objections from you
I would like to merge it tomorrow afternoon.

jsmeix commented at 2021-06-23 12:35:¶

With https://github.com/rear/rear/pull/2633/commits/a7cd1eb583c7b88f23d01682a65f70e70f09d0b7
things behave now more obvious and therefore more fail-safe for the user
when KEEP_BUILD_DIR is true and there is something mounted within the build area.

In init/default/030_update_recovery_system.sh I added

mkdir $TMP_DIR/qqq1
mkdir $TMP_DIR/qqq2
mount /dev/sda6 $TMP_DIR/qqq1
mount /dev/sda6 $TMP_DIR/qqq2

so during "rear mkrescue" I have

# mount | grep "qqq"
/dev/sda6 on /tmp/rear.L6o6LK1zMOrbfvm/tmp/qqq1 type ext2 (rw,relatime)
/dev/sda6 on /tmp/rear.L6o6LK1zMOrbfvm/tmp/qqq2 type ext2 (rw,relatime)

and then rear -d mkrescue ends with

Exiting rear mkrescue (PID 18512) and its descendant processes ...
Running exit tasks
Caution - there is something mounted within the build area
  /dev/sda6 on /tmp/rear.L6o6LK1zMOrbfvm/tmp/qqq1 type ext2 (rw,relatime)
  /dev/sda6 on /tmp/rear.L6o6LK1zMOrbfvm/tmp/qqq2 type ext2 (rw,relatime)
You must manually umount that before you may remove the build area
To remove the build area use (with caution): rm -Rf --one-file-system /tmp/rear.L6o6LK1zMOrbfvm

Ignoring the manual umount is still safe with the shown 'rm' command:

# rm -Rf --one-file-system /tmp/rear.L6o6LK1zMOrbfvm
rm: skipping '/tmp/rear.L6o6LK1zMOrbfvm/tmp/qqq2', since it's on a different device
rm: skipping '/tmp/rear.L6o6LK1zMOrbfvm/tmp/qqq1', since it's on a different device

It removes all in the build area except the mounted things and their mountpoints.

jsmeix commented at 2021-06-23 13:10:¶

And https://github.com/rear/rear/pull/2633/commits/61646a64876a1244c7c0d237d9611da9499cb346
avoids that my test in
https://github.com/rear/rear/pull/2633#issuecomment-866797427
removes all on my /dev/sda6 with just rear -v mkrescue
(no worries - my /dev/sda6 is only for tests to play around).

jsmeix commented at 2021-06-24 10:02:¶

With https://github.com/rear/rear/pull/2633/commits/42f78dbb25e28afc5bbfb3309b7dfdd969109f17
and https://github.com/rear/rear/pull/2633/commits/7da72435380da7a48f2ef180f94b16540c19ec8e
and the following artificial test code at the beginning of init/default/030_update_recovery_system.sh

ls -l | Log
grep foo /etc/QQQ
mkdir $TMP_DIR/qqq1
mkdir $TMP_DIR/qqq2
mount /dev/sda6 $TMP_DIR/qqq1
mount /dev/sda6 $TMP_DIR/qqq2
grep -Q foo /etc/QQQ || PrintError "QQQ"

I get

# usr/sbin/rear mkrescue
QQQ
Failed to 'rm -Rf --one-file-system /tmp/rear.iCkYW6MvU8qvNDL/tmp'
Could not remove build area /tmp/rear.iCkYW6MvU8qvNDL (something still exists therein)
Something is still mounted within the build area
  /dev/sda6 on /tmp/rear.iCkYW6MvU8qvNDL/tmp/qqq1 type ext2 (rw,relatime)
  /dev/sda6 on /tmp/rear.iCkYW6MvU8qvNDL/tmp/qqq2 type ext2 (rw,relatime)
You must manually umount it, then you could manually remove the build area
To manually remove the build area use (with caution): rm -Rf --one-file-system /tmp/rear.iCkYW6MvU8qvNDL

and this log (excerpts)

2021-06-24 11:58:59.589284878 Relax-and-Recover 2.6 / Git
2021-06-24 11:58:59.592650685 Running rear mkrescue (PID 21119 date 2021-06-24 11:58:59)
2021-06-24 11:58:59.596200904 Using log file: /root/rear.github.master.jsmeix-stdout-stderr-non-debug-1/var/log/rear/rear-linux-h9wr.log
2021-06-24 11:58:59.600363374 Including /root/rear.github.master.jsmeix-stdout-stderr-non-debug-1/etc/rear/os.conf
2021-06-24 11:58:59.606397258 Including conf/Linux-i386.conf
2021-06-24 11:58:59.610722539 Including conf/GNU/Linux.conf
2021-06-24 11:59:10.861188522 Including conf/SUSE_LINUX.conf
2021-06-24 11:59:10.866099084 Including /root/rear.github.master.jsmeix-stdout-stderr-non-debug-1/etc/rear/local.conf
2021-06-24 11:59:10.869724425 ======================
2021-06-24 11:59:10.872729287 Running 'init' stage
2021-06-24 11:59:10.875594847 ======================
2021-06-24 11:59:10.884275346 Including init/default/005_verify_os_conf.sh
2021-06-24 11:59:10.890000399 Including init/default/010_EFISTUB_check.sh
2021-06-24 11:59:10.895663416 Including init/default/010_set_drlm_env.sh
2021-06-24 11:59:10.901338734 Including init/default/030_update_recovery_system.sh
2021-06-24 11:59:10.905017616 total 88
                              -rw-r--r-- 1 root root 35147 Jun 24 11:46 COPYING
                              -rw-r--r-- 1 root root   451 Jun 24 11:46 MAINTAINERS
                              -rw-r--r-- 1 root root 10762 Jun 24 11:46 Makefile
                              -rw-r--r-- 1 root root 10967 Jun 24 11:46 README.adoc
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 doc
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 etc
                              drwxr-xr-x 6 root root  4096 Jun 24 11:46 packaging
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 tests
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 usr
                              drwxr-xr-x 4 root root  4096 Jun 24 11:47 var
2021-06-24 11:59:10.930763644 Including init/default/050_check_rear_recover_mode.sh
.
.
.
2021-06-24 11:59:53.641265427 Finished running mkrescue workflow
2021-06-24 11:59:53.648397383 Exiting rear mkrescue (PID 21119) and its descendant processes ...
2021-06-24 11:59:56.859240431 rear,21119 usr/sbin/rear mkrescue
                                `-rear,10571 usr/sbin/rear mkrescue
                                    `-pstree,10572 -Aplau 21119
2021-06-24 11:59:58.022877345 Running exit tasks
2021-06-24 11:59:58.029241261 Finished rear mkrescue in 59 seconds
2021-06-24 11:59:58.033172997 Removing build area /tmp/rear.iCkYW6MvU8qvNDL
2021-06-24 11:59:58.448253364 Failed to 'rm -Rf --one-file-system /tmp/rear.iCkYW6MvU8qvNDL/tmp'
2021-06-24 11:59:58.581260923 Could not remove build area /tmp/rear.iCkYW6MvU8qvNDL (something still exists therein)
2021-06-24 11:59:58.589368114 Something is still mounted within the build area
2021-06-24 11:59:58.593637195   /dev/sda6 on /tmp/rear.iCkYW6MvU8qvNDL/tmp/qqq1 type ext2 (rw,relatime)
                                /dev/sda6 on /tmp/rear.iCkYW6MvU8qvNDL/tmp/qqq2 type ext2 (rw,relatime)
2021-06-24 11:59:58.598490664 You must manually umount it, then you could manually remove the build area
2021-06-24 11:59:58.603197902 To manually remove the build area use (with caution): rm -Rf --one-file-system /tmp/rear.iCkYW6MvU8qvNDL
2021-06-24 11:59:58.607662026 End of program 'rear' reached

versus in debug mode (excerpts)

# usr/sbin/rear -d mkrescue
Relax-and-Recover 2.6 / Git
Running rear mkrescue (PID 10725 date 2021-06-24 12:03:09)
Command line options: usr/sbin/rear -d mkrescue
Using log file: /root/rear.github.master.jsmeix-stdout-stderr-non-debug-1/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.9V5nWVoBHYyYmRi
QQQ
Running workflow mkrescue on the normal/original system
.
.
.
Running exit tasks
Caution - there is something mounted within the build area
  /dev/sda6 on /tmp/rear.9V5nWVoBHYyYmRi/tmp/qqq1 type ext2 (rw,relatime)
  /dev/sda6 on /tmp/rear.9V5nWVoBHYyYmRi/tmp/qqq2 type ext2 (rw,relatime)
You must manually umount that before you may remove the build area
To remove the build area use (with caution): rm -Rf --one-file-system /tmp/rear.9V5nWVoBHYyYmRi

and its log (excerpts)

...
2021-06-24 12:03:20.673298263 Including init/default/030_update_recovery_system.sh
2021-06-24 12:03:20.676899272 total 88
                              -rw-r--r-- 1 root root 35147 Jun 24 11:46 COPYING
                              -rw-r--r-- 1 root root   451 Jun 24 11:46 MAINTAINERS
                              -rw-r--r-- 1 root root 10762 Jun 24 11:46 Makefile
                              -rw-r--r-- 1 root root 10967 Jun 24 11:46 README.adoc
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 doc
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 etc
                              drwxr-xr-x 6 root root  4096 Jun 24 11:46 packaging
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 tests
                              drwxr-xr-x 4 root root  4096 Jun 24 11:46 usr
                              drwxr-xr-x 4 root root  4096 Jun 24 11:47 var
grep: /etc/QQQ: No such file or directory
grep: invalid option -- 'Q'
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
2021-06-24 12:03:20.704199281 Including init/default/050_check_rear_recover_mode.sh
.
.
.
2021-06-24 12:04:05.247836397 Finished running mkrescue workflow
2021-06-24 12:04:05.254992052 Exiting rear mkrescue (PID 10725) and its descendant processes ...
2021-06-24 12:04:08.295207847 rear,10725 usr/sbin/rear -d mkrescue
                                `-rear,32123 usr/sbin/rear -d mkrescue
                                    `-pstree,32124 -Aplau 10725
/root/rear.github.master.jsmeix-stdout-stderr-non-debug-1/usr/share/rear/lib/_input-output-functions.sh: line 151: kill: (32129) - No such process
2021-06-24 12:04:08.321663825 Running exit tasks
2021-06-24 12:04:08.326032604 Exit task 'cleanup_build_area_and_end_program'
2021-06-24 12:04:08.332642206 Finished rear mkrescue in 59 seconds
2021-06-24 12:04:08.340113223 Caution - there is something mounted within the build area
2021-06-24 12:04:08.344308085   /dev/sda6 on /tmp/rear.9V5nWVoBHYyYmRi/tmp/qqq1 type ext2 (rw,relatime)
                                /dev/sda6 on /tmp/rear.9V5nWVoBHYyYmRi/tmp/qqq2 type ext2 (rw,relatime)
2021-06-24 12:04:08.349201334 You must manually umount that before you may remove the build area
2021-06-24 12:04:08.353594126 To remove the build area use (with caution): rm -Rf --one-file-system /tmp/rear.9V5nWVoBHYyYmRi
2021-06-24 12:04:08.357820208 End of program 'rear' reached
2021-06-24 12:04:08.362030502 Exit task '(( EXIT_FAIL_MESSAGE )) && echo 'rear mkrescue failed, check /root/rear.github.master.jsmeix-stdout-stderr-no
2021-06-24 12:04:08.366721128 Exit task 'exec 8>&-'
2021-06-24 12:04:08.370969397 Exit task 'exec 7>&-'
2021-06-24 12:04:08.375235205 Exit task 'exec 6<&-'
2021-06-24 12:04:08.379536124 Exit task ''

jsmeix commented at 2021-06-24 10:11:¶

Because now all looks good to me
I would like to merge it soon
unless there are objections right now.
If bugs appear later I will of course fix them.

jsmeix commented at 2021-06-24 11:05:¶

I did https://github.com/rear/rear/pull/2633/commits/82c505e2cba23d31727bd552a5c1d8bc1ca91ff3
and https://github.com/rear/rear/pull/2633/commits/3dca74e59ab73487308c7b9aa6ebb50d7044e1d4
because an early Error() exit test in usr/sbin/rear

    # Create temporary working area:
    BUILD_DIR="$( mktemp -d -t rear.XXXXXXXXXXXXXXX || Error "Could not create build area '$BUILD_DIR'" )"
    ROOTFS_DIR=$BUILD_DIR/rootfs
    TMP_DIR=$BUILD_DIR/tmp
    mkdir -p $ROOTFS_DIR || Error "Could not create $ROOTFS_DIR"
    mkdir -p $TMP_DIR || Error "Could not create $TMP_DIR"
    Error "QQQQ"

revealed that Error() hung up because it calls grep with not yet set filenames
cf. "Beware of the emptiness" in https://github.com/rear/rear/wiki/Coding-Style
and cleanup_build_area_and_end_program() was not yet sourced and
cleanup_build_area_and_end_program() called not yet sourced functions.

jsmeix commented at 2021-06-24 11:50:¶

Phew!
What "yet another can of worms" that was.
Hopefully I didn't mess up something or introduce new bugs.

[Export of Github issue for rear/rear.]

#2633 PR merged: Have stdout and stderr also in non-debug modes¶