README': No such file or directory"¶

Labels: enhancement, support / question, fixed / solved / done

sandroLuk opened issue at 2018-11-15 09:53:¶

Hi there, i got the following issue. It's strange because on a similar system it works fine.

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.4 / Git
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):
OS_VENDOR=SUSE_LINUX
OS_VERSION=12
ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
OUTPUT=ISO
OUTPUT_URL=file:///opt/rear
GRUB_RESCUE=1
Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
VMWare VM
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86 Compativle VM
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
BIOS and GRUB
Storage (lokal disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
Virtual VMWare Disks
Description of the issue (ideally so that others can reproduce it):
rear -v mkrescue failes with the error:

 
Relax-and-Recover 2.4 / Git
Running rear mkrescue (PID 8202)
Using log file: /var/log/rear/rear-sgrtsrv460.log
Creating disk layout
Using sysconfig bootloader 'grub2'
Creating root filesystem layout
To log into the recovery system via ssh set up /root/.ssh/authorized_keys or specify SSH_ROOT_PASSWORD
Copying logfile /var/log/rear/rear-sgrtsrv460.log into initramfs as '/tmp/rear-sgrtsrv460-partial-2018-11-15T10:47:49+01:00.log'
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Copying all files in /lib*/firmware/
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (183796136 bytes) in 23 seconds
Copying resulting files to file location
Saving /var/log/rear/rear-sgrtsrv460.log as rear-sgrtsrv460.log to file location
/sgrtsrv460 at file location file /tmp/rear.aaja71UVdjBgWZt/tmp/README to /opt/rear
Some latest log messages since the last called script 950_copy_result_files.sh:
  2018-11-15 10:48:56.000324116 Including output/default/950_copy_result_files.sh
  2018-11-15 10:48:56.009472073 Copying resulting files to file location
  2018-11-15 10:48:56.053861074 Saving /var/log/rear/rear-sgrtsrv460.log as rear-sgrtsrv460.log to file location
  2018-11-15 10:48:56.058067224 Copying result files '/tmp/rear.aaja71UVdjBgWZt/tmp/VERSION /tmp/rear.aaja71UVdjBgWZt/tmp/README /tmp/rear.aaja71UVdjBgWZt/tmp/rear-sgrtsrv460.log' to /opt/r/sgrtsr
''/sgrtsrv460/VERSION'jBgWZt/tmp/VERSION' -> '/opt/rear'$'
  cp: cannot stat '/tmp/rear.aaja71UVdjBgWZt/tmp/README': No such file or directory
Aborting due to an error, check /var/log/rear/rear-sgrtsrv460.log for details
Exiting rear mkrescue (PID 8202) and its descendant processes
Running exit tasks
Terminated

Workaround, if any:
none
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
output file of rear -d -D mkrescue
rear-sgrtsrv460.log

abbbi commented at 2018-11-15 20:30:¶

there seems some strange with the rear path, eg, the logfile is full of control+M or ^M., for example:

++ [[ -z file:///opt/re**ar^M** ]]
cp -v /tmp/rear.bHyZIJCRlJSjtrL/tmp/README $'/opt/rear\r/sgrtsrv460/'

that seems pretty strange to me (note the \r in the path)

Probably your config file includes some special character in the:

OUTPUT_URL=file:///opt/rear

line?

sandroLuk commented at 2018-11-16 07:31:¶

Hi Michael, you were right, it was the solution! I've pasted the local.conf from my notepad++ and the default linebreaks were set to \r\n. I've changed them to \n. replaced the local.conf on the system and not it runs fine. Thank you!

gdha commented at 2018-11-16 07:44:¶

@jsmeix perhaps we should add a small scan on the local.conf file on these copy/paste things from windows (Ctrl-M) as it is not the first time these smaller annoying issues pop-up?

jsmeix commented at 2018-11-16 08:55:¶

@abbbi
many thanks for your analysis and for finding the root cause!

According to
https://github.com/rear/rear/issues/1965#issuecomment-439306377
this issue is solved.

@sandroLuk
FYI:
In ReaR also all config files are read and executed
as bash scripts via source cf. "Character encoding" in
https://github.com/rear/rear/wiki/Coding-Style
source` had failed

@gdha
I had a quick look how one could implement a (hopefully) simple sanity check
when all config files are read in sbin/rear in "Combining configuration files"
but I think this is not possible with reasonable effort because
on my openSUSE Leap 15.0 system which is like SLES12:

# unix2dos -n etc/rear/local.conf etc/rear/local.dos.conf
unix2dos: converting file etc/rear/local.conf to file etc/rear/local.dos.conf in DOS format...

# bash -n etc/rear/local.conf && echo OK || echo FAILED
OK

# bash -n etc/rear/local.dos.conf && echo OK || echo FAILED
OK

# source etc/rear/local.conf && echo OK || echo FAILED
OK

# source etc/rear/local.dos.conf && echo OK || echo FAILED
' is not a typo you can use command-not-found to lookup the package that contains it, like this:
    cnf 
FAILED

Accordingly simple bash -n is insufficient.
One must actually source the file to see if it is valid.

Actually source the file is done in sbin/rear at "Combining configuration files"
but in this case here source did not show an error because
https://github.com/rear/rear/files/2584647/rear-sgrtsrv460.log
reads:

2018-11-15 10:17:48.359177828 Including /etc/rear/local.conf
2018-11-15 10:17:48.360697084 Entering debugscripts mode via 'set -x'.
+ source /etc/rear/local.conf
++ OUTPUT=$'ISO\r'
++ OUTPUT_URL=$'file:///opt/rear\r'
++ GRUB_RESCUE=$'1\r'
+ test 1
+ Debug 'Leaving debugscripts mode (back to previous bash flags and options settings).'

But I wonder about the Source function in lib/framework-functions.sh
that currently blindly proceeds if source fails at

    # The actual work (source the source file):
    source "$source_file"

if ReaR should not better error out here when source fails
because I cannot imagine how things could further work at all
when already source had failed for a script or a config file,
cf. "Try to care about possible errors" in
https://github.com/rear/rear/wiki/Coding-Style

jsmeix commented at 2018-11-16 09:01:¶

@gdha
regarding wong values for config variables (as in this case here):

It needs to be checked individually in the code/scripts
that deal with a particular config variable if its value is valid.
E.g. for file names like test -e "$FILE_NAME_VARIABLE" || ...
(the double quotes are mandatory for this kind of test).

jsmeix commented at 2018-11-16 09:14:¶

@gdha
I found out why in current ReaR the Source function in lib/framework-functions.sh
must blindly proceed when source fails (i.e. when source returns with an
non-zero exit code) because in current ReaR scripts usually "blindly return"
the exit code of the last command and currently we do not care about
that exit code.

In current ReaR code things work pefectly o.k. when a script finished
intentionally with a last command that results non-zero exit code.

Also for config files that is possible and currently allowed like

FOO="$( ls qqqq )"

as last assignment in local.conf which results:

# source etc/rear/local.conf && echo OK || echo FAILED
ls: cannot access 'qqqq': No such file or directory
FAILED

I will add an explanatory comment to the Source function
in lib/framework-functions.sh why source therein must
blindly proceed if it returns with non-zero exit code.

jsmeix commented at 2018-11-16 09:31:¶

Regarding

in current ReaR scripts usually "blindly return" the exit code
of the last command and currently we do not care about
that exit code

see also
https://github.com/rear/rear/pull/1542
where a few of those cases are already fixed
(only those where a script explicitly calls return).

jsmeix commented at 2018-11-16 11:28:¶

Finally - I should have done that initially - I even had a look at
usr/share/rear/output/default/950_copy_result_files.sh
and I think things are perfectly right therein because
it errors out when things failed.

jsmeix commented at 2018-11-16 12:37:¶

FYI how it looks with
https://github.com/rear/rear/commit/6ed63d2a5ef5e18becd232c41f9f5ce21fd8d927

# usr/sbin/rear -D mkrescue
...
Making ISO image
Wrote ISO image: /root/rear.github.master/var/lib/rear/output/rear-g243.iso (111M)
Copying resulting files to nfs location
Saving /root/rear.github.master/var/log/rear/rear-g243.log as rear-g243.log to nfs location
Copying result files '/root/rear.github.master/var/lib/rear/output/rear-g243.iso /tmp/rear.cuLc8u5ZYGEOhDT/tmp/VERSION /tmp/rear.cuLc8u5ZYGEOhDT/tmp/README /tmp/rear.cuLc8u5ZYGEOhDT/tmp/rear-g243.log' to /tmp/rear.cuLc8u5ZYGEOhDT/outputfs/g243 at nfs location
Exiting rear mkrescue (PID 27440) and its descendant processes
Running exit tasks

gdha commented at 2018-11-16 15:07:¶

Converting a local.conf file into DOS format (extra Carriage Return character [^M or \r] after the linefeed):

# unix2dos -o local.conf

Running rear results in:

ERROR: Problem transferring result files to rsync://gdha@nas/volume1/homes/gdha/rsync^M

However, a plain source of the DOSified local.conf file will not show any errors:

# source local.conf && echo OK || echo FAIL
OK

An easy check to see whether it contains ^M is with the od utility (should be part of coreutils and standard available on all Linux flavours):

# od -x local.conf.good | grep -q 0d && echo Ctrl-M found || echo File is good
File is good
# od -x local.conf | grep -q 0d && echo Ctrl-M found || echo File is good
Ctrl-M found

jsmeix commented at 2018-11-16 15:39:¶

@gdha
thanks for your additional testing!

Interestingly your result differs from mine which is good
because now we see how difficult it is in practice
to develop a good and simple test.

You don't need the big od hammer ;-)
to check for a single byte value,
tr (which is already used at many places in ReaR)
is sufficient:

# test "$( echo -e "First\nSecond\nLast" | unix2dos | tr -d -c '\r' )" && echo CR found
CR found

Accordingly we could do in sbin/rear something like

for config in site local rescue ; do
    if test -r "$CONFIG_DIR/$config.conf" ; then
        # Delete all characters except '\r' and error out if the resulting string is not empty:
        test "$( tr -d -c '\r' < $CONFIG_DIR/$config.conf )" && Error "Carriage return character in $CONFIG_DIR/$config.conf (perhaps DOS or Mac format)"
        Source "$CONFIG_DIR/$config.conf" || true
    fi
done

and similar at the other places where config files are read.

Perhaps @schlomo may even know a plain bash-only method?

jsmeix commented at 2018-11-16 15:56:¶

How it fails for me with only the above change in sbin/rear
and unix2dos -o etc/rear/local.conf

# usr/sbin/rear -D mkrescue
Relax-and-Recover 2.4 / Git
Running rear mkrescue (PID 27341)
Using log file: /root/rear.github.master/var/log/rear/rear-g243.log
ERROR: Carriage return character in /root/rear.github.master/etc/rear/local.conf (perhaps DOS or Mac format)
Some latest log messages since the last called script :
  2018-11-16 16:50:02.114346295 Including conf/GNU/Linux.conf
  2018-11-16 16:50:02.115307754 Entering debugscripts mode via 'set -x'.
  2018-11-16 16:50:02.130256839 Leaving debugscripts mode (back to previous bash flags and options settings).
  2018-11-16 16:50:02.131711397 Including conf/SUSE_LINUX.conf
  2018-11-16 16:50:02.133636149 Entering debugscripts mode via 'set -x'.
  2018-11-16 16:50:02.137216886 Leaving debugscripts mode (back to previous bash flags and options settings).
  basename: missing operand
  Try 'basename --help' for more information.
Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-g243.log for details
Exiting rear mkrescue (PID 27341) and its descendant processes
Running exit tasks
Terminated

Note the misleading and useless latest log messages since the last called script
which I am fixing right now...

jsmeix commented at 2018-11-16 15:59:¶

How it fails for me with only the above change in sbin/rear
and unix2dos -o etc/rear/local.conf
plus https://github.com/rear/rear/commit/fca495ae01f8e119033bc26b2609ad878b72163a

# usr/sbin/rear -D mkrescue
Relax-and-Recover 2.4 / Git
Running rear mkrescue (PID 27551)
Using log file: /root/rear.github.master/var/log/rear/rear-g243.log
ERROR: Carriage return character in /root/rear.github.master/etc/rear/local.conf (perhaps DOS or Mac format)
Some latest log messages since the last called script /root/rear.github.master/usr/sbin/rear:

Aborting due to an error, check /root/rear.github.master/var/log/rear/rear-g243.log for details
Exiting rear mkrescue (PID 27551) and its descendant processes
Running exit tasks
Terminated

jsmeix commented at 2018-11-16 16:18:¶

The only place in ReaR where od is mentioned (but not used) is
in the comments of the function percent_encode() in lib/global-functions.sh
so that we should avoid to introduce using od in particular when it is
used in sbin/rear we need od mandatorily in the recovery system.

jsmeix commented at 2018-11-20 10:09:¶

With https://github.com/rear/rear/pull/1969 merged
we have now a test for CR characters in usual user config files
so that now ReaR errors out early (i.e. where the root cause is),
for an example how it looks like see
https://github.com/rear/rear/pull/1969#issue-231944295

[Export of Github issue for rear/rear.]

#1965 Issue closed: mkrescue error "cp: cannot stat '/tmp/rear.aaja71UVdjBgWZt/tmp/README': No such file or directory"¶