#1858 Issue closed: Error while running rear mkbackup on SUSE Linux on IBM POWER8

Labels: support / question, fixed / solved / done

langerkunal opened issue at 2018-07-12 10:28:

Relax-and-Recover (ReaR) Issue Template

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

  • ReaR version ("/usr/sbin/rear -V"): Relax-and-Recover 2.4 / 2018-06-21

  • OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):

# cat etc/rear/os.conf
  • ReaR configuration files ("cat /etc/rear/site.conf" or "cat /etc/rear/local.conf"):
# cat etc/rear/local.conf
# backup to iso uses the NETFS workflow
BACKUP_URL=nfs://   # this could  be a mounted USB disk on /backup
  • System architecture (x86 compatible or POWER and/or what kind of virtual machine):

  • Are you using BIOS or UEFI or another way to boot?

  • Brief description of the issue:
    When I am trying to execute ./usr/sbin/rear -d -D mkbackup I am getting below error every time

Relax-and-Recover 2.4 / 2018-06-21
Using log file: /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log
ERROR: Mount command 'mount -v -t nfs -o rw,noatime /tmp/rear.z7aay5CVizxI5te/outputfs' failed.
Aborting due to an error, check /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log for details
Exiting rear mkbackup (PID 54978) and its descendant processes
Running exit tasks
You should also rm -Rf /tmp/rear.z7aay5CVizxI5te

Upon looking into the log file I see that the utility fails to create /tmp/rear.XXXXXXXXXX/outputfs.

2018-07-12 15:37:55.224922090 Including prep/NETFS/default/060_mount_NETFS_path.sh
2018-07-12 15:37:55.226082436 Entering debugscripts mode via 'set -x'.
+ source /home/rear/rear-2.4/usr/share/rear/prep/NETFS/default/060_mount_NETFS_path.sh
++ mkdir -p -v /tmp/rear.z7aay5CVizxI5te/outputfs
mkdir: created directory '/tmp/rear.z7aay5CVizxI5te/outputfs'
++ StopIfError 'Could not mkdir '\''/tmp/rear.z7aay5CVizxI5te/outputfs'\'''
++ ((  0 != 0  ))
++ AddExitTask 'rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'
++ EXIT_TASKS=("$*" "${EXIT_TASKS[@]}")
++ Debug 'Added '\''rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'\'' as an exit task'
++ test 1
++ Log 'Added '\''rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'\'' as an exit task'
+++ date '+%Y-%m-%d %H:%M:%S.%N '
++ local 'timestamp=2018-07-12 15:37:55.232776077 '
++ test 1 -gt 0
++ echo '2018-07-12 15:37:55.232776077 Added '\''rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'\'' as an exit task'
2018-07-12 15:37:55.232776077 Added 'rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2' as an exit task
++ [[ -n '' ]]
++ mount_url nfs:// /tmp/rear.z7aay5CVizxI5te/outputfs
++ local url=nfs://

Actually it creates the file but immediately deletes it. Need help to get this resolved.

  • Work-around, if any: N/A

jsmeix commented at 2018-07-12 11:37:

what you think is an error in the log file is not an error, that
StopIfError 'Could not mkdir '\''/tmp/rear.z7aay5CVizxI5te/outputfs'\'''
is only bash set -x output where the StopIfError function is called
but that function tests the exit code of the last comand (( 0 != 0 ))
and everything is o.k. - in particular because the last command
mkdir -p -v /tmp/rear.z7aay5CVizxI5te/outputfs
logs its success mkdir: created directory '/tmp/rear.z7aay5CVizxI5te/outputfs'.

The actual error in your log file is later - inspect your log file near its end
where it errors out and then go up to the command that actually failed
(this could be a lot of lines because the error out procedure results
many lines in the log file after the actual error happened).

I can reproduce your issue when I use an invalid OUTPUT_URL as you do.
I used


and got (excerpts):

# usr/sbin/rear -D mkbackup
Created initrd.cgz with gzip default compression (61377717 bytes) in 8 seconds
ERROR: Mount command 'mount -v -t -o rw,noatime /tmp/rear.FIGGFraatqWevy5/outputfs' failed.
Aborting due to an error, check /root/rear.github.jsmeix/var/log/rear/rear-f144.log for details

When OUTPUT_URL should point to the same thing as BACKUP_URL
you do not need to specify OUTPUT_URL see man rear that reads

If you do not specify OUTPUT_URL variable then by default
it will be aligned to what was defined by variable BACKUP_URL.
And, the rescue image will then be copied to the same location
as your backup of the system disk(s).

When you use a SUSE system have a look at the template config files


and start with one of them and adapt and enhance it
(possibly with excerpts from other template config files)
according to what you need.

In general I would recommend to have a look at
for example therein the section about
"First steps with Relax-and-Recover"

langerkunal commented at 2018-07-12 11:55:

Hi Johannes, I did follow the steps that you mentioned. But even then it fails with below error despite me giving Full permissions to /tmp directory:

# ./usr/sbin/rear -d -D mkbackup
Relax-and-Recover 2.4 / 2018-06-21
Using log file: /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log
Using backup archive '/tmp/rear.Ty1YR4zyXFfQK9q/outputfs/DHS4DPRIM/backup.tar.gz'
Creating disk layout
Doing SLES12-SP1 (and later) btrfs subvolumes setup because the default subvolume path contains '@/.snapshots/'
Using sysconfig bootloader 'grub2'
Creating root filesystem layout
Handling network interface 'eth0'
eth0 is a physical device
Handled network interface 'eth0'
To log into the recovery system via ssh set up /root/.ssh/authorized_keys or specify SSH_ROOT_PASSWORD
Copying logfile /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log into initramfs as '/tmp/rear-DHS4DPRIM-partial-2018-07-12T17:18:18+05:30.log'
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Omit copying files in /lib*/firmware/ (FIRMWARE_FILES='no')
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (75124123 bytes) in 12 seconds
**ERROR: Could not mkdir '/tmp/rear.Ty1YR4zyXFfQK9q/outputfs/DHS4DPRIM'
Aborting due to an error, check /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log for details**
Exiting rear mkbackup (PID 32471) and its descendant processes
Running exit tasks
You should also rm -Rf /tmp/rear.Ty1YR4zyXFfQK9q
DHS4DPRIM:/home/rear/rear-2.4 #

langerkunal commented at 2018-07-12 12:08:

When I am trying to create directory under /tmp/rear.XXXXXXXX namely outputfs and outputfs/DHS4DPRIM (which is my hostname) through command line it worked fine

 # mkdir -p -v -m0750 outputfs/DHS4DPRIM
mkdir: created directory 'outputfs'
mkdir: created directory 'outputfs/DHS4DPRIM'
DHS4DPRIM:/tmp/rear.UqsoNS5jR2YRBKx #

But from the utility it is experiencing permission denied error:

**++ '[' -z /tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM ']'
++ mkdir -p -v -m0750 /tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM
mkdir: cannot create directory '/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM': Permission denied
++ StopIfError 'Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\'''
++ ((  1 != 0  ))
++ Error 'Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\'''
++ LogPrintError 'ERROR: Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\'''
++ Log 'ERROR: Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\'''
+++ date '+%Y-%m-%d %H:%M:%S.%N '**

schabrolles commented at 2018-07-12 13:40:

@langerkunal, could you test again without TMPDIR=/mnt2 ?

langerkunal commented at 2018-07-12 13:57:

Already tried without TMPDIR as well. But same error coming.

jsmeix commented at 2018-07-12 14:37:

when a program that runs as root (here usr/sbin/rear) gets Permission denied
the root cause is likely not in that program but outside and probably it is
some additional security stuff like the Linux kernel security modules
AppArmor or SELinux that prohibit even root to do what root can normally do.
On the other hand I cannot explain why usr/sbin/rear cannot do something as root
when the same works as root on the command line.

For me things "just work" on SLES12 so that there is very likely no gereral problem in ReaR
but I think there is something special on your system why things do not work there.

langerkunal commented at 2018-07-17 11:48:

Any update on the solution or workaround for this?

schabrolles commented at 2018-07-17 12:12:

@langerkunal, I tried but cannot replicate the issue on my Power System LPAR with SLES12sp3.
It working well on my side. I agree with @jsmeix, I think there is something "special" on your system that prevents rear to create directories and files under /tmp.

jsmeix commented at 2018-07-17 12:13:

what do you expect from us?
We cannot imagine what happens on your particular system.
Our systems work for us.
So you would need to dig into your system and provide info to us
so that we could understand what is differnet on your system
compared to our systems.

langerkunal commented at 2018-07-17 13:09:

@schabrolles The problem here is that command line to create directory under /tmp works perfectly fine but when I execute REaR it throws error. I will try again on new SLES system and update if it works or throws error again

schabrolles commented at 2018-07-17 13:30:

@langerkunal, yes please... trying with a fresh SLES12 is the best option to move forward and understand what happened on your system.

jsmeix commented at 2018-07-19 11:01:

thanks for your new separated issue
so that we can continue there.

Because the initial issue here (that was the invalid OUTPUT_URL)
is solved I close this issue here.

langerkunal commented at 2018-07-19 11:04:

SUSE linux issue still persists on freshly installed system

jsmeix commented at 2018-07-19 11:47:

the crucial point is that the mkdir as root gets Permission denied issue
happens both on SUSE and on Ubuntu which indicates the root cause
is not Linux distribution specific but some more general cause
(regardless that currently I have no good idea what it could be).

[Export of Github issue for rear/rear.]