#1858 Issue closed: Error while running rear mkbackup on SUSE Linux on IBM POWER8¶
Labels: support / question, fixed / solved / done
langerkunal opened issue at 2018-07-12 10:28:¶
Relax-and-Recover (ReaR) Issue Template¶
Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):
-
ReaR version ("/usr/sbin/rear -V"): Relax-and-Recover 2.4 / 2018-06-21
-
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):
# cat etc/rear/os.conf OS_VENDOR=SUSE_LINUX OS_VERSION=12.3
- ReaR configuration files ("cat /etc/rear/site.conf" or "cat /etc/rear/local.conf"):
# cat etc/rear/local.conf OUTPUT=ISO OUTPUT_URL=10.23.172.99:/nimfs/Linux_OS_Backups # backup to iso uses the NETFS workflow BACKUP=NETFS BACKUP_URL=nfs://10.23.172.99/nimfs/Linux_OS_Backups/ # this could be a mounted USB disk on /backup TMPDIR=/mnt2
-
System architecture (x86 compatible or POWER and/or what kind of virtual machine):
IBM POWER8 -
Are you using BIOS or UEFI or another way to boot?
-
Brief description of the issue:
When I am trying to execute ./usr/sbin/rear -d -D mkbackup I am getting below error every time
Relax-and-Recover 2.4 / 2018-06-21 Using log file: /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log ERROR: Mount command 'mount -v -t nfs -o rw,noatime 10.23.172.99:/nimfs/Linux_OS_Backups/ /tmp/rear.z7aay5CVizxI5te/outputfs' failed. Aborting due to an error, check /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log for details Exiting rear mkbackup (PID 54978) and its descendant processes Running exit tasks You should also rm -Rf /tmp/rear.z7aay5CVizxI5te Terminated
Upon looking into the log file I see that the utility fails to create /tmp/rear.XXXXXXXXXX/outputfs.
2018-07-12 15:37:55.224922090 Including prep/NETFS/default/060_mount_NETFS_path.sh
2018-07-12 15:37:55.226082436 Entering debugscripts mode via 'set -x'.
+ source /home/rear/rear-2.4/usr/share/rear/prep/NETFS/default/060_mount_NETFS_path.sh
++ mkdir -p -v /tmp/rear.z7aay5CVizxI5te/outputfs
mkdir: created directory '/tmp/rear.z7aay5CVizxI5te/outputfs'
++ StopIfError 'Could not mkdir '\''/tmp/rear.z7aay5CVizxI5te/outputfs'\'''
++ (( 0 != 0 ))
++ AddExitTask 'rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'
++ EXIT_TASKS=("$*" "${EXIT_TASKS[@]}")
++ Debug 'Added '\''rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'\'' as an exit task'
++ test 1
++ Log 'Added '\''rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'\'' as an exit task'
+++ date '+%Y-%m-%d %H:%M:%S.%N '
++ local 'timestamp=2018-07-12 15:37:55.232776077 '
++ test 1 -gt 0
++ echo '2018-07-12 15:37:55.232776077 Added '\''rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2'\'' as an exit task'
2018-07-12 15:37:55.232776077 Added 'rmdir -v /tmp/rear.z7aay5CVizxI5te/outputfs >&2' as an exit task
++ [[ -n '' ]]
++ mount_url nfs://10.23.172.99/nimfs/Linux_OS_Backups/ /tmp/rear.z7aay5CVizxI5te/outputfs
++ local url=nfs://10.23.172.99/nimfs/Linux_OS_Backups/
Actually it creates the file but immediately deletes it. Need help to get this resolved.
- Work-around, if any: N/A
jsmeix commented at 2018-07-12 11:37:¶
@langerkunal
what you think is an error in the log file is not an error, that
StopIfError 'Could not mkdir '\''/tmp/rear.z7aay5CVizxI5te/outputfs'\'''
is only bash set -x output where the StopIfError function is
called
but that function tests the exit code of the last comand
(( 0 != 0 ))
and everything is o.k. - in particular because the last command
mkdir -p -v /tmp/rear.z7aay5CVizxI5te/outputfs
logs its success
mkdir: created directory '/tmp/rear.z7aay5CVizxI5te/outputfs'.
The actual error in your log file is later - inspect your log file near
its end
where it errors out and then go up to the command that actually failed
(this could be a lot of lines because the error out procedure results
many lines in the log file after the actual error happened).
I can reproduce your issue when I use an invalid OUTPUT_URL as you
do.
I used
OUTPUT=ISO BACKUP=NETFS BACKUP_OPTIONS="nfsvers=3,nolock" BACKUP_URL=nfs://10.160.4.244/nfs OUTPUT_URL=10.160.4.244/nfs
and got (excerpts):
# usr/sbin/rear -D mkbackup ... Created initrd.cgz with gzip default compression (61377717 bytes) in 8 seconds ERROR: Mount command 'mount -v -t 10.160.4.244/nfs -o rw,noatime 10.160.4.244:/nfs /tmp/rear.FIGGFraatqWevy5/outputfs' failed. Aborting due to an error, check /root/rear.github.jsmeix/var/log/rear/rear-f144.log for details ...
When OUTPUT_URL should point to the same thing as BACKUP_URL
you do not need to specify OUTPUT_URL see man rear that reads
If you do not specify OUTPUT_URL variable then by default it will be aligned to what was defined by variable BACKUP_URL. And, the rescue image will then be copied to the same location as your backup of the system disk(s).
When you use a SUSE system have a look at the template config files
usr/share/rear/conf/examples/SLE11-ext3-example.conf usr/share/rear/conf/examples/SLE12-btrfs-example.conf usr/share/rear/conf/examples/SLE12-SP1-btrfs-example.conf usr/share/rear/conf/examples/SLE12-SP2-btrfs-example.conf usr/share/rear/conf/examples/SLE11-SLE12-SAP-HANA-UEFI-example.conf
and start with one of them and adapt and enhance it
(possibly with excerpts from other template config files)
according to what you need.
In general I would recommend to have a look at
https://en.opensuse.org/SDB:Disaster_Recovery
for example therein the section about
"First steps with Relax-and-Recover"
langerkunal commented at 2018-07-12 11:55:¶
Hi Johannes, I did follow the steps that you mentioned. But even then it fails with below error despite me giving Full permissions to /tmp directory:
# ./usr/sbin/rear -d -D mkbackup Relax-and-Recover 2.4 / 2018-06-21 Using log file: /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log Using backup archive '/tmp/rear.Ty1YR4zyXFfQK9q/outputfs/DHS4DPRIM/backup.tar.gz' Creating disk layout Doing SLES12-SP1 (and later) btrfs subvolumes setup because the default subvolume path contains '@/.snapshots/' Using sysconfig bootloader 'grub2' Creating root filesystem layout Handling network interface 'eth0' eth0 is a physical device Handled network interface 'eth0' To log into the recovery system via ssh set up /root/.ssh/authorized_keys or specify SSH_ROOT_PASSWORD Copying logfile /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log into initramfs as '/tmp/rear-DHS4DPRIM-partial-2018-07-12T17:18:18+05:30.log' Copying files and directories Copying binaries and libraries Copying kernel modules Omit copying files in /lib*/firmware/ (FIRMWARE_FILES='no') Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression Created initrd.cgz with gzip default compression (75124123 bytes) in 12 seconds **ERROR: Could not mkdir '/tmp/rear.Ty1YR4zyXFfQK9q/outputfs/DHS4DPRIM' Aborting due to an error, check /home/rear/rear-2.4/var/log/rear/rear-DHS4DPRIM.log for details** Exiting rear mkbackup (PID 32471) and its descendant processes Running exit tasks You should also rm -Rf /tmp/rear.Ty1YR4zyXFfQK9q Terminated DHS4DPRIM:/home/rear/rear-2.4 #
langerkunal commented at 2018-07-12 12:08:¶
When I am trying to create directory under /tmp/rear.XXXXXXXX namely outputfs and outputfs/DHS4DPRIM (which is my hostname) through command line it worked fine
# mkdir -p -v -m0750 outputfs/DHS4DPRIM mkdir: created directory 'outputfs' mkdir: created directory 'outputfs/DHS4DPRIM' DHS4DPRIM:/tmp/rear.UqsoNS5jR2YRBKx #
But from the utility it is experiencing permission denied error:
**++ '[' -z /tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM ']' ++ mkdir -p -v -m0750 /tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM mkdir: cannot create directory '/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM': Permission denied ++ StopIfError 'Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\''' ++ (( 1 != 0 )) ++ Error 'Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\''' ++ LogPrintError 'ERROR: Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\''' ++ Log 'ERROR: Could not mkdir '\''/tmp/rear.UqsoNS5jR2YRBKx/outputfs/DHS4DPRIM'\''' +++ date '+%Y-%m-%d %H:%M:%S.%N '**
schabrolles commented at 2018-07-12 13:40:¶
@langerkunal, could you test again without TMPDIR=/mnt2 ?
langerkunal commented at 2018-07-12 13:57:¶
Already tried without TMPDIR as well. But same error coming.
jsmeix commented at 2018-07-12 14:37:¶
@langerkunal
when a program that runs as root (here usr/sbin/rear) gets
Permission denied
the root cause is likely not in that program but outside and probably it
is
some additional security stuff like the Linux kernel security modules
AppArmor or SELinux that prohibit even root to do what root can normally
do.
On the other hand I cannot explain why usr/sbin/rear cannot do
something as root
when the same works as root on the command line.
For me things "just work" on SLES12 so that there is very likely no
gereral problem in ReaR
but I think there is something special on your system why things do not
work there.
langerkunal commented at 2018-07-17 11:48:¶
Any update on the solution or workaround for this?
schabrolles commented at 2018-07-17 12:12:¶
@langerkunal, I tried but cannot replicate the issue on my Power System
LPAR with SLES12sp3.
It working well on my side. I agree with @jsmeix, I think there is
something "special" on your system that prevents rear to create
directories and files under /tmp.
jsmeix commented at 2018-07-17 12:13:¶
@langerkunal
what do you expect from us?
We cannot imagine what happens on your particular system.
Our systems work for us.
So you would need to dig into your system and provide info to us
so that we could understand what is differnet on your system
compared to our systems.
langerkunal commented at 2018-07-17 13:09:¶
@schabrolles The problem here is that command line to create directory under /tmp works perfectly fine but when I execute REaR it throws error. I will try again on new SLES system and update if it works or throws error again
schabrolles commented at 2018-07-17 13:30:¶
@langerkunal, yes please... trying with a fresh SLES12 is the best option to move forward and understand what happened on your system.
jsmeix commented at 2018-07-19 11:01:¶
@langerkunal
thanks for your new separated issue
https://github.com/rear/rear/issues/1875
so that we can continue there.
Because the initial issue here (that was the invalid OUTPUT_URL)
is solved I close this issue here.
langerkunal commented at 2018-07-19 11:04:¶
SUSE linux issue still persists on freshly installed system
jsmeix commented at 2018-07-19 11:47:¶
@langerkunal
the crucial point is that the mkdir as root gets Permission denied
issue
happens both on SUSE and on Ubuntu which indicates the root cause
is not Linux distribution specific but some more general cause
(regardless that currently I have no good idea what it could be).
[Export of Github issue for rear/rear.]