#1914 PR merged: Issue #1913 - NETFS+tar backup exit code testing insufficient (may fail to catch real backup errors)¶
Labels: enhancement, bug, fixed / solved / done
rmetrich opened issue at 2018-09-20 09:45:¶
The ReaR code is making the assumption that the $BACKUP_PROG return value is returned from the child process. This is true for rsync and unsupported methods, but not for tar because tar uses the following pipes:
tar ... | <encrypt> ... | <splitter>
which ends up for basic case with the following pipes:
tar ... | cat | dd of=backup.tar.gz
In fact, the return code is hence the one for dd.
When there is an error, e.g. ENOSPC, dd returns 1, causing ReaR to
believe tar returned 1, which is non-critical (1 means there was
file modifications while reading them).
Also, if tar suffered an issue, e.g. an invalid parameter was passed
to tar, then dd would return 0, causing a fake success to be
returned.
This code fixes all these issues by returning the exit code of the last
pipe failing (can be dd, or cat, or tar).
It also enhances the exit code processing code, by displaying the
command's output if possible.
TEST CASES¶
SUCCESS¶
Config¶
# cat etc/rear/site.conf
OUTPUT=ISO
OUTPUT_URL=file:///backup
BACKUP=NETFS
BACKUP_URL=file:///backup
ONLY_INCLUDE_VG=( 'rhel' )
BACKUP_PROG_INCLUDE=( '/usr/bin' )
BACKUP_ONLY_INCLUDE="yes"
Result¶
# ./usr/sbin/rear -v mkbackuponly
Relax-and-Recover 2.4 / Git
Running rear mkbackuponly (PID 339)
Using log file: /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.339.log
Using backup archive '/backup/vm-rhel7/backup.tar.gz'
Creating disk layout
Excluding Volume Group data
Excluding component fs:/backup
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda with GPT BIOS boot partition)
Creating tar archive '/backup/vm-rhel7/backup.tar.gz'
Archived 25 MiB [avg 4397 KiB/sec] OK
Archived 25 MiB in 7 seconds [avg 3769 KiB/sec]
Saving /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.339.log as /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.log
Exiting rear mkbackuponly (PID 339) and its descendant processes
Running exit tasks
SUCCESS WITH WARNING¶
Config¶
# cat etc/rear/site.conf
OUTPUT=ISO
OUTPUT_URL=file:///backup
BACKUP=NETFS
BACKUP_URL=file:///backup
ONLY_INCLUDE_VG=( 'rhel' )
BACKUP_PROG_INCLUDE=( '/usr/bin' "/proc/*" )
BACKUP_ONLY_INCLUDE="yes"
Result¶
# ./usr/sbin/rear -v mkbackuponly
Relax-and-Recover 2.4 / Git
Running rear mkbackuponly (PID 29790)
Using log file: /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.29790.log
Using backup archive '/backup/vm-rhel7/backup.tar.gz'
Creating disk layout
Excluding Volume Group data
Excluding component fs:/backup
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda with GPT BIOS boot partition)
Creating tar archive '/backup/vm-rhel7/backup.tar.gz'
Archived 31 MiB [avg 6377 KiB/sec] OK
WARNING: tar ended with return code 1 and below output:
---snip---
tar: /proc/sys/net/ipv6/route/flush: Warning: Cannot open: Permission denied
tar: /proc/sys/vm/compact_memory: Warning: Cannot open: Permission denied
tar: /proc/tty/driver/serial: file changed as we read it
----------
This means that files have been modified during the archiving
process. As a result the backup may not be completely consistent
or may not be a perfect copy of the system. Relax-and-Recover
will continue, however it is highly advisable to verify the
backup in order to be sure to safely recover this system.
Archived 31 MiB in 6 seconds [avg 5314 KiB/sec]
Saving /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.29790.log as /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.log
Exiting rear mkbackuponly (PID 29790) and its descendant processes
Running exit tasks
No space on device¶
Config¶
# cat etc/rear/site.conf
OUTPUT=ISO
OUTPUT_URL=file:///backup
BACKUP=NETFS
BACKUP_URL=file:///backup
ONLY_INCLUDE_VG=( 'rhel' )
/backup configured too small to host the backup.
Result¶
# ./usr/sbin/rear -v mkbackuponly
Relax-and-Recover 2.4 / Git
Running rear mkbackuponly (PID 4267)
Using log file: /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.4267.log
Using backup archive '/backup/vm-rhel7/backup.tar.gz'
Creating disk layout
Excluding Volume Group data
Excluding component fs:/backup
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda with GPT BIOS boot partition)
Creating tar archive '/backup/vm-rhel7/backup.tar.gz'
Archived 756 MiB [avg 10754 KiB/sec] OK
ERROR: dd failed with return code 1 and below output:
---snip---
dd: writing to '/backup/vm-rhel7/backup.tar.gz': No space left on device
----------
This means that the archiving process ended prematurely, or did
not even start. As a result it is unlikely you can recover this
system properly. Relax-and-Recover is therefore aborting execution.
Some latest log messages since the last called script 500_make_backup.sh:
...
Aborting due to an error, check /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.4267.log for details
Exiting rear mkbackuponly (PID 4267) and its descendant processes
Running exit tasks
Terminated
TAR failing due to invalid option¶
Config¶
# cat etc/rear/site.conf
OUTPUT=ISO
OUTPUT_URL=file:///backup
BACKUP=NETFS
BACKUP_URL=file:///backup
ONLY_INCLUDE_VG=( 'rhel' )
BACKUP_PROG_OPTIONS=( "${BACKUP_PROG_OPTIONS[@]}" --dummyoption )
Result¶
# ./usr/sbin/rear -v mkbackuponly
Relax-and-Recover 2.4 / Git
Running rear mkbackuponly (PID 6300)
Using log file: /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.6300.log
Using backup archive '/backup/vm-rhel7/backup.tar.gz'
Creating disk layout
Excluding Volume Group data
Excluding component fs:/backup
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda with GPT BIOS boot partition)
Creating tar archive '/backup/vm-rhel7/backup.tar.gz'
Preparing archive operationOK
ERROR: tar failed with return code 64 and below output:
---snip---
tar: unrecognized option '--dummyoption'
----------
This means that the archiving process ended prematurely, or did
not even start. As a result it is unlikely you can recover this
system properly. Relax-and-Recover is therefore aborting execution.
Some latest log messages since the last called script 500_make_backup.sh:
...
Aborting due to an error, check /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.6300.log for details
Exiting rear mkbackuponly (PID 6300) and its descendant processes
Running exit tasks
Terminated
2nd pipe failing (encryption, simulated here through code hacking)¶
Config¶
# cat etc/rear/site.conf
OUTPUT=ISO
OUTPUT_URL=file:///backup
BACKUP=NETFS
BACKUP_URL=file:///backup
ONLY_INCLUDE_VG=( 'rhel' )
In /usr/share/rear/backup/NETFS/default/500_make_backup.sh, modify
line 48 as shown below:
BACKUP_PROG_CRYPT_OPTIONS="cat --dummyoption"
Result¶
# ./usr/sbin/rear -v mkbackuponly
Relax-and-Recover 2.4 / Git
Running rear mkbackuponly (PID 8019)
Using log file: /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.8019.log
Using backup archive '/backup/vm-rhel7/backup.tar.gz'
Creating disk layout
Excluding Volume Group data
Excluding component fs:/backup
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda with GPT BIOS boot partition)
Creating tar archive '/backup/vm-rhel7/backup.tar.gz'
Preparing archive operationOK
ERROR: cat failed with return code 1 and below output:
---snip---
cat: unrecognized option '--dummyoption'
----------
This means that the archiving process ended prematurely, or did
not even start. As a result it is unlikely you can recover this
system properly. Relax-and-Recover is therefore aborting execution.
Some latest log messages since the last called script 500_make_backup.sh:
...
Aborting due to an error, check /home/rmetrich/GIT/rear/var/log/rear/rear-vm-rhel7.8019.log for details
Exiting rear mkbackuponly (PID 8019) and its descendant processes
Running exit tasks
Terminated
jsmeix commented at 2018-09-20 10:01:¶
@rmetrich
I think this pull request contains unrelated changes, see
https://github.com/rear/rear/pull/1914/files
where curently 8 files would be changed.
rmetrich commented at 2018-09-20 10:05:¶
Ouch! need to resync somehow ...
jsmeix commented at 2018-09-20 12:45:¶
@gdha
could you also have a look and review it?
jsmeix commented at 2018-09-21 12:18:¶
@rmetrich
many thanks for your fix and enhancement!
[Export of Github issue for rear/rear.]