#2667 Issue closed: 'rear recover' does not entirely remove build area (any more).

Labels: fixed / solved / done, minor bug

hpannenb opened issue at 2021-08-06 13:47:

  • ReaR version ("/usr/sbin/rear -V"): 2.6 / current git master

  • OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
    Centos 7

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

# ReaR - site.conf

BACKUP=NETFS
BACKUP_URL=iso://backup

BACKUP_PROG_EXCLUDE+=( '/var/tmp/*' '/var/backup/*' '/backup/*' )
OUTPUT=ISO
OUTPUT_URL=null

ISO_MAX_SIZE=2500

USE_STATIC_NETWORKING=yes
USE_RESOLV_CONF=no

USER_INPUT_TIMEOUT=30

SSH_ROOT_PASSWORD='root'
  • Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
    KVM guest
  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
    x86
  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
    GRUB
  • Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
    Local disk
  • Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT" or "lsblk" as makeshift):
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0              11:0    1  1.2G  0 rom  
vda             253:0    0   16G  0 disk 
|-vda1          253:1    0  500M  0 part /mnt/local/boot
`-vda2          253:2    0 15.5G  0 part 
  |-centos-swap 252:0    0  512M  0 lvm  
  `-centos-root 252:1    0   15G  0 lvm  /mnt/local
  • Description of the issue (ideally so that others can reproduce it):
    Compared to the official 2.6 version the current git master version's rear recover cannot entirely remove the build area. An orphaned outputfs remains causing the following warning
...
Running exit tasks
Could not remove build area /var/tmp/rear.K0wbKZyga3vmP3O (something still exists therein)
To manually remove the build area use (with caution): rm -Rf --one-file-system /var/tmp/rear.K0wbKZyga3vmP3O
...

  • Workaround, if any:
    Not really required since the build area is removed anyway since it exists in the rescue environment only.
    But it might be a nasty side effect (in future) and is a "change" in behaviour to the prior release version of ReaR.

  • Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

RESCUE centos7:/var/tmp/rear.K0wbKZyga3vmP3O # ls -laR
.:
total 0
drwx------ 3 root root 0 Aug  6 12:28 .
drwxr-xr-x 3 root root 0 Aug  6 12:28 ..
drwxr-xr-x 2 root root 0 Aug  6 12:25 outputfs

./outputfs:
total 0
drwxr-xr-x 2 root root 0 Aug  6 12:25 .
drwx------ 3 root root 0 Aug  6 12:28 ..

hpannenb commented at 2021-08-06 17:50:

I suppose it has something to do with @jsmeix major work (see also commit 619a3b3ff76780e327269f73407fc25467c7801b) upwards. Anyhow there is an empty outputfs not beeing removed thus the build area cannot be purged in the rescue environment. As mentioned it might be kind of a cosmetic issue rather than a real one or a works-as-designed point I lost track of.

pcahyna commented at 2021-08-06 18:06:

Thanks for the report. I think you mean my major work? It seems related indeed, but I haven't seen it in my testing (maybe it was my oversight, because it is a warning, not an error?)

hpannenb commented at 2021-08-06 18:35:

Sorry. Obviously these were your commits. At the very end of the `rear recover``this happens (reproducible) with the current git version.

patrikdahlsll commented at 2021-08-13 07:45:

We seem to experience the same issue, built a new version of Rear yesterday.
[user@host ~]$ /usr/sbin/rear -V Relax-and-Recover 2.6-git.4526.6589c8d.master.changed / 2021-08-06

Error
[user@host ~]$ sudo rear mkrescue /*** Stuff Deleted Here ***/ Could not remove build area /var/tmp/rear.KCmijgTOJjTqUbV (something still exists therein) To manually remove the build area use (with caution): rm -Rf --one-file-system /var/tmp/rear.KCmijgTOJjTqUbV

patrikdahlsll commented at 2021-08-17 13:57:

I did some more testing and I actually see a .lockfile left behind in /var/tmp/rear.POCVTK1YcYthBK9/outputfs/hostname
Not sure if this is the same issues as OP described?

/var/tmp after a few days

[user@hostname ~]$ ls -ltr /var/tmp/
total 0
drwx------. 3 root root 22 Aug 12 18:15 rear.KCmijgTOJjTqUbV
drwx------. 3 root root 22 Aug 12 20:02 rear.U90gz6n67xTWFgf
drwx------. 3 root root 22 Aug 13 20:02 rear.EQeMlFVe33gGJlP
drwx------. 3 root root 22 Aug 14 20:02 rear.yjAzqU9UUurUHO3
drwx------. 3 root root 22 Aug 15 20:02 rear.xty6UHH89BiuiSD
drwx------. 3 root root 22 Aug 16 20:03 rear.4qZcRXCwTiBB702

/usr/sbin/rear -v mkrescue

[user@hostname ~]$ sudo /usr/sbin/rear -v mkrescue
[sudo] password for user:
Relax-and-Recover 2.6-git.4526.6589c8d2.master.changed / 2021-08-06
Running rear mkrescue (PID 590092 date 2021-08-17 15:28:20)
Using log file: /var/log/rear/rear-hostname.log
Running workflow mkrescue on the normal/original system
Using autodetected kernel '/boot/vmlinuz-4.18.0-305.10.2.el8_4.x86_64' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
Using guessed bootloader 'GRUB' for 'rear recover' (found in first bytes on /dev/sda)
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /var/lib/rear/layout/disklayout.conf)
Creating recovery system root filesystem skeleton layout
Copying logfile /var/log/rear/rear-hostname.log into initramfs as '/tmp/rear-hostname-partial-2021-08-17T15:28:25+02:00.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/4.18.0-305.10.2.el8_4.x86_64 (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
Broken symlink '/usr/lib/modules/4.18.0-305.10.2.el8_4.x86_64/build' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/modules/4.18.0-305.10.2.el8_4.x86_64/source' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-240.8.1.el8_3.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-193.19.1.el8_2.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-193.28.1.el8_2.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-240.1.1.el8_3.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Testing that the recovery system in /var/tmp/rear.POCVTK1YcYthBK9/rootfs contains a usable system
Skipped ldd test for '/usr/lib64/nagios/plugins/check_selinux' (owner 'nrpe' not in TRUSTED_FILE_OWNERS)
Skipped ldd test for '/usr/lib64/nagios/plugins/check_multiprocs' (owner 'nrpe' not in TRUSTED_FILE_OWNERS)
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (544183325 bytes) in 58 seconds
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-hostname.iso (532M)
Exiting rear mkrescue (PID 590092) and its descendant processes ...
Running exit tasks
Could not remove build area /var/tmp/rear.POCVTK1YcYthBK9 (something still exists therein)
To manually remove the build area use (with caution): rm -Rf --one-file-system /var/tmp/rear.POCVTK1YcYthBK9

ls -latR /var/tmp/rear.POCVTK1YcYthBK9

[user@hostname ~]$ sudo ls -latR /var/tmp/rear.POCVTK1YcYthBK9
/var/tmp/rear.POCVTK1YcYthBK9:
total 4
drwxrwxrwt. 13 root root 4096 Aug 17 15:30 ..
drwx------.  3 root root   22 Aug 17 15:30 .
drwx------.  3 root root   30 Aug 17 15:30 outputfs

/var/tmp/rear.POCVTK1YcYthBK9/outputfs:
total 0
drwx------. 3 root root 22 Aug 17 15:30 ..
drwxr-x---. 2 root root 23 Aug 17 15:30 hostname
drwx------. 3 root root 30 Aug 17 15:30 .

/var/tmp/rear.POCVTK1YcYthBK9/outputfs/hostname:
total 0
drwxr-x---. 2 root root 23 Aug 17 15:30 .
-rw-------. 1 root root  0 Aug 17 15:30 .lockfile
drwx------. 3 root root 30 Aug 17 15:30 ..

gdha commented at 2021-08-17 15:42:

@pcahyna @jsmeix I was so bold to address this issue to you as the last changes were made by you on this area of cleanup or not to cleanup.

pcahyna commented at 2021-08-17 17:28:

@gdha sure, I think I am the right person, but unfortunately I am afraid I can look at the problem only next week, not earlier.

jsmeix commented at 2021-09-03 10:02:

@hpannenb
I can reproduce it.
After "rear -v recover" I have

RESCUE localhost:~ # ls -ld /var/tmp/rear*
drwx------ 3 root root 0 Sep  3 11:57 /var/tmp/rear.aYeRj4poiApK787

RESCUE localhost:~ # find /var/tmp/rear.aYeRj4poiApK787 -ls
26478 0 drwx------ 3 root root 0 Sep 3 11:57 /var/tmp/rear.aYeRj4poiApK787
26849 0 drwxr-xr-x 2 root root 0 Sep 3 11:54 /var/tmp/rear.aYeRj4poiApK787/outputfs

I did not notice it because I always run rear in debug mode (usually with -D)
and in debug mode the build area is kept anyway.

jsmeix commented at 2021-09-03 10:41:

Got it:
In lib/_input-output-functions.sh there is

remove_temporary_mountpoint '$BUILD_DIR/outputfs' || BugError "..."

which results (with special added set -x) in the log

++ remove_temporary_mountpoint '$BUILD_DIR/outputfs'
++ test -d '$BUILD_DIR/outputfs'

so $BUILD_DIR is not evaluated (and then $BUILD_DIR/outputfs is no directory).

jsmeix commented at 2021-09-03 11:06:

@hpannenb @patrikdahlsll
could you test whether or not my changes in
https://github.com/rear/rear/pull/2675/files
also makes things work for your use cases?

patrikdahlsll commented at 2021-09-06 10:07:

Same problem? I did the following, not a Git expert.

git clone -b jsmeix-fix-issue-2667 git://github.com/rear/rear.git

[user@hostname ~]$ rear --version
`Relax-and-Recover 2.6-git.4531.190cf104.jsmeixfixissue2667.changed / 2021-09-03`

Sorry for formating, output from
cat site.conf
rear mkrescue
ls -latr buildarea
rear -D mkrescue

[user@hostname ~]$ sudo cat /etc/rear/site.conf
OUTPUT=ISO
BACKUP=CDM
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/python3.6/site-packages:/usr/lib64/python3.6/site-packages/hawkey:/usr/lib64/bind9-export:/usr/lib64/eog:/usr/lib64/samba:/usr/lib64/firefox"


[user@hostname ~]$ sudo rear mkrescue
Broken symlink '/usr/lib/modules/4.18.0-305.10.2.el8_4.x86_64/build' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/modules/4.18.0-305.10.2.el8_4.x86_64/source' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-240.8.1.el8_3.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-193.19.1.el8_2.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-193.28.1.el8_2.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-240.1.1.el8_3.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Skipped ldd test for '/usr/lib64/nagios/plugins/check_selinux' (owner 'nrpe' not in TRUSTED_FILE_OWNERS)
Skipped ldd test for '/usr/lib64/nagios/plugins/check_multiprocs' (owner 'nrpe' not in TRUSTED_FILE_OWNERS)
ERROR:
====================
BUG in /usr/share/rear/lib/_input-output-functions.sh line 321:
'Directory /var/tmp/rear.tUSHcoVO9caZrAX/outputfs not empty, cannot remove'
--------------------
Please report this issue at https://github.com/rear/rear/issues
and include at least all related parts from /var/log/rear/rear-hostname.log
preferably the whole debug information via 'rear -D mkrescue'
====================
Some latest log messages since the last called script 980_umount_output_dir.sh:
  2021-09-06 11:28:33.539512164 Finished running mkrescue workflow
  2021-09-06 11:28:33.548450313 Exiting rear mkrescue (PID 1273774) and its descendant processes ...
  2021-09-06 11:28:36.578445245 rear,1273774 /sbin/rear mkrescue
                                  `-rear,1325360 /sbin/rear mkrescue
                                      `-pstree,1325361 -Aplau 1273774
  2021-09-06 11:28:36.604105283 Running exit tasks
  2021-09-06 11:28:36.611716539 Finished rear mkrescue in 133 seconds
  2021-09-06 11:28:36.615849223 Removing build area /var/tmp/rear.tUSHcoVO9caZrAX
Use debug mode '-d' for some debug messages or debugscript mode '-D' for full debug messages with 'set -x' output
Aborting due to an error, check /var/log/rear/rear-hostname.log for details
Terminated
[user@hostname ~]$


[user@hostname ~]$ sudo ls -latr /var/tmp/rear.tUSHcoVO9caZrAX/
total 4
drwx------.  3 root root   30 Sep  6 11:28 outputfs
drwx------.  3 root root   22 Sep  6 11:28 .
drwxrwxrwt. 34 root root 4096 Sep  6 11:28 ..

[user@hostname ~]$ sudo ls -latr /var/tmp/rear.tUSHcoVO9caZrAX/outputfs
total 0
drwx------. 3 root root 30 Sep  6 11:28 .
drwxr-x---. 2 root root 23 Sep  6 11:28 hostname
drwx------. 3 root root 22 Sep  6 11:28 ..

[user@hostname ~]$ sudo ls -latr /var/tmp/rear.tUSHcoVO9caZrAX/outputfs/hostname
total 0
drwx------. 3 root root 30 Sep  6 11:28 ..
-rw-------. 1 root root  0 Sep  6 11:28 .lockfile
drwxr-x---. 2 root root 23 Sep  6 11:28 .


[user@hostname ~]$ sudo rear -D mkrescue
Relax-and-Recover 2.6-git.4531.190cf104.jsmeixfixissue2667.changed / 2021-09-03
Running rear mkrescue (PID 1325570 date 2021-09-06 11:32:52)
Command line options: /sbin/rear -D mkrescue
Using log file: /var/log/rear/rear-hostname.log
Using build area: /var/tmp/rear.wPgkkP26fOdvCt1
Running workflow mkrescue on the normal/original system
Using '/bin/mkisofs' to create ISO filesystem images
Using autodetected kernel '/boot/vmlinuz-4.18.0-305.10.2.el8_4.x86_64' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Using guessed bootloader 'GRUB' for 'rear recover' (found in first bytes on /dev/sda)
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /var/lib/rear/layout/disklayout.conf)
Creating recovery system root filesystem skeleton layout
Handling network interface 'ens160'
ens160 is a physical device
Handled network interface 'ens160'
Included current keyboard mapping (via 'dumpkeys -f')
Included default US keyboard mapping /lib/kbd/keymaps/legacy/i386/qwerty/defkeymap.map.gz
Included other keyboard mappings in /lib/kbd/keymaps
Copying logfile /var/log/rear/rear-hostname.log into initramfs as '/tmp/rear-hostname-partial-2021-09-06T11:32:58+02:00.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/4.18.0-305.10.2.el8_4.x86_64 (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
Skip copying broken symlink '/etc/mtab' target '/proc/1353351/mounts' on /proc/ /sys/ /dev/ or /run/
Broken symlink '/usr/lib/modules/4.18.0-305.10.2.el8_4.x86_64/build' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/modules/4.18.0-305.10.2.el8_4.x86_64/source' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-240.8.1.el8_3.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-193.19.1.el8_2.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-193.28.1.el8_2.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/firmware/4.18.0-240.1.1.el8_3.x86_64/intel-ucode/06-8c-01' in recovery system because 'readlink' cannot determine its link target
Testing that the recovery system in /var/tmp/rear.wPgkkP26fOdvCt1/rootfs contains a usable system
Testing each binary with 'ldd' and look for 'not found' libraries within the recovery system
Skipped ldd test for '/usr/lib64/nagios/plugins/check_selinux' (owner 'nrpe' not in TRUSTED_FILE_OWNERS)
Skipped ldd test for '/usr/lib64/nagios/plugins/check_multiprocs' (owner 'nrpe' not in TRUSTED_FILE_OWNERS)
Testing that the existing programs in the PROGS array can be found as executable command within the recovery system
Testing that each program in the REQUIRED_PROGS array can be found as executable command within the recovery system
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (539068831 bytes) in 54 seconds
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-hostname.iso (527M)
Exiting rear mkrescue (PID 1325570) and its descendant processes ...
Running exit tasks
To remove the build area use (with caution): rm -Rf --one-file-system /var/tmp/rear.wPgkkP26fOdvCt1
[user@hostname ~]$

jsmeix commented at 2021-09-07 12:53:

@patrikdahlsll
your issue seems to be specific for BACKUP=CDM
because for me "rear mkrescue" and "rear mkbackup" work with BACKUP=NETFS.

I am not a Rubrik CDM Cloud Data Management user
so I cannot reproduce how it really behaves but I will try...

pcahyna commented at 2021-09-07 14:04:

Thank you @hpannenb for noticing the bug and @jsmeix for fixing it, sorry for having introduced the problem.
@patrikdahlsll I suspect there are two different issues. And looking at what you have provided, I suspect .lockfile may be the problem. Can you please provide /var/log/rear/rear-hostname.log that was produced by the failing run?

pcahyna commented at 2021-09-07 14:34:

@jsmeix I can reproduce the problem without CDM quite easily.

git clone -b jsmeix-fix-issue-2667 git://github.com/rear/rear.git
cd rear/
cat >> etc/rear/local.conf <<EOF
OUTPUT=ISO
#BACKUP=CDM
EOF
./usr/sbin/rear mkrescue
(...)
ERROR: 
====================
BUG in /root/rear/usr/share/rear/lib/_input-output-functions.sh line 321:
'Directory /var/tmp/rear.1z4h8eA73CUU9eb/outputfs not empty, cannot remove'
--------------------
Please report this issue at https://github.com/rear/rear/issues
and include at least all related parts from /root/rear/var/log/rear/....log
preferably the whole debug information via 'rear -D mkrescue'
====================
Some latest log messages since the last called script 980_umount_output_dir.sh:
  2021-09-07 10:16:56.484539609 Finished running mkrescue workflow
  2021-09-07 10:16:56.505907556 Exiting rear mkrescue (PID 24655) and its descendant processes ...
  2021-09-07 10:16:59.530845833 rear,24655 ./usr/sbin/rear mkrescue
                                  `-rear,43913 ./usr/sbin/rear mkrescue
                                      `-pstree,43914 -Aplau 24655
  2021-09-07 10:16:59.559714020 Running exit tasks
  2021-09-07 10:16:59.566259476 Finished rear mkrescue in 237 seconds
  2021-09-07 10:16:59.570458104 Removing build area /var/tmp/rear.1z4h8eA73CUU9eb
Use debug mode '-d' for some debug messages or debugscript mode '-D' for full debug messages with 'set -x' output
Aborting due to an error, check /root/rear/var/log/rear/....log for details
Terminated

I suspect the fact that both OUTPUT_URL and BACKUP_URL are unset is triggering the problem. echo OUTPUT_URL=null >> etc/rear/local.conf fixes it.
Now the question is, what is the intended semantics of OUTPUT_URL= (empty, which is the default) if BACKUP_URL= as well (otherwise OUTPUT_URL inherits the value of BACKUP_URL by default) and whether it is intended to be different from OUTPUT_URL=null.
Note that BACKUP_URL= (empty) is quite common if using external backup methods, because they usually have no use for BACKUP_URL.

jsmeix commented at 2021-09-07 14:38:

@pcahyna
no need to say sorry for that little annoyance bug.
You had cleaned up a possibly destructive mess of various scattered code
which is a major improvement towards more fail-safe behaviour of ReaR.
Again many thanks for all your work!

pcahyna commented at 2021-09-07 14:39:

To reproduce the problem in debug modes: adding KEEP_BUILD_DIR=no to the configuration file will do the trick. (By default, the vale is yes in debug modes, hiding the problem.)

jsmeix commented at 2021-09-07 14:41:

@pcahyna
thank you for your reproducer!
I could not reproduce it with

OUTPUT=ISO
BACKUP=CDM
BACKUP_URL=nfs://192.168.122.1/nfs

(I needed to skip prep/CDM/default/450_check_cdm_client.sh to not error out there).
I did not notice that both OUTPUT_URL and BACKUP_URL are unset in
https://github.com/rear/rear/issues/2667#issuecomment-913522802

pcahyna commented at 2021-09-07 14:41:

Can you please provide /var/log/rear/rear-hostname.log that was produced by the failing run?

actually, you don't have to, I can reproduce the problem now.

jsmeix commented at 2021-09-07 14:46:

Regarding whether or not OUTPUT_URL and/or BACKUP_URL must be set
I found in man rear:

An example to use TSM for backup and ISO for output
would be to add these lines to /etc/rear/local.conf
(no need to define a BACKUP_URL when
 using an external backup solution):

BACKUP=TSM
OUTPUT=ISO

so it seems there are cases that are intended to work
with both OUTPUT_URL and BACKUP_URL unset.

See also
https://github.com/rear/rear/issues/1532#issuecomment-336810460
and
https://github.com/rear/rear/blob/master/usr/share/rear/prep/default/040_check_backup_and_output_scheme.sh

According to
https://github.com/rear/rear/blob/master/doc/user-guide/16-Rubrik-CDM.adoc
it seems also BACKUP=CDM is meant to work
without OUTPUT_URL and/or BACKUP_URL.

patrikdahlsll commented at 2021-09-07 14:58:

Hi,
If I understand correctly you don't need any further logfiles from me atm?

I have verified that adding to OUTPUT_URL=null to site.conf solved the problem as suggested.
We have not used it with previous version of rear (2.5-something) with CDM, we just let it drop the iso in /var/lib/rear/output/.

Current site.conf

OUTPUT=ISO
BACKUP=CDM
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/python3.6/site-packages:/usr/lib64/python3.6/site-packages/hawkey:/usr/lib64/bind9-export:/usr/lib64/eog:/usr/lib64/samba:/usr/lib64/firefox"
OUTPUT_URL=null

jsmeix commented at 2021-09-07 15:02:

@patrikdahlsll
thank you for your prompt feedback and for your workaround!

We have to find out what makes things like plain

OUTPUT=ISO
BACKUP=CDM

no longer work which is a related but separated issue.

pcahyna commented at 2021-09-07 16:22:

so it seems there are cases that are intended to work
with both OUTPUT_URL and BACKUP_URL unset.

BACKUP_URL unset (empty) is for sure supported, because many backup methods do not make any use of it. One still needs some destination for the output image (OUTPUT_URL) though. When OUTPUT_URL=null, ReaR simply leaves the ISO image under /var/lib/rear and does not copy it anywhere, which may be a satisfactory solution in some cases. The question is, is OUTPUT_URL empty or unset intended to be the equivalent of the literal null? If so, one might simply set OUTPUT_URL=null, if otherwise unset, in default.conf or in some script. This will simplify the code which would not need to deal with the two cases: empty and null. See for example https://github.com/rear/rear/blob/f06e0d22eeb3bd45ad8ce0b8fce0b538e4534f93/usr/share/rear/prep/default/040_check_backup_and_output_scheme.sh#L69 which correctly errors out if one uses OUTPUT_URL=null with USB but does not complain in case of an empty OUTPUT_URL.
Or https://github.com/rear/rear/blob/f06e0d22eeb3bd45ad8ce0b8fce0b538e4534f93/usr/share/rear/output/default/250_create_lock.sh#L9 which needs to be updated to use quotes around $scheme because $scheme may be empty (which may well be the root cause of the problem here).

hpannenb commented at 2021-09-07 17:33:

@hpannenb @patrikdahlsll
could you test whether or not my changes in
https://github.com/rear/rear/pull/2675/files
also makes things work for your use cases?

Hi, @jsmeix.

With Your fix it worked for my use case. The mentioned error is not occurring any more and /var/tmp/rear.* has been removed.

Regards,
Holger.

jsmeix commented at 2021-09-08 11:57:

@pcahyna
thank you for pointing me into the right direction
that https://github.com/rear/rear/issues/2676
is related to scheme_supports_filesystem()

@hpannenb
thank you for verifying it.
It helps us a lot when also others verify things.

jsmeix commented at 2021-09-09 11:06:

With https://github.com/rear/rear/pull/2675 merged
this issue should be fixed.

patrikdahlsll commented at 2021-09-09 11:44:

Hi, do you want me to open a separate issue regarding our problem that build area is not removed if we had our old site.conf? Or shouls that problem be solved also?

jsmeix commented at 2021-09-09 13:29:

@patrikdahlsll
both issues should be solved
since https://github.com/rear/rear/pull/2675 is merged.

In particular a plain config like

OUTPUT=ISO
BACKUP=CDM

should again work.

But I cannot actually test BACKUP=CDM issues, cf.
https://github.com/rear/rear/issues/2667#issuecomment-914281285

So I would appreciate feedback whether or not
the current GitHub ReaR master code that contains in particular
https://github.com/rear/rear/commit/675eed16a5a048ace4cfdfadaa2b65c211604d88
again works with a plain config like

OUTPUT=ISO
BACKUP=CDM

patrikdahlsll commented at 2021-09-09 13:55:

Thanks, I will give it a try today or tomorrow.

patrikdahlsll commented at 2021-09-19 10:11:

Hi,

Forgot to inform you that your fixes seem to work perfect.
We tested the new version last weekend.

jsmeix commented at 2021-09-20 07:21:

@patrikdahlsll
thank you for your feedback!
Even a bit late feedback helps us because the more feedback the better.

[Export of Github issue for rear/rear.]