#3069 Issue open: BACKUP=NSR: "rear checklayout" always results exit code 1

Labels: bug, waiting for info

jsmeix opened issue at 2023-11-07 10:27:

  • ReaR version:
    2.7

  • OS version:
    SLES15 SP3

  • ReaR configuration file:
    local.conf excerpt for what is needed here

OUTPUT=ISO
BACKUP=NSR

I ( @jsmeix ) would like to report an issue with BACKUP=NSR
that a SUSE customer reported to us.

Regarding BACKUP=NSR:
In general there is nothing what I could do
in case of issues with third-party backup tools
because I do not have such software
so I can neither test nor reproduce anything.

A SUSE customer is using ReaR version 2.7

The customer got the following issue:

With BACKUP=NSR "rear checklayout" always results exit code 1
even if nothing of the disk layout has changed.
So "rear checklayout" would create a new ISO every day
even if this is not necessary.

The customer found
https://github.com/rear/rear/issues/2162
that tells (excerpts):

So IMHO removing the line
NSRSERVER=$(cat $VAR_DIR/recovery/nsr_server )
from the script 650_check_iso_recoverable.sh in the existing code
will not harm the execution of the NSR workflow

and

Log "NSRSERVER ($NSRSERVER) was defined in $CONFIG_DIR/local.conf"

The customer does not want to set a fixed NSRSERVER in local.conf
because NSRSERVER is dynamically determined via "rear mkrescue"
and stored in /var/lib/rear/recovery/nsr_server,
see rear/rescue/NSR/default/460_save_nsr_server_name.sh

In
https://github.com/rear/rear/issues/2162
there were 2 proposed workarounds:

Proposed workaround(s):
In 650_check_iso_recoverable.sh comment in the line as follows
#NSRSERVER=$(cat $VAR_DIR/recovery/nsr_server )
or
use the more sofisticated code snippet taken from 400_verify_nsr.sh instead:
[...]

if [[ ! -z "$NSRSERVER" ]]; then
    Log "NSRSERVER ($NSRSERVER) was defined in $CONFIG_DIR/local.conf"
elif [[ -f $VAR_DIR/recovery/nsr_server ]]; then
    NSRSERVER=$( cat $VAR_DIR/recovery/nsr_server )
else
    Error "Could not retrieve the EMC NetWorker Server name. Define NSRSERVER in $CONFIG_DIR/local.conf file"
fi

[...]

The first workaround was implemented but it causes that now
with BACKUP=NSR "rear checklayout" always results exit code 1
even if nothing of the disk layout has changed.

The customer tried the second workaround which works as expected.
In particular with the second workaround it works again as before
https://github.com/rear/rear/issues/2162

Furthermore the customer just linked it

usr/share/rear/layout/save/NSR/default # ls -l
total 4
lrwxrwxrwx 1 ... 400_verify_nsr.sh -> ../../../../verify/NSR/default/400_verify_nsr.sh
-rw-r--r-- 1 ... 650_check_iso_recoverable.sh

which also works for the customer
so this could be a third possible workaround.

So the questions are:

Is the customer right, that the second workaround should be used?
If yes, could ReaR be fixed this way?
Or would the second workaround cause other unwanted side effects?
Or could the third possible workaround be the best solution?

The customer would prefer the third possible workaround
provided this does not cause unwanted side effects.

  • Workaround:

Create
usr/share/rear/layout/save/NSR/default/400_verify_nsr.sh
as relative symlink that points to
../../../../verify/NSR/default/400_verify_nsr.sh

jsmeix commented at 2023-11-07 10:33:

@hpannenb @gdha @viper1986
could you please have a look here.

In particular I would be interested to know if reverting the fix
https://github.com/rear/rear/commit/420c177d446abec8a64b9683c1a3a3088eae11af
for the issue
https://github.com/rear/rear/issues/2162
and instead creating

usr/share/rear/layout/save/NSR/default/400_verify_nsr.sh

as relative symlink that points to

../../../../verify/NSR/default/400_verify_nsr.sh

would also still fix
https://github.com/rear/rear/issues/2162
plus make "rear checklayout" again result exit code 1
only when the disk layout has changed.

jsmeix commented at 2023-11-07 10:50:

Hmm - wait! - perhaps I misunderstand things:

Perhaps the third possible workaround is
to leave the fix
https://github.com/rear/rear/commit/420c177d446abec8a64b9683c1a3a3088eae11af
for
https://github.com/rear/rear/issues/2162
as is
and only add

usr/share/rear/layout/save/NSR/default/400_verify_nsr.sh

as relative symlink that points to

../../../../verify/NSR/default/400_verify_nsr.sh

to have
https://github.com/rear/rear/issues/2162
still fixed
plus make "rear checklayout" again result exit code 1
only when the disk layout has changed.

schlomo commented at 2023-11-12 19:03:

@jsmeix I guess to verify potential fixes and to evaluate the right approach somebody will need to run some tests on a system with NSR installed.

If a personal and direct interaction with this SUSE customer is possible to do so, then please reach out to @gdha or me as this might yield the fastest results.

Do I understand correctly that upstream ReaR already behaves well and you are now looking for a potential back-port for SUSE?

jsmeix commented at 2023-11-13 08:32:

@schlomo
current upstream ReaR does not behave well
because it contains the change
https://github.com/rear/rear/commit/420c177d446abec8a64b9683c1a3a3088eae11af
of
https://github.com/rear/rear/issues/2162

ReaR from before that change behaved well for what
our customer needs but that does not behave well
for what @viper1986 needs according to
https://github.com/rear/rear/issues/2162#issue-460949301

Our customer has already a workaround for his needs in place.

This issue is about to get a proper ReaR upstream fix
that makes things behave well for both use cases,
the one our customer has and the one @viper1986 has.

In
https://github.com/rear/rear/issues/2162#issuecomment-541343374
@hpannenb proposed two "workarounds"
where the first one was implemented
which causes the issue for our customer
but for the more sophisticated second one
our customer reported that this works for him.

So it seems the more sophisticated second one
makes things behave well for both use cases,
the one our customer has and the one @viper1986 has.

Accordingy I will implement that second workaround now
as a GitHub pull request to somehow move this issue forward...

jsmeix commented at 2023-11-13 08:47:

@hpannenb @viper1986
could you please have a look at
https://github.com/rear/rear/pull/3077

jsmeix commented at 2023-11-13 12:42:

https://github.com/rear/rear/pull/3077
didn't work.

schlomo commented at 2023-11-13 13:00:

The suggestion from the discussion in #3077 is to actually remove the check for NSR backups from the checklayout workflow and instead adding support for easily running custom code as part of the checklayout workflow.

Specifically this would also mean removing the complete 650_check_iso_recoverable.sh
https://github.com/rear/rear/blob/01f503ed05beef78d8397719b9dce25fcdb211c6/usr/share/rear/layout/save/NSR/default/650_check_iso_recoverable.sh#L1-L18

Any thoughts on implementing that? Wouldn't that actually solve the customer problem, too?

I know that this essentially reverts #653 and might force users like @tomglx to adjust the way how they use ReaR. However I hope that this is acceptable if it will prevent troubles for other NSR users.

schlomo commented at 2023-11-13 13:47:

BTW, looking at the non-default code paths in layout/save it seems like only NSR has that type of additional backup monitoring functionality:
image

[Export of Github issue for rear/rear.]