#705 Issue `closed`: Some NSR Remarks¶

Labels: support / question, needs sponsorship, external tool

fireclown opened issue at 2015-11-19 15:21:¶

While using the ReaR NSR-Mode, i found some things that did not
work like expected. Well, I do not like it that ReaR is doing
it's NetWorker stuff at a few hundred Systems ;-)

I did some changes to some of the rear scripts, to make it easier to
handle for me and i hope some of that will be also useful for you.

First you will find some remarks to that changes:¶

/usr/share/rear/output/NSR/default/95_nsr_save_result_files.sh
PATCH1:
It is not smart form me that "rear mkrescue" will do a save-command
automatically. If I really want to do one, there is the possibility
to do "rear mkbackup"
-> To Skip it I added NSR_SAVE-Variable in /etc/rear/local.conf: NSR_SAVE="NO"

PATCH2:
In many Cases it is not possible to identify the correct Pool.
The Client could be saved to different Pools or in a worse case
the datas are also stored in a Clone-Pool.
For the moment I have no idea how to do it automatically.
-> My Workaround is to set the NSR_DEFAULT_POOL_N in /etc/rear/local.conf.

PATCH3:
In many Cases the Part to verify the backup will not work correctly,
in cause that the Saveset will be the Filesystem and not the files
itself.
I changed that part. But for me that part it is only useful if i
really need to debug. Or maybe if I do the rear-command manually.

/usr/share/rear/conf/default.conf
The basic NSR-Config will cause in space-problem in the /tmp-filesystem
if i use it with a NetWorker-Server.
-> Idea: handle the exclude-variable within /etc/rear/local.conf
-> Workaround: For the Moment I changed COPY_AS_IS_EXCLUDE_NSR
in the default.conf
Anyway it could be a good thing to check if ther will be enough space
in the /tmp Filesystem

One last remark:
The new Enhancement (from #653), that should check if there is a ISO
in the Backup, should also be managed (enable/disable) by a Variable
in /etc/rear/local.conf

My version of 95_nsr_save_result_files.sh:

#
# saving result files via NSR

########## PATCH1 - START ##########
### Prohibit direkt Backup 
if [ "$NSR_SAVE" = "NO" ]; then
    Log      "Do not save files with NSR (EMC NetWorker)"
    LogPrint "Do not save files with NSR (EMC NetWorker)"
    exit 0
fi
########## PATCH1 - END ##########
test ${#RESULT_FILES[@]} -gt 0 || Error "No files to copy (RESULT_FILES is empty)"

LogPrint "Saving result files with NSR (EMC NetWorker)"
NSR_RESULT_FILES=()

# decide where to put the result files for saving them with NSR
# if NSR_RESULT_FILE_PATH is unset, then save the result files where they are
# NOTE: Make sure that your NSR installation will not silently skip files in $TMP_DIR !
test -z "$NSR_RESULT_FILE_PATH" && NSR_RESULT_FILE_PATH=$TMP_DIR

if ! test -d "$NSR_RESULT_FILE_PATH" ; then
     mkdir -v -p "$NSR_RESULT_FILE_PATH" 1>&8
     StopIfError "Could not create '$NSR_RESULT_FILE_PATH'"
fi


if test "$NSR_RESULT_FILE_PATH" != "$TMP_DIR" ; then
    cp -v  "${RESULT_FILES[@]}" "$NSR_RESULT_FILE_PATH" 1>&8
    StopIfError "Could not copy result files to '$NSR_RESULT_FILE_PATH'"
    NSR_RESULT_FILES=( 
        $(
            for fname in "${RESULT_FILES[@]}" ; do 
                echo "$NSR_RESULT_FILE_PATH/$(basename "$fname")"
            done
         )
    )
else
    NSR_RESULT_FILES=( "${RESULT_FILES[@]}" )
fi

if test -s "$CONFIG_DIR/templates/RESULT_usage_$OUTPUT.txt" ; then
    cp -v $CONFIG_DIR/templates/RESULT_usage_$OUTPUT.txt "$NSR_RESULT_FILE_PATH/README" 1>&8
    StopIfError "Could not copy '$CONFIG_DIR/templates/RESULT_usage_$OUTPUT.txt'"
    NSR_RESULT_FILES=( "${NSR_RESULT_FILES[@]}" "$NSR_RESULT_FILE_PATH"/README )
fi

NSRSERVER=$(cat $VAR_DIR/recovery/nsr_server )
CLIENTNAME=$(hostname)
########## PATCH2 - START ##########
### In many Cases it is not possible to identify the correct Pool,
### Even if it is a Clone-Pool.
### Workaround in /etc/rear/local.conf: NSR_DEFAULT_POOL_NAME="TestPool"
#POOLNAME="$( mminfo -s $NSRSERVER -a -q "client=$CLIENTNAME" -r "pool" | head -1 )"
POOLNAME="TestPool"
########## PATCH2 - END ##########
[[ -z "$POOLNAME" ]] && POOLNAME="$NSR_DEFAULT_POOL_NAME"
[[ -z "$NSR_RETENTION_TIME" ]] && NSR_RETENTION_TIME="1 day"

Log "Saving files '${NSR_RESULT_FILES[@]}' with save"
save -s $NSRSERVER -c $CLIENTNAME -b "$POOLNAME" -y "$NSR_RETENTION_TIME" "${NSR_RESULT_FILES[@]}" 1>&8
StopIfError "Could not save result files with save"

# show the saved result files
LogPrint "If the NSR_RETENTION_TIME=\"$NSR_RETENTION_TIME\" is too low please add NSR_RETENTION_TIME variable in $CONFIG_DIR/local.conf"
########## PATCH3 - START ##########
### ! In many Cases the Part to verify the backup will not work correctly
LogPrint "ssid       pool     files savetime    retent  name"
LogPrint "============================================"
read TIME OUT <<< $(mminfo -s $NSRSERVER -aot -q client=epsdist -r 'nsavetime,ssid(11),pool,nfiles(5),savetime,ssretent,name'|tail -1)
echo $OUT 
echo
echo "Files:"
nsrinfo -s $NSRSERVER -t $TIME epsdist | grep -E $( echo ${NSR_RESULT_FILES[@]} | sed -e "s/ /|/g") > $TMP_DIR/saved_result_files
########## PATCH3 - END ##########
LogPrint "$(cat $TMP_DIR/saved_result_files)"

gdha commented at 2015-11-19 15:37:¶

@fireclown Which version of rear were you using? In github sources there were some serious modification made around NSR. It would be nice that you verify this as it could resolve already some of your issues mentioned above?

fireclown commented at 2015-11-19 15:51:¶

i am using rear-1.17.2-1.el5 from the epel-repository.

I will look tomorrow for changes in you soruces.

fireclown commented at 2015-11-20 08:08:¶

Alright, now, i have checked ReaR with the actual git-sources and
I'm sorry it does not really help :(

(PATCH1) "rear mkrescue" will do already a NetWorker save-command.
Solution: Only "rear mkbackup" should do that.
Suggestion: If ther will be no other way Enable/Disable by Variable in local.conf

(PATCH2) maybe a good idea, but it only work for jukebox devices. I am
also using a DataDomain, so it does not work for me and also
not for AFTD-Devices.
The other open Problem is to avoid Clone-Pools.
Suggestion: Sorry, open for automatic detection.

(PATCH3) it is better, now you can find the saveset, for me that's ok,
I suggest also showing the ssid.
Just that part wanted also to show the files in the backup, then
you will need my part of that script, to show the files in the saveset.

./conf/default.conf
Makes the same problems then before, i run out of space in /tmp,
so i have to extend the COPY_AS_IS_EXCLUDE_NSR-Variable.
Suggestion: Possibility to extend Exclude in local.conf

default/65_check_iso_recoverable.sh
I do not like it to be done with "rear mkrescue"
mkrescue should only create an ISO like doing it with NETFS
and like it is written in the help.
> mkbackup create rescue media and backup system
> mkbackuponly backup system without creating rescue media
> mkrescue create rescue media only
or there should be a possibility to Enable/Disable it.
or it one smart way could be to integrate within a "rear checklayout"

gdha commented at 2015-11-20 08:56:¶

@fireclown Thank you for taking your time to check out the latest code.
Just want to emphasize when external backup programs are involved there is no need to run rear mkbackup as only a rescue image is required, therefore, rear mkrescue is sufficient.

Concerning Patch 1 I would prefer a generic variable to save the ISO image (with external backup programs) and the default should be 1 (yes save ISO).
Currently, we already have a variable as such in place:

./conf/default.conf:TSM_RESULT_SAVE=y
./output/TSM/default/95_dsmc_save_result_files.sh:if [[ "$TSM_RESULT_SAVE" = "n" ]]; then

However, I think we should broaden the variable in the sense it is understandable and useable by other backup mechanism such as TSM, DP, NSR, BAREOS and so on. What do you think?
Any suggestions?

Concerning Patch 2 - define your definition in NSR_DEFAULT_POOL_NAME could be the trick. Perhaps, take some time to write a small article about it and we would be glad to publish it on our rear website?

Concerning Patch 3 - I'm not a NSR user and therefore, I'm not sure what ssid means for NSR. There are other NSR users, perhaps, they can discuss the pro/contra's with you as I have no clue.

Concerning COPY_AS_IS_EXCLUDE_NSR - running out of space in /tmp: have you seen:

./conf/default.conf:# the TMPDIR influences the mktemp command to define an alternative basedir instead of /tmp
./conf/default.conf:TMPDIR=""

Concerning default/65_check_iso_recoverable.sh - see Patch 1 remarks. It is about the same.

fireclown commented at 2015-11-20 13:42:¶

Patch1:
yes, that looks like that what i need. I think it will be helpful if
you can make it generic and available for all Backup-Programs.

My motivation is, that i am also a NetWorker-Admin and i want to have
more control about the external accesses to it. The *Result_SAVE can
prevent a few hundred requests (maybe at the same time).

Patch2:
the automatic pool research is a little bit tricky, I am just
trying to find a solution. I will tell you ;-)

Patch3:
Yes I can make it, but my ISO will be very huge (189GB) if
I don't Exclude other Directories.

The ssid is the saveset id

so it could be added like that:

mminfo -s $NSRSERVER -a -q "client=$CLIENTNAME,name=${SNAME},pool=${POOLNAME}" \
    -r "pool,ssid,ssretent,name" > $TMP_DIR/saved_result_files

gdha commented at 2015-12-08 13:59:¶

gdha commented at 2017-05-17 13:20:¶

As we cannot do anything here without 'being contracted' by a NSR customer I close this issue until further notice

[Export of Github issue for rear/rear.]

#705 Issue closed: Some NSR Remarks¶