#643 Issue `closed`: NSR "# show the saved result file" does not work when using USE_CFG2HTML=y¶

Labels: enhancement, waiting for info, support / question, fixed / solved / done

keilma opened issue at 2015-08-21 06:29:¶

Hi,

the "show the saved result file" part in 95_nsr_save_result_files.sh does not work when USE_CFG2HTML=y is used. The problem is how NetWorker names it's save sets. When USE_CFG2HTML=n, only one file is saved (the commands are manual save's, but rear is using the same syntax in the script):

save -s mknw01 -e day -b dd160_ALL /var/lib/rear/output/rear-mkrear.iso
Name for this backup in the media database:
[root@mkrear testfolder]# mminfo -s mknw01 -q client=mkrear -r savetime,name
08/20/2015 /var/lib/rear/output/rear-mkrear.iso

When saving two files, what is the same as using USE_CFG2HTML=y, the following save command is used:

save -s mknw01 -e day -b dd160_ALL /var/lib/rear/output/rear-mkrear.iso /var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html
Name for this backup in the media database:
[root@mkrear testfolder]# mminfo -s mknw01 -q client=mkrear -r savetime,name
08/20/2015 /var/lib/rear/

In 95_nsr_save_result_files.sh the following grep command is used:

grep -E $( echo ${NSR_RESULT_FILES[@]} | sed -e "s/ /|/g") > $TMP_DIR/saved_result_files

${NSR_RESULT_FILES[@]} contains /var/lib/rear/output/rear-mkrear.iso and /var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html and therefore finds no match when the name in the NetWorker media database is only "/var/lib/rear/".

Some parts of my debug log:

2015-08-21 08:25:05 Saving files '/var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html /var/lib/rear/output/rear-mkrear.iso' with s
ave
++ save -s mknw01.marcelk.brs.local -c mkrear.marcelk.brs.local -b dd160_ALL -y week /var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.loca
l.html /var/lib/rear/output/rear-mkrear.iso
...
++ mminfo -s mknw01.marcelk.brs.local -a -q client=mkrear.marcelk.brs.local -r pool,ssretent,name
+++ echo /var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html /var/lib/rear/output/rear-mkrear.iso
+++ sed -e 's/ /|/g'
++ grep -E '/var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html|/var/lib/rear/output/rear-mkrear.iso'

And how my mminfo query looks:

[root@mkrear rear]# mminfo -s mknw01.marcelk.brs.local -a -q client=mkrear.marcelk.brs.local -r pool,ssretent,name
dd160_ALL     08/27/2015 /var/lib/rear/
dd160_ALL     08/27/2015 /var/lib/rear/
dd160_ALL     08/21/2015 /var/lib/rear/
dd160_ALL     08/27/2015 /var/lib/rear/
dd160_ALL     08/28/2015 /var/lib/rear/

Regards
Marcel

gdha commented at 2015-08-28 13:01:¶

@keilma before making any conclusions - are the two files saved via NSR, or not?
And if yes, it would mean that checking /var/lib/rear would be enough with the mminfo command?

keilma commented at 2015-08-28 14:28:¶

@gdha it depends. When cfg2html is installed and used there are two files saved and the saveset is named /var/lib/rear:

[root@mknw01 ~]# mminfo -avot -q client=mkrear,name=/var/lib/rear/ -r name,savetime,nsavetime
 name                            date     save time
/var/lib/rear/                08/20/2015 1440092413
/var/lib/rear/                08/20/2015 1440092718
/var/lib/rear/                08/20/2015 1440092958
/var/lib/rear/                08/20/2015 1440093256
/var/lib/rear/                08/20/2015 1440093346
/var/lib/rear/                08/20/2015 1440096820
/var/lib/rear/                08/21/2015 1440138305
[root@mknw01 ~]# nsrinfo -t 1440138305 mkrear
scanning client `mkrear' for savetime 1440138305(Fri 21 Aug 2015 08:25:05 AM CEST) from the backup namespace
/var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html
/var/lib/rear/output/rear-mkrear.iso
/var/lib/rear/recovery/cfg2html/
/var/lib/rear/recovery/
/var/lib/rear/output/
/var/lib/rear/
/var/lib/
/var/
/
9 objects found

When cfg2html is not installed or set to "no" the save set is named /var/lib/rear/output/rear-mkrear.iso because that's the only file saved:

[root@mknw01 ~]# mminfo -avot -q client=mkrear -r name,savetime,nsavetime
 name                            date     save time
...
/var/lib/rear/output/rear-mkrear.iso 08/28/2015 1440771610
[root@mknw01 ~]# nsrinfo -t 1440771610 mkrear
scanning client `mkrear' for savetime 1440771610(Fri 28 Aug 2015 04:20:10 PM CEST) from the backup namespace
/var/lib/rear/output/rear-mkrear.iso
/var/lib/rear/output/
/var/lib/rear/
/var/lib/
/var/
/
6 objects found

When cfg2html is used you can grep for /var/lib/rear, if not you have to use /var/lib/rear/output/rear-mkrear.iso.

gdha commented at 2015-08-31 06:46:¶

@keilma what would be the output if you saved a 3th file, e.g. /var/log/rear/rear-$(hostname).log ? I guess the output would be different then as well? Perhaps there is a better way to intercept the desired output?

keilma commented at 2015-08-31 16:58:¶

@gdha yes, it's different:

[root@mkrear rear]# save -s mknw01 -e day -b dd160_ALL /var/lib/rear/output/rear-mkrear.iso /var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html /var/log/rear/rear-mkrear.log
[root@mkrear rear]# mminfo -s mknw01 -avot -q client=mkrear -r name,savetime,nsavetime
...
/var/                         08/31/2015 1441039587

[root@mkrear rear]# nsrinfo -s mknw01 -t 1441039587 mkrear.marcelk.brs.local
scanning client `mkrear.marcelk.brs.local' for savetime 1441039587(Mon 31 Aug 2015 06:46:27 PM CEST) from the backup namespace on server mknw01
/var/lib/rear/output/rear-mkrear.iso
/var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html
/var/log/rear/rear-mkrear.log
/var/lib/rear/recovery/cfg2html/
/var/lib/rear/recovery/
/var/lib/rear/output/
/var/lib/rear/
/var/log/rear/
/var/log/
/var/lib/
/var/
/
12 objects found

One solution could be to use the "-N" switch of save to give the save set another name:

[root@mkrear rear]# save -s mknw01 -e day -b dd160_ALL -N ReaR_Files-_$(hostname) /var/lib/rear/output/rear-mkrear.iso /var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html /var/log/rear/rear-mkrear.log
[root@mkrear rear]# mminfo -s mknw01 -avot -q client=mkrear -r name,savetime,nsavetime 
...
ReaR_Files-_mkrear.marcelk.brs.local 08/31/2015 1441039864
[root@mkrear rear]# nsrinfo -s mknw01 -t 1441039864 mkrear.marcelk.brs.local
scanning client `mkrear.marcelk.brs.local' for savetime 1441039864(Mon 31 Aug 2015 06:51:04 PM CEST) from the backup namespace on server mknw01
/var/lib/rear/output/rear-mkrear.iso
/var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html
/var/log/rear/rear-mkrear.log
/var/lib/rear/recovery/cfg2html/
/var/lib/rear/recovery/
/var/lib/rear/output/
/var/lib/rear/
/var/log/rear/
/var/log/
/var/lib/
/var/
/
12 objects found

An easier way could be to use the output from save for your grep command:

[root@mkrear rear]# save -s mknw01 -e day -b dd160_ALL /var/lib/rear/output/rear-mkrear.iso /var/lib/rear/recovery/cfg2html/mkrear.marcelk.brs.local.html
57777:save:Multiple client instances of mkrear.marcelk.brs.local, using the first entry
libDDBoost version: major: 3, minor: 0, patch: 2, engineering: 0, build: 480717
86704:save: Successfully established DDCL session for save-set ID '4125394708' (mkrear.marcelk.brs.local:/var/lib/rear/).
...
save: /var/lib/rear/  137 MB 00:00:05      9 files
94694:save: The backup of save set '/var/lib/rear/' succeeded.

In this example the save set is named "/var/lib/rear".
Hope that helps.

Regards
Marcel

tomglx commented at 2015-09-10 06:57:¶

Is there a good reason not to solve this issue by using the -N option?
I've created a patch that creates a saveset name and uses this for saving and verifying. If a networker save fails, the entire saveset won't be queryable. So a partial save won't be
concealed by using this option.

-save -s $NSRSERVER -c $CLIENTNAME -b "$POOLNAME" -y "$RETENTION_TIME" "${NSR_RESULT_FILES[@]}" 1>&8
+SNAME="REAR.$(date +%Y%m%d)"
+save -s $NSRSERVER -c $CLIENTNAME -N "${SNAME}" -b "$POOLNAME" -y "$RETENTION_TIME" "${NSR_RESULT_FILES[@]}" 1>&8

-mminfo -s $NSRSERVER -a -q "client=$CLIENTNAME" -r "pool,ssretent,name" | \
-    grep -E $( echo ${NSR_RESULT_FILES[@]} | sed -e "s/ /|/g") > $TMP_DIR/saved_result_files
+mminfo -s $NSRSERVER -a -q "client=$CLIENTNAME,name=${SNAME},pool=${POOLNAME}" \
+       -r "pool,ssretent,name" > $TMP_DIR/saved_result_files

What do you think about this?

Regards, Thomas

gdha commented at 2015-09-10 07:27:¶

@tomglx I like the idea - why not create a pull request?
@keilma Would you be able to test this out and return your findings?
Sorry people I cannot test this out myself...

keilma commented at 2015-09-10 07:29:¶

Hi,

@tomglx: yes, I also mentioned the -N switch in my post above and I think that's the best way to solve this.
@gdha: sure, can you create a new branch with this fix or should I edit my scripts manually?

Regards
Marcel

gdha commented at 2015-09-23 08:17:¶

@keilma You need to click on Fork on the rear project on Github (login with your account first) to make a branch of rear yourself, which you can clone and do what you like with it. From this branch you can create a pull request towards the master branch of rear. Not that difficult once you get the hang of it ;-)