#378 Issue closed: Ability to perform manual Netbackup restore instead of automated?

Labels: won't fix / can't fix / obsolete

andyeff opened issue at 2014-03-12 12:11:

Hi there,

I'm running Rear with the ISO + NBU (Netbackup) restore method, and am wondering if there is a way to have the restore process allow the restore engineer to send a selective Netbackup restore operation to the server, rather than use the automatic system currently in place.

For example: I boot a server from ISO, and run 'rear recover'. I press Enter twice to confirm I am restoring to the same machine and that I want to use the latest backup set.

REAR will then (as I understand it) query backups from 5 days ago, and start restoring from them. HOWEVER, due to the way the backup tapes are handled, this is unfortunately a bit inflexible for me, as some backup tapes may not be available. What I would want to do instead is to stop the automatic netbackup from running, and pause the scripts so I can send a more specific restore operation to the server.

For example, commenting out / removing lines 66-71 from the relevant script:
https://github.com/rear/rear/blob/79f5c64a3e8e12beeb4b924c6e6ccfd9bae70720/usr/share/rear/restore/NBU/default/40_restore_with_nbu.sh#L66-L71

And having the rear recover operation pause while I restore manually, then resume its finishing operations after the recovery is complete (replace the commented-out lines with: read -p "Press Enter to continue" for example.

Is there a way to do something like this with certain parameters in local.conf ? Or would I best need to alter the existing scripts on the machine before running the initial 'rear mkrescue' and generating the ISO?

andyeff commented at 2014-03-12 16:29:

I actually ran into some oddities while doing a manual restore - it looks to be a problem with the Netbackup side rather than Relax and Recover.

As an aside - The netbackup restore sequence prompts to either use the current backup (passes the -s parameter to bprestore), or from an 'earlier than' date (passes the -e parameter to bprestore).

Is there an easy way to either be prompted for both, or use both parameters? For example my test Full backup that I was using to restore from is dated 03/11/2014 15:50:20 , so if I manually specify -s 03/11/2014 15:50:19 and -e 03/11/2014 15:50:21 , it will force Netbackup to only use that Full backup set, and ignore any earlier backups, and any later incremental backups.

gdha commented at 2014-03-13 19:27:

@andyeff not sure in which way I can help you as I do not consider myself a NBU specialist?

andyeff commented at 2014-03-13 19:44:

@gdha Ah, no problem - I was thinking about this today while doing some test backup + restores.

The verify script that runs during a Netbackup restore ( https://github.com/rear/rear/blob/master/usr/share/rear/verify/NBU/default/39_request_point_in_time_restore_parameters.sh ) only prompts for date to use with the -e parameter. This sets a date that netbackup won't go later than to restore backups.

What I decided on was altering this script and also the one used during restore ( https://github.com/rear/rear/blob/master/usr/share/rear/restore/NBU/default/40_restore_with_nbu.sh ) to also be able to prompt and store a date that can be passed to netbackup as a -s parameter. This let me then specify a date range of backup sets to restore from, which avoided the problem of older or newer backups not being present in the tape library of Netbackup.

Admittedly this alteration was mostly just to prove that Relax-and-Recover works for our requirements (and it does - we had a mailbox server back up and running in approx 15 minutes!), but it seems like it might be a useful extra feature for Netbackup users.

I don't have the edits handy at the moment unfortunately, but tomorrow morning I will upload the changes as gists so I can better explain. It is really allowing the ability to enter either a "to" date ( -e parameter with bprestore, which the NBU restore method currently supports), a "from" date ( -s parameter with bprestore, which the NBU restore method does by default if no date is entered, it generates a date 5 days before todays date and passes that as the -s param), and also a to AND from date (passing both a -s and -e parameter to bprestore to restrict restoring backups from between the two date/times given).

schlomo commented at 2014-03-13 21:48:

​I am trying to understand what you say:

Do you want to give NBU a "restore window", e.g. a time span from which to
pick the restore?

Can it be that the actual problem with the current code is that if you
specify a date on which no backup was made then NBU won't simply take the
next best thing but fail? Maybe you can tell NBU how to interpret this
single date to mean "last backup at or before DATE" instead of "backup from
DATE or fail"?

andyeff commented at 2014-03-13 22:01:

Hi @schlomo , my understanding is that there are two options that can be used for the NBU restore.

  1. Do not specify a date/time - and just press Enter when prompted. In 40_restore_with_nbu.sh, this will cause REAR to set -s (the start date/time of restore window) to be 5 days ago. If there are any backups dated from up to 5 days ago, they will be restored in sequence, up until the latest backup. I think in general this is probably appropriate behaviour for most cases.
  2. Specify a date/time. This will pass the date/time to NBU as the end date to restore to, so for example if you wanted to restore from backups to a moment in time 3 days ago, but no later, this would be useful. But it does not include a start time this way, so NBU will attempt to restore from the very first available backup, which could be several weeks or maybe even months ago depending on how long backup sets are retained for.

My desire would be for a third option so two sets of dates could be specified, which would pass both the -e and -s times to NBU, restricting the restores to a range, to avoid either trying to restore from very old and unnecessary backups, or to also skip all but the most recent backups (For example, if backups are taken every day, perhaps you only want to restore the previous days' backup and not all 5 days prior).

schlomo commented at 2014-03-13 22:33:

I see. Basically we need to work around NBUs lack of a consolidated backup
like TSM has. There you set a point in time and it will simply take each
file from whatever backup set it needs to do what you meant.

Not beeing an NBU expert I might ask something silly. Would it be possible
to ask NBU for the backup scheme in use and then make a smarter automatic
choice?

On 13 March 2014 23:01, Andy Eff notifications@github.com wrote:

Hi @schlomo https://github.com/schlomo , my understanding is that there
are two options that can be used for the NBU restore.

Do not specify a date/time - and just press Enter when prompted. In
40_restore_with_nbu.sh, this will cause REAR to set -s (the start date/time
of restore window) to be 5 days ago. If there are any backups dated from up
to 5 days ago, they will be restored in sequence, up until the latest
backup. I think in general this is probably appropriate behaviour for most
cases.
2.

Specify a date/time. This will pass the date/time to NBU as the end
date to restore to, so for example if you wanted to restore from backups to
a moment in time 3 days ago, but no later, this would be useful. But it
does not include a start time this way, so NBU will attempt to restore from
the very first available backup, which could be several weeks or maybe even
months ago depending on how long backup sets are retained for.

My desire would be for a third option so two sets of dates could be
specified, which would pass both the -e and -s times to NBU, restricting
the restores to a range, to avoid either trying to restore from very old
and unnecessary backups, or to also skip all but the most recent backups
(For example, if backups are taken every day, perhaps you only want to
restore the previous days' backup and not all 5 days prior).

—
Reply to this email directly or view it on GitHubhttps://github.com/rear/rear/issues/378#issuecomment-37592457
.

andyeff commented at 2014-03-13 22:40:

Hmm unfortunately I am not that familiar with Netbackup myself - I have only really been using it in the last few weeks after I was tasked with creating a process to allow my team to be able to do full bare-metal restores for servers that weren't being backed up through a vmware method.

I will take a closer look tomorrow at the schemes in use and whether any of the other binaries in the Netbackup client can help with this and reply back as soon as possible. Thank you for your patience so far!

andyeff commented at 2014-03-19 16:48:

Sorry for the delay in getting back to you both.
@schlomo Regarding NBU's backup scheme, we're using a very standard 'normal' backup, where we take one Full backup a week, and then daily incrementals. After considering how to best deal with this, I believe the only change I would request is changing the default '5 day lookback' to be a 7 day lookback, so that a Full backup set can still be found if a restore is needed on the same day that a new Full backup set would otherwise be taken. (e.g. the server has a fatal crash in the morning, before the evening backup has a chance to run, and the most recent backup set to recover from is 7 days old)

This would I think involve changing the "-s" parameter on Line 21: https://github.com/rear/rear/blob/master/usr/share/rear/restore/NBU/default/40_restore_with_nbu.sh#L21
from:

Recent_Month_Hour=`LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \`date -d "-5 days" "+%m/%d/%Y"\` $1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'`

to:

Recent_Month_Hour=`LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \`date -d "-7 days" "+%m/%d/%Y"\` $1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'`

I'm also happy to change the script on my deployed servers after installing, if you'd prefer to keep the 5 day setting instead.

schlomo commented at 2014-03-19 19:04:

I can live with the 7 day setting. Please post to the mailing list and if
nobody disagrees then we will just change it.

On 19 March 2014 17:48, Andy Eff notifications@github.com wrote:

Sorry for the delay in getting back to you both.
@schlomo https://github.com/schlomo Regarding NBU's backup scheme,
we're using a very standard 'normal' backup, where we take one Full backup
a week, and then daily incrementals. After considering how to best deal
with this, I believe the only change I would request is changing the
default '5 day lookback' to be a 7 day lookback, so that a Full backup set
can still be found if a restore is needed on the same day that a new Full
backup set would otherwise be taken. (e.g. the server has a fatal crash in
the morning, before the evening backup has a chance to run, and the most
recent backup set to recover from is 7 days old)

This would I think involve changing the "-s" parameter on Line 21:
https://github.com/rear/rear/blob/master/usr/share/rear/restore/NBU/default/40_restore_with_nbu.sh#L21
from:

Recent_Month_Hour=LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \date -d "-5 days" "+%m/%d/%Y"$1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'

to:

Recent_Month_Hour=LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \date -d "-7 days" "+%m/%d/%Y"$1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'

I'm also happy to change the script on my deployed servers after installing, if you'd prefer to keep the 5 day setting instead.

—
Reply to this email directly or view it on GitHubhttps://github.com/rear/rear/issues/378#issuecomment-38076146
.

roseswe commented at 2014-03-19 19:14:

Why not adding a new variable to local.conf, defaulting to 7 days, so
everybody can change it if need, without touching the sources?

Maybe this can be done more generic so other backup solutions can use
this setting(s) too?

BR,
Ralph

Am 19.03.2014 20:04, schrieb Schlomo Schapiro:

I can live with the 7 day setting. Please post to the mailing list and if
nobody disagrees then we will just change it.

::

Recent_Month_Hour=LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \date -d "-5 days" "+%m/%d/%Y"$1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'

to:

Recent_Month_Hour=LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \date -d "-7 days" "+%m/%d/%Y"$1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'

schlomo commented at 2014-03-20 09:22:

Sounds like a good idea. Who will do the patch? I can't test it.

On 19 March 2014 20:14, Ralph Roth notifications@github.com wrote:

Why not adding a new variable to local.conf, defaulting to 7 days, so
everybody can change it if need, without touching the sources?

Maybe this can be done more generic so other backup solutions can use
this setting(s) too?

BR,
Ralph

Am 19.03.2014 20:04, schrieb Schlomo Schapiro:

I can live with the 7 day setting. Please post to the mailing list and if
nobody disagrees then we will just change it.

::

Recent_Month_Hour=LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \date -d "-5 days" "+%m/%d/%Y"$1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'

to:

Recent_Month_Hour=LANG=C /usr/openv/netbackup/bin/bplist -C ${NBU_CLIENT_SOURCE} -l -s \date -d "-7 days" "+%m/%d/%Y"$1 2>&8 | head -n 1 | awk '{print $5,$6,$7}'

—

Reply to this email directly or view it on GitHubhttps://github.com/rear/rear/issues/378#issuecomment-38094443
.

gdha commented at 2014-04-16 12:11:

What is the proposed variable name? RESTORE_LOOKBACK_DAYS ??

gdha commented at 2015-01-25 17:17:

@andyeff @schlomo @rear/contributors Maybe RESTORE_NR_DAYS seems a better name for the variable. OK?

andyeff commented at 2015-01-25 17:17:

Sounds good!

schlomo commented at 2015-01-25 19:32:

If this variable is specific to NBU it must start with NBU_ because only then will rear dump actually show it.

gdha commented at 2015-02-06 19:50:

The whole discussion is not needed any more as commit https://github.com/rear/rear/commit/45717daf09ce05e53fcf27827299f7d8678adb98#diff-9df2ad19942f62489ef3385c320f4bfc removed the Get_Start_Date function.

[Export of Github issue for rear/rear.]