#192 Issue `closed`: BACKUP=DP OUTPUT=PXE: PXE files are not copied to OUTPUT_URL¶

Labels: enhancement, discuss / RFC

pavoldomin opened issue at 2013-01-29 10:14:¶

Hello,

my config sets OUTPUT=PXE and BACKUP=DP.
PXE files (kernel, initrd, menu, message, MAC links) are placed to /var/lib/rear/output, but not to a location defined by OUTPUT_URL.

gdha commented at 2013-01-29 15:26:¶

I guess the problem is with the output/ISO/Linux-i386/90_transfer_image.sh script where the OUTPUT_URL is used to transfer the image (that's good), but we seem to assume that the destination path exists (if we use BACKUP=NETFS scheme then it is normally the case), but in case we use another BACKUP scheme such as RBME or DP (or another external backup program) then we might run into problems such as following error (non-fatal for rear?) saying cp: cannot create regular file /tmp/rear.XXX/outputfs/hostname: not a directory

pavoldomin commented at 2013-01-29 19:02:¶

This is indeed the case if BACKUP=DP, OUTPUT=ISO and there is no ${NETFS_PREFIX} directory under OUTPUT_URL ( part of log file: https://gist.github.com/adbaca2ba03c4f9ae85e )

However, with the BACKUP=DP, OUTPUT=PXE, no routine tries to copy PXE files to OUTPUT_URL at all.

With BACKUP=NETFS, OUTPUT_URL is honored with OUTPUT=PXE as well as OUTPUT=ISO, controlled by output/NETFS/default/20_make_prefix_dir.sh and output/NETFS/default/95_copy_result_files.sh

It would be more consistent, if OUTPUT_URL is honored same way, regardless the BACKUP method used.

This seem to me now more enhancement request, than bug :)

Another possibility is to extend PXE_TFTP_PATH and PXE_CONFIG_PATH to support url: I opened another issue #193

pavoldomin commented at 2013-01-31 09:20:¶

In order to be more constructive, i tried to patch the rear in order to separate backup url and output url handling, for your consideration.

I introduced OUTPUT_PREFIX variable and cloned output_path function to backup_path function.
I removed output handling functions from NETFS output stage and moved to default output stage.

Tested yet only with rear mkbackup, rear recover testing in progress.

Of course, the change is not only cosmetic and patch may break something i have not considered; please just throw it away, if its non-sense ;)

https://gist.github.com/82017c57797357fc2e35

gdha commented at 2013-02-01 09:56:¶

@pavoldomin very interesting patch. Idea is good, but not sure if it is bulletproof?? The OUTPUT_PREFIX variable is a good idea. Perhaps, we should align this towards an BACKUP_PREFIX variable too? In general we can just copy NETFS_PREFIX to BACKUP_PREFIX, but I think it is better we make parallel ways (which are quite similar in naming convention).
Are you sure you didn't mix up bpath with opath in all cases? Please do enough tests to be sure the paths can be different in all aspects. Great job - I wish we got more of these inputs.
When you proof that it works all the way why don't you make a "pull request" for it?

pavoldomin commented at 2013-02-05 09:15:¶

Sorry for my slow latency; I have successfully tested on some setups (ISO output and NETFS & DP backup, old DL proliant with the OS on cciss logical drives). We still need to test on another configurations.

Few more thoughts (I am not able to test these targets, so I'm not providing patch)

Perhaps it would make sense removing also output scripts for RSYNC?

--- a/usr/share/rear/output/RSYNC/default/20_make_prefix_dir.sh
+++ /dev/null

and

--- a/usr/share/rear/output/RSYNC/default/90_copy_result_files.sh
+++ /dev/null

Not quite sure, but i believe also part of output/RAMDISK/Linux-i386/90_copy_ramdisk.sh may now be cut below RESULT_FILES= block.
implement OUTPUT_PREFIX also for fish|ftp|ftps|hftp|http|https|sftp|rsync (it would however break target path for existing configurations)
NETFS_PREFIX <=> BACKUP_PREFIX and implement also for fish|ftp|ftps|hftp|http|https|sftp|rsync
But - as it can change behavior of existing configurations (${RESULT_FILES[@]} might be uploaded to different place) - it requires a discussion.

gdha commented at 2013-02-06 15:21:¶

Indeed serious testing is still required.

pavoldomin commented at 2013-02-07 21:17:¶

Hi,

I have now tested this commit in the complete disaster recovery scenarios, at various combinations of BACKUP=DP, BACKUP=NETFS, and OUTPUT=ISO, OUTPUT=PXE, on couple of our enterprise linux setups, SAN or cciss/hpsa logical drive booting. Works just fine. Also fiddled with the KEEP_OLD_NETFS_COPY/KEEP_OLD_OUTPUT_COPY options.

Also managed to try BACKUP=RSYNC with the BACKUP_URL=ssh://..., no problem. I just could not get working OUTPUT_URL=rsync://... - because I got no rsync server running on other site - and it cannot be told to use ssh connection at the moment.
Update: just realized that this is already fixed by a1ff4a89f61bfa6e1ef188b6408d3f48245f0152

pavoldomin commented at 2013-05-16 19:39:¶

We detected minor bug introduced with the patch.
If OUTPUT_URL is the same as BACKUP_URL and in the same time we enable KEEP_OLD_NETFS_COPY, than resulting output files (pxe or iso image) are moved to .old backup dir. I believe this happens in both mkbackup and mkbackuponly modes.

This is because of the flaw

Using log file: /var/log/rear/rear-srv.log
Simulation mode activated, Relax-and-Recover base directory: /usr/share/rear
…
Source output/default/15_save_copy_of_prefix_dir.sh
…
Source output/default/95_copy_result_files.sh
…
Source backup/NETFS/default/15_save_copy_of_prefix_dir.sh
…
Source backup/NETFS/default/50_make_backup.sh
…

i.e. the prefix dir is copied in the backup phase, after the output result files are stored there in output phase.
The solution I can think of, would be either to place both 15_save_copy_of_prefix_dir.sh scripts before backup and output are sourced, or call cp -a instead of mv inside.

What do you think?

gdha commented at 2013-07-03 11:43:¶

@pavoldomin I think as we have 2 variables now KEEP_OLD_OUTPUT_COPY and NETFS_KEEP_OLD_BACKUP_COPY we can close this issue?

gdha commented at 2013-09-03 11:08:¶

In my opinion it's safe to close this issue. If needed it can be re-opened

[Export of Github issue for rear/rear.]

#192 Issue closed: BACKUP=DP OUTPUT=PXE: PXE files are not copied to OUTPUT_URL¶