#1966 Issue `closed`: Borg integration cannot handle local backups¶

Labels: enhancement, needs sponsorship, external tool, no-issue-activity

Signum opened issue at 2018-11-19 00:39:¶

ReaR version ("/usr/sbin/rear -V"): 2.4
OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):

Distributor ID: Debian
Description: Debian GNU/Linux 9.6 (stretch)
Release: 9.6
Codename: stretch

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):

HP DL 360 hardware server

System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):

AMD64

Storage (lokal disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):

local disk

Description of the issue (ideally so that others can reproduce it):

Apparently the Borg integration can either handle SSH connections or a local USB device. In my case I am usually using a mounted NFS file system to access the backup server. It looks like REAR currently does not support that scenario. If the BORGBACKUP_HOST variable is set then it enforces an SSH connection. If it is not set then a local USB device must be present. But a repository that is accessible at e.g. /mnt/backup-server/rear-volume cannot be used.

If I try to omit BORGBACKUP_HOST and also the USB setting then REAR dies with messages like:

Using log file: /opt/rear/var/log/rear/rear-reartest.log
ERROR: Mount command 'mount -v -o rw,noatime / /tmp/rear.cg4UEAoZfJI8rUT/borg_backup' failed.
Some latest log messages since the last called script 250_mount_usb.sh:
  2018-11-19 00:43:46.233832962 Including prep/BORG/default/250_mount_usb.sh
  mkdir: created directory '/tmp/rear.cg4UEAoZfJI8rUT/borg_backup'

Workaround, if any:

None known.

gozora commented at 2018-11-19 07:05:¶

Hi @Signum,

Did you try to use backup over SSH to localhost?

This should work fine for you ...

V.

Signum commented at 2018-11-19 13:00:¶

Thank you. That works indeed. But doesn't that add overhead for the SSH encryption?

schlomo commented at 2018-11-19 13:05:¶

I would expect a significant performance degradation when using localhost SSH compared to direct mounting.

@Signum do you feel like preparing a pull request to fix this? This is a typical case of a use case that didn't show up so far. I would suggest to add logic that compares BORGBACKUP_HOST with the local host name and with localhost in order to activate direct mounting. It might be possible that it is enough to adjust the variables to trigger the right code path. That way you could use the same configuration everywhere and the backup would work locally if it is on the backup server.

In https://github.com/schlomo/rbme/blob/master/rbme#L486 (my own backup tool) I had to solve the same problem.

Signum commented at 2018-11-19 13:18:¶

@schlomo I am pretty sure I can provide a decent pull request, yes. Thanks for the discussion. Feel free to leave this issue open until then. Give me roughly a week.

gozora commented at 2018-11-19 13:24:¶

When I was writing BORG support for ReaR, I was thinking about direct mounting, but using localhost over SSH looked to be much more universal solution comparing to writing mount code for all the different networking file systems out there.
Currently @Signum have NFS mounted but soon Samba share might appear, afterwards some "fuse you name it file system".
But in general if we can have universal code for Borg honoring universal mounting I have not problem with that.

V.

gozora commented at 2018-11-19 13:26:¶

@Signum I think that ssh encryption factor on HP DL 360 is negligible ;-)

V.

Signum commented at 2018-11-19 14:48:¶

While trying with "localhost" I just realized that the boot medium would have to mount the NFS mountpoint to that borg can access the backup. I couldn't find anything in the REAR docs on how to do that. Is it supported? Because if NFS mounts wouldn't work then an automated recovery would not work.

gozora commented at 2018-11-19 15:05:¶

You need to include NFS related sw like kernel modules and binaries like mount.nfs, all the rpc.* stuff .., into ReaR recovery system.
One way to do it would be to use MODULES and PROGS_BORG options in local.conf.

V.

jsmeix commented at 2018-11-19 15:24:¶

I am not a BORG user so only a blind shot in the dark:

Because
for f in $( find usr/share/rear/ -type f | grep -i borg ) ; do grep -H 'BACKUP_URL' $f ; done
does not show something I assume the current BORG code in ReaR
does not use the BACKUP_URL config variable.

For BACKUP=NETFS primarily the BACKUP_URL config variable
determines what stuff gets included into the ReaR recovery system via
usr/share/rear/prep/NETFS/default/050_check_NETFS_requirements.sh
and also via its symbolic link
usr/share/rear/prep/BLOCKCLONE/default/050_check_NETFS_requirements.sh
for BACKUP=BLOCKCLONE.

Therefore I wonder if that code for BACKUP=NETFS might be relatively easily
also used for BACKUP=BORG with an appropriate BACKUP_URL setting
because there should be no conflicts with the other existing code for
BACKUP=BORG when BACKUP_URL has an arbitrary value?

jsmeix commented at 2018-11-19 15:35:¶

Another not so blind shot in the dark:

It seems what @Signum needs is some functionality of BACKUP=NETFS
together with BACKUP=BORG which looks somewhat related to
"Using Multiple Backups for Relax-and-Recover" as described in
https://github.com/rear/rear/blob/master/doc/user-guide/11-multiple-backups.adoc

@Signum
have a look if it might work as a first step towards what you need when you do
some kind of "Mis-Using Multiple Backups for Relax-and-Recover"
to combine BACKUP=NETFS support together with BACKUP=BORG
in one same ReaR recovery system.

Signum commented at 2018-11-19 15:43:¶

@jsmeix Thanks for the clarification. Yes, I would like to use NETFS but not copy the entire machine but rather do incremental backups. Multiple backups seemed pretty complicated. My task is to run a PostgreSQL server at a remove location with no IT staff on-site. So having REAR seemed to be the best way for disaster recovery. And preferable the "Automatic" variant.

I will re-read the page about multiple backups. Although it feels like I'm doing something I shouldn't be.

github-actions commented at 2020-08-24 01:34:¶

Stale issue message

[Export of Github issue for rear/rear.]

#1966 Issue closed: Borg integration cannot handle local backups¶