#2833 Issue closed: Issue with ReaR and Borg

Labels: support / question, fixed / solved / done

ZENAdmin-Ops opened issue at 2022-07-01 05:30:

Relax-and-Recover (ReaR) Issue Template

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

  • ReaR version ("/usr/sbin/rear -V"):
    2.6

  • OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
    Ubuntu 22.04

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

this_file_name=$( basename ${BASH_SOURCE[0]} )
LOGFILE="$LOG_DIR/rear-$HOSTNAME-$WORKFLOW-${this_file_name%.*}.log"
BACKUP=BORG
BACKUP_ONLY_INCLUDE="yes"
BACKUP_PROG_INCLUDE=( '/home/*' '/opt/*' )
BORGBACKUP_ARCHIVE_PREFIX="rear"
# BORGBACKUP_HOST="borg.server.name"
# BORGBACKUP_USERNAME="borg_server_username"
# BORGBACKUP_REPO="/path/to/borg/repository/on/borg/server"
BORGBACKUP_PRUNE_KEEP_DAILY=7
BORGBACKUP_PRUNE_KEEP_WEEKLY=5
BORGBACKUP_PRUNE_KEEP_MONTHLY=12
BORGBACKUP_COMPRESSION="zlib,9"
BORGBACKUP_ENC_TYPE="keyfile"
export BORG_KEYS_DIR="~/root/.config/borg/keys"
# export BORG_CACHE_DIR=" /root/.config/borg/cache"
export BORG_PASSPHRASE='actual phrase removed'
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"
export BORG_REMOTE_PATH="/media/zen/REAR-000/Borg"

  • Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
    Hyper-V VM. Gen 2

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):

  • Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
    Local USB formatted by Rear

  • Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):
    N/A

  • Description of the issue (ideally so that others can reproduce it):

Hello,

I am posting this question on the Github issue tracker, because the rear-users list is not working at the moment.

I think that my issue is more than likely a configuration issue on my part, rather than a Rear issue.

I'm new to Rear. I currently have a working Rear backup to a local USB drive using NETFS.

I want to integrate Borg into my Rear config.

I currently have Borg backup to the local USB drive working when I use Borg directly.

The issue that I'm having is when trying to call Borg via Rear.

I've tried a number of different settings in the conf file and none have worked

Here is an extract of the error from Rear

$ sudo rear -C borg_backup mkbackuponly -v
Relax-and-Recover 2.6 / Git
Running rear mkbackuponly (PID 86844)
Using log file: /var/log/rear/rear-Ubuntu-DR6.86844.log
Sourcing additional configuration file '/etc/rear/borg_backup.conf'
Running workflow mkbackuponly on the normal/original system
Couldn't list Borg repository '' on USB
Borg: /tmp/rear.X1bb8vA4MPiJqXK/borg_backup is not a valid repository. Check repo config.
ERROR: Failed to initialize Borg repository, borg rc 2!
Some latest log messages since the last called script 300_init_archive.sh:
  2022-07-01 15:12:03.633219635 Including prep/BORG/default/300_init_archive.sh
  2022-07-01 15:12:04.330816517 Couldn't list Borg repository '' on USB
  2022-07-01 15:12:04.336928993 Borg: /tmp/rear.X1bb8vA4MPiJqXK/borg_backup is not a valid repository. Check repo config.
Aborting due to an error, check /var/log/rear/rear-Ubuntu-DR6.86844.log for details
Exiting rear mkbackuponly (PID 86844) and its descendant processes ...
Running exit tasks

In an ideal world, if someone can see the error in my conf file, that would be awesome.

Failing that, some troubleshooting suggestions would be appreciated.

Thank you

Vaughan

jsmeix commented at 2022-07-01 11:42:

@ZENAdmin-Ops
I am not a Borg backup user so I cannot actually help with
Borg backup specific issues.

Do you know about our Borg backup documentation in
doc/user-guide/04-scenarios.adoc
which is currently online at
https://github.com/rear/rear/blob/master/doc/user-guide/04-scenarios.adoc
therein the section "ReaR with Borg back end"?

Your above posted site.conf or local.conf looks incomplete.
I am missing generic things like

OUTPUT=ISO
OUTPUT_URL=nfs://foo.bar.xy/mnt/backup/iso

or alternatively

OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

Also without a complete "rear -D mkrescue"
and/or a "rear -D mkbackup" debug log file
we have basically no chance to imagine
what is actually going on on your system.

ZENAdmin-Ops commented at 2022-07-01 14:11:

@jsmeix

I have the following working configuration (but this doesn't use Borg)

I have been able to restore this backup. But I don't want to run this daily.

/etc/rear/basic_system.conf
sudo rear -C basic_system mkbackup

OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
USB_UEFI_PART_SIZE=1000
USB_RETAIN_BACKUP_NR=10

Thanks for your post.

I have been using a different example as my template for the integration of the Borg backup
https://github.com/rear/rear/blob/master/doc/user-guide/11-multiple-backups.adoc

After your comments, I had a closer look at the section of the user guide which you refer to. And while I still don't have a working config as I write. I have been able to correct the error with Rear being unable to locate the Borg repository.

This is definitely progress.

Thanks!

Sometimes you just need a fresh set of eyes to provide a different perspective.

I'll do some further testing tomorrow.

ZENAdmin-Ops commented at 2022-07-02 04:27:

Hello,

Here's my latest attempt

sudo rear -C borg_backup mkbackuponly -v
Relax-and-Recover 2.6 / Git
Running rear mkbackuponly (PID 93493)
Using log file: /var/log/rear/rear-Ubuntu-DR6.93493.log
Sourcing additional configuration file '/etc/rear/borg_backup.conf'
Running workflow mkbackuponly on the normal/original system
Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)
Using autodetected kernel '/boot/vmlinuz-5.15.0-39-generic' as kernel in the recovery system
Creating disk layout
Using guessed bootloader 'EFI' (found in first bytes on /dev/sda)
Verifying that the entries in /tmp/rear.EyiICJA28bUrrvG/tmp/backuplayout.conf are correct ...
Creating backup archive 'rear_2' in Borg repository /Borg on USB
Creating archive at "/tmp/rear.EyiICJA28bUrrvG/borg_backup/Borg::rear_2"
/home/*: [Errno 2] No such file or directory: '*'
/opt/*: [Errno 2] No such file or directory: '*'
ERROR: Borg failed to create backup archive, borg rc 1!
Some latest log messages since the last called script 500_make_backup.sh:
  2022-07-02 14:14:04.909088454 Including backup/BORG/default/500_make_backup.sh
  2022-07-02 14:14:04.918857173 Creating backup archive 'rear_2' in Borg repository /Borg on USB
  Creating archive at "/tmp/rear.EyiICJA28bUrrvG/borg_backup/Borg::rear_2"
  /home/*: [Errno 2] No such file or directory: '*'
  /opt/*: [Errno 2] No such file or directory: '*'
Aborting due to an error, check /var/log/rear/rear-Ubuntu-DR6.93493.log for details
Exiting rear mkbackuponly (PID 93493) and its descendant processes ...
Running exit tasks
Terminated

Failing because of the backup selection

/home/*: [Errno 2] No such file or directory: '*'
/opt/*: [Errno 2] No such file or directory: '*'

Here is the: borg_backup.conf

this_file_name=$( basename ${BASH_SOURCE[0]} )
LOGFILE="$LOG_DIR/rear-$HOSTNAME-$WORKFLOW-${this_file_name%.*}.log"

OUTPUT=USB
BACKUP=BORG

USB_DEVICE=/dev/disk/by-label/REAR-000

BORGBACKUP_REPO="/Borg"
# BORGBACKUP_UMASK="0002"

BACKUP_ONLY_INCLUDE="yes"
BACKUP_PROG_INCLUDE=( '/home/*' '/opt/*' )
BORGBACKUP_ARCHIVE_PREFIX="rear"
# BORGBACKUP_HOST="borg.server.name"
# BORGBACKUP_USERNAME="borg_server_username"
# BORGBACKUP_REPO="/path/to/borg/repository/on/borg/server"
BORGBACKUP_PRUNE_KEEP_DAILY=7
BORGBACKUP_PRUNE_KEEP_WEEKLY=5
BORGBACKUP_PRUNE_KEEP_MONTHLY=12
BORGBACKUP_COMPRESSION="zlib,9"
BORGBACKUP_ENC_TYPE="keyfile"

COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" )
export BORG_KEYS_DIR=( "$ROOT_HOME_DIR/.config/borg/keys/" )

export BORG_PASSPHRASE='<passphrase-here>'
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

The issue appears to be this statement

BACKUP_PROG_INCLUDE=( '/home/*' '/opt/*' )

But the syntax looks OK to me?

ZENAdmin-Ops commented at 2022-07-02 04:34:

The answer appears to be to change

BACKUP_PROG_INCLUDE=( '/home/*' '/opt/*' )

To this

BACKUP_PROG_INCLUDE=( '/home/' '/opt/' )

Which seems odd, because the default.conf includes examples like the following

BACKUP_PROG_EXCLUDE=( '/tmp/*' '/dev/shm/*' "$VAR_DIR/output/*" )

jsmeix commented at 2022-07-05 11:46:

@ZENAdmin-Ops
I am not a Borg backup user so I cannot actually help with
Borg backup specific issues.

From what is see in the code it goes like this:

With BACKUP_PROG_INCLUDE=( '/home/*' '/opt/*' )
usr/share/rear/backup/BORG/default/400_create_include_exclude_files.sh
creates $TMP_DIR/backup-include.txt which contains

/home/*
/opt/*

that is read by
usr/share/rear/backup/BORG/default/500_make_backup.sh
into a include_list array which then contains
the array elements /home/* and /opt/*
and that array is used in the borg_create() function in
usr/share/rear/lib/borg-functions.sh
in a borg create call like

borg create ... "${include_list[@]}"

which evaluates to

borg create ... /home/* /opt/*

which seems to let 'borg' fail with

/home/*: [Errno 2] No such file or directory: '*'
/opt/*: [Errno 2] No such file or directory: '*'

so it seems 'borg' does not do "globbing"
i.e. what bash does as "pathname expansion"
or something similar.

In contrast with BACKUP_PROG_INCLUDE=( '/home/' '/opt/' )
usr/share/rear/backup/BORG/default/400_create_include_exclude_files.sh
creates $TMP_DIR/backup-include.txt which contains

/home/
/opt/

that is read by
usr/share/rear/backup/BORG/default/500_make_backup.sh
into a include_list array which then contains
the array elements /home/ and /opt/
and that array is used in the borg_create() function in
usr/share/rear/lib/borg-functions.sh
in a borg create call like

borg create ... "${include_list[@]}"

which evaluates to

borg create ... /home/ /opt/

which works with 'borg'.

Tertium datur:
With BACKUP_PROG_INCLUDE=( /home/* /opt/* )
usr/share/rear/backup/BORG/default/400_create_include_exclude_files.sh
creates $TMP_DIR/backup-include.txt which contains
what bash pathname expansion results for /home/* and /opt/*
i.e. what

# echo /home/* /opt/*

outputs which is all files and directories in /home/ and /opt/
(but not recursively for files in directories and sub-directories)
and then $TMP_DIR/backup-include.txt is read by
usr/share/rear/backup/BORG/default/500_make_backup.sh
into a include_list array which then contains
all files and directories in /home/ and /opt/
and that array is used in the borg_create() function in
usr/share/rear/lib/borg-functions.sh
in a borg create call like

borg create ... "${include_list[@]}"

which evaluates to

borg create ... [all files and directories in /home/ and /opt/]

which likely also works with 'borg'.

jsmeix commented at 2022-07-05 12:06:

Via
https://github.com/rear/rear/commit/9fc626918dc61b855e796d38b7fcee579136e88c
there is now a a more generic description in default.conf
about quoting to avoid bash pathname expansion
and a better example

BACKUP_PROG_INCLUDE=( /some/directory /path/to/some/file )

jsmeix commented at 2022-07-05 12:41:

Note the subtle difference:

BACKUP_PROG_INCLUDE=( /home/* /opt/* )

evaluates to all files and directories in /home/ and /opt/
(but not recursively for files in directories and sub-directories
and also not to the directories /home and /opt itself)
like on my current system

# echo /home/* /opt/*
/home/johannes /home/rear /opt/containerd /opt/google

so a backup tool that is called as

backup_tool /home/johannes /home/rear /opt/containerd /opt/google

will backup all in

/home/johannes /home/rear /opt/containerd /opt/google

but it may not backup the parent directories

/home /opt

(in particular owner group permissions of that parent directories)
so restoring that backup may not restore those parent directories
(but create them anew from scratch with some default settings).

So in general it is better to keep it simple and specify

BACKUP_PROG_INCLUDE=( /home /opt )

(bash pathname expansion for '/home' and '/opt' results '/home' and '/opt')
to backup the directories /home and /opt
with all what those directories contain.

jsmeix commented at 2022-07-07 06:25:

I assume this issue is sufficiently answered so I close it.
Further comments may be added regardless that it is closed
or it could be reopened if needed.

[Export of Github issue for rear/rear.]