#2221 Issue closed: calculate_req_space too dumb

Labels: support / question, needs sponsorship, fixed / solved / done

ILMostro opened issue at 2019-09-01 20:02:

  • ReaR version ("/usr/sbin/rear -V"): 2.4 / Git

  • OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"): rhel7.7

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

OUTPUT=BACKUP
OUTPUT_URL=rsync://remotesrv:/extrdisk/host112
BACKUP=RSYNC
BACKUP_URL=rsync://remotesrv:/extrdisk/host112
BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/sys/*' '/proc/*' '/backup/*' '/media/*' '/var/tmp/*' '/var/crash/*' '/run/*' '/var/lib/libvirt/*')
BACKUP_ONLY_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/sys/*' '/proc/*' '/backup/*' '/media/*' '/var/tmp/*' '/var/crash/*' '/run/*' '/var/lib/libvirt/*')
NETFS_KEEP_OLD_BACKUP_COPY=

  • Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR): baremetal x86_64 PC

  • Description

Backups fail prematurely due to incorrect target disk size calculations. As of right now (2.4), the corresponding script ( backup/RSYNC/default/450_calculate_req_space.sh ) simply looks at the df output of local partitions, disregarding any excluding items from the config files. The script doesn't even take into account any reasonably default locations for partitions;
e.g. udisks2 uses /run/media/username/ to auto-mount trivial partitions. These should be excluded by default. What's even worse, there is no way to exclude this at the moment at all, as evidenced by my attempt to exclude items in /run/*. Consequently, rear fails with errors

ERROR: Not enough disk space available on remotesrv:/extrdisk/host112 ...
  • Workaround

For the time being, there is an option to exclude these by using the EXCLUDE_MOUNTPOINTS() option in the config file.

jsmeix commented at 2019-09-03 15:55:

@ILMostro
I am not a BACKUP=RSYNC user but I notice
it seems there are errors in your etc/rear/local.conf

As far as I know OUTPUT=BACKUP is not supported in ReaR,
see "man rear"
https://github.com/rear/rear/blob/master/doc/rear.8.adoc

BACKUP_ONLY_EXCLUDE works different,
see usr/share/rear/conf/default.conf
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf

To the actual issue:

In usr/share/rear/backup/RSYNC/default/450_calculate_req_space.sh
https://github.com/rear/rear/blob/master/usr/share/rear/backup/RSYNC/default/450_calculate_req_space.sh
you get the $RSYNC_PROTO=ssh case.

When you know you have sufficient space you could as another workaround
perhaps simply skip that script (add a return 0 line at its beginning).

Alternatively you may switch to the $RSYNC_PROTO=rsync case
by using a bit different BACKUP_URL syntax as described in the comments of
usr/share/rear/prep/RSYNC/default/100_check_rsync.sh
https://github.com/rear/rear/blob/master/usr/share/rear/prep/RSYNC/default/100_check_rsync.sh
that reads (excerpt):

#### OLD STYLE:
# BACKUP_URL=[USER@]HOST:PATH           # using ssh (no rsh)
#
# with rsync protocol PATH is a MODULE name defined in remote /etc/rsyncd.conf file
# BACKUP_URL=[USER@]HOST::PATH          # using rsync
# BACKUP_URL=rsync://[USER@]HOST[:PORT]/PATH    # using rsync (is not compatible with new style!!!)

#### NEW STYLE:
# BACKUP_URL=rsync://[USER@]HOST[:PORT]/PATH    # using ssh
# BACKUP_URL=rsync://[USER@]HOST[:PORT]::/PATH  # using rsync

But that is all from plain looking at the files.
I am not a BACKUP=RSYNC user so I could be wrong.

jsmeix commented at 2019-09-10 15:05:

@ILMostro
I assume your thumbs up emoji at
https://github.com/rear/rear/issues/2221#issuecomment-527521698
means the issue is sufficiently solved for you so that I close it accordingly.

If you find a more reliably working way how to determine the available disk space
we would appreciate a GitHub pull request with such an enhancement from you.

[Export of Github issue for rear/rear.]