#1160 Issue closed: Support multiple backups also for BACKUP_URL=usb¶
Labels: enhancement, fixed / solved / done
jsmeix opened issue at 2017-01-10 15:31:¶
With current ReaR 2.0 using multiple backups
is not supported for things like
OUTPUT=USB USB_DEVICE=/dev/disk/by-label/REAR-000 BACKUP_URL=usb:///dev/disk/by-label/REAR-000
because one gets the backups in different directories
on the REAR-000 medium, cf.
https://github.com/rear/rear/issues/1158#issuecomment-271604297
which is essentially the same issue as in
https://github.com/rear/rear/issues/1145
In contrast to support for advanced backup features
like BACKUP_TYPE=incremental/differential
I am much more interested in getting support
for multiple backups also for BACKUP_URL=usb.
jsmeix commented at 2017-01-10 15:56:¶
@dwerner1
currently I am playing around with this change in
usr/share/rear/prep/USB/default/060_set_usb_device.sh
(excerpt)
# Set USB_PREFIX fallback if not specified: test "$USB_PREFIX" || USB_PREFIX="rear/$HOSTNAME/$(date +%Y%m%d.%H%M)"
and then in etc/rear/local.conf (excerpt)
USB_PREFIX="rear/$HOSTNAME"
which results after
usr/sbin/rear -C basic_system -d -D mkbackup usr/sbin/rear -C home_backup -d -D mkbackuponly
this on the REAR-000 medium (excerpt)
rear/e205 rear/e205/kernel rear/e205/backup-basic_system.log rear/e205/backup-basic_system.tar.gz rear/e205/syslinux.cfg rear/e205/backup-home_backup.tar.gz rear/e205/initrd.cgz rear/e205/backup-home_backup.log rear/e205/rear-e205-mkbackup-basic_system-12222.log
dwerner1 commented at 2017-01-10 16:04:¶
That looks very interesting! I have just read a bit deeper this document
https://github.com/rear/rear/blob/master/doc/user-guide/11-multiple-backups.adoc
.. which I have just been flying over before. The idea of splitting the environment in basic, home, opt etc is doubt free a tempting and reasonable way!
jsmeix commented at 2017-01-10 16:07:¶
Currently with the above "rear recover" fails
with the already known issue
RESCUE e205:~ # rear -C basic_system -d -D recover
Relax-and-Recover 2.00 / Git
Using log file: /var/log/rear/rear-e205.log
730: Sourcing additional configuration file '/etc/rear/basic_system.conf'
730: Running workflow recover within the ReaR rescue/recovery system
730: Using backup archive 'backup-basic_system.tar.gz'
730: NOTICE: Will do driver migration
730: Select a backup archive.
++ select choice in '"${backup_times[@]}"' '"Abort"'
1) Abort
#?
but when I put a plain
return
as very fist line in
verify/USB/NETFS/default/540_choose_backup_archive.sh
both
RESCUE e205:~ # rear -C basic_system -d -D recover
and a subseqent
RESCUE e205:~ # rear -C home_backup -d -D restoreonly
just work for me :-)
This means that from my current point
the issue is already basically solved.
jsmeix commented at 2017-01-10 16:08:¶
Tomorrow I will make a GitHub pull request...
jsmeix commented at 2017-01-11 10:26:¶
Does not yet work for "mkrescue" because e.g. after
usr/sbin/rear -C basic_system -d -D mkrescue
the backups are gone and one has on the REAR-000 medium
only a new recovery system but without the old backups:
rear/e205/backup-basic_system.log rear/e205/initrd.cgz rear/e205/kernel rear/e205/rear-e205-mkrescue-basic_system-20547.log rear/e205/syslinux.cfg
cf. what is was before in
https://github.com/rear/rear/issues/1160#issuecomment-271613872
jsmeix commented at 2017-01-11 10:29:¶
Same problem after a subsequent run of
usr/sbin/rear -C basic_system -d -D mkbackup
which removes all existing old backups so that
afterwards the existing backup-home_backup.tar.gz
is no longer there.
Only the new backup-basic_system.tar.gz exists then.
jsmeix commented at 2017-01-11 15:34:¶
I think I found the reason why existing backups got deleted:
It is in default.conf USB_RETAIN_BACKUP_NR=2
plus the matching code for USB_RETAIN_BACKUP_NR in
output/USB/Linux-i386/300_create_extlinux.sh
cf.
https://github.com/rear/rear/issues/1158#issuecomment-271518391
I use now in local.conf
USB_RETAIN_BACKUP_NR=999
and then existing backups are not deleted.
jsmeix commented at 2017-01-12 09:47:¶
Summary what hacks make it work for me:
Code changes
(I run it in a current GitHub master code clone):
# git status ... modified: usr/share/rear/prep/USB/default/060_set_usb_device.sh modified: usr/share/rear/verify/USB/NETFS/default/540_choose_backup_archive.sh ... # git diff usr/share/rear/prep/USB/default/060_set_usb_device.sh ... @@ -9,7 +9,8 @@ ... -USB_PREFIX="rear/$HOSTNAME/$(date +%Y%m%d.%H%M)" +# Set USB_PREFIX fallback if not specified: +test "$USB_PREFIX" || USB_PREFIX="rear/$HOSTNAME/$(date +%Y%m%d.%H%M)" ... # git diff usr/share/rear/verify/USB/NETFS/default/540_choose_backup_archive.sh ... @@ -1,3 +1,4 @@ +return scheme=$(url_scheme "$BACKUP_URL") ...
Config settings:
OUTPUT=USB USB_DEVICE=/dev/disk/by-label/REAR-000 USB_PREFIX="rear/$HOSTNAME/fixedbackupdir" USB_RETAIN_BACKUP_NR=999 BACKUP=NETFS BACKUP_URL=usb:///dev/disk/by-label/REAR-000
USB_PREFIX must start with rear/$HOSTNAME
because in current code there are hardcoded places
where rear/$HOSTNAME is used. E.g. with plain
USB_PREFIX="mybackupdir" the ReaR recovery
system won't boot (syslinux setup goes wrong then).
Now I need to find a not too ugly way how to integrate that
into current ReaR master code in a backward compatible way,
then I will make a GitHub pull request...
In the meantime:
@dwerner1
please test how far my above described changes also make
it work for your particular use case and provide feedback here
what goes wrong and/or what is missing for your use case.
jsmeix commented at 2017-01-12 10:13:¶
The syslinux menu labels get ugly when things like
USB_PREFIX="rear/$HOSTNAME/fixedbackupdir"
is set, e.g. in my rear/e205/fixedbackupdir/syslinux.cfg
I have "menu label fixe-db-ac up:di"
label e205-fixedbackupdir
menu label fixe-db-ac up:di rescue image
say e205-fixedbackupdir - Recover e205 rescue image (fixedbackupdir)
...
label e205-fixedbackupdir
menu label fixe-db-ac up:di rescue image - AUTOMATIC RECOVER
say e205-fixedbackupdir - Recover e205 rescue image (fixedbackupdir)
The reason is that current code works under the assumtion
USB_PREFIX="rear/$HOSTNAME/$(date +%Y%m%d.%H%M)"
i.e. things like "rear/e205/20170110.1618" would result
"menu label 2017-01-10 16:18"
dwerner1 commented at 2017-01-12 12:08:¶
I can confirm that everything works as expected with current git version
and this etc/rear/local.conf:
BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
USB_PREFIX="rear/$HOSTNAME/fixedbackupdir"
USB_RETAIN_BACKUP_NR=999
BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA
In usr/share/rear/conf/default.conf I have set
USB_DEVICE_PARTED_LABEL=gpt
USB_DEVICE_FILESYSTEM_PERCENTAGE=5
jsmeix commented at 2017-01-12 13:31:¶
@dwerner1
many thanks for the prompt feedback!
For clarification:
I assume you meant that everything works as expected
with current git version plus my above described hacks.
I am astonished that even
BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA
works now because I never tested that.
Regarding changes in default.conf,
see the initial comment in default.conf:
# Here we define and describe all configuration variables and set them to a default. # # Do not change them here. Set them in your site.conf or local.conf file as needed.
i.e. set
USB_DEVICE_PARTED_LABEL=gpt USB_DEVICE_FILESYSTEM_PERCENTAGE=5
in your local.conf file and leave default.conf unchanged.
dwerner1 commented at 2017-01-12 13:43:¶
Sorry, I had not read your comment proper enough, I had not recognized that the changes in the files have to be done by me, I assumed they were already coming with the actual git code! I'll test again right away!
dwerner1 commented at 2017-01-12 15:03:¶
OK, with the current git version, your above mentioned changes and local.conf like
BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
USB_PREFIX="rear/$HOSTNAME/fixedbackupdir"
USB_RETAIN_BACKUP_NR=999
USB_DEVICE_PARTED_LABEL=gpt
USB_DEVICE_FILESYSTEM_PERCENTAGE=5
BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA
old copies are obviously overwritten
On partition /dev/sdb1 labeled REAR-000, the folder
/mnt/rear/debianrear/fixedbackupdir looks like
drwx------ 2 root root 4096 Jan 12 15:40 .
drwx------ 3 root root 4096 Jan 12 15:40 ..
-rw-r--r-- 1 root root 72284660 Jan 12 15:48 initrd.cgz
-rw-r--r-- 1 root root 4153384 Dec 3 16:17 kernel
-rw------- 1 root root 5986309 Jan 12 15:48 rear-debianrear.log
-rw------- 1 root root 951 Jan 12 15:48 syslinux.cfg
On partition /dev/sdb2 labeled REAR-DATA, the folder
/mnt/rear/debianrear/fixedbackupdir looks like
drwxr-x--- 2 root root 45 Jan 12 15:51 .
drwx------ 3 root root 28 Jan 12 15:40 ..
-rw------- 1 root root 8737325 Jan 12 15:51 backup.log
-rw------- 1 root root 1343316345 Jan 12 15:51 backup.tar.gz
after having done rear -d -D mkbackup 3 times
dwerner1 commented at 2017-01-12 15:04:¶
Ah, not overwritten, more deleted ..
jsmeix commented at 2017-01-12 15:34:¶
With things like
USB_PREFIX="rear/$HOSTNAME/fixedbackupdir"
existing backup.tar.gz files get overwritten
(i.e. replaced by a new backup.tar.gz file), cf.
"Relax-and-Recover versus backup and restore" in
https://en.opensuse.org/SDB:Disaster_Recovery
that describes this behaviour for BACKUP_URL=nfs
This is the intended bahaviour when using one single
fixed directory where the backup gets written to.
This behaviour is the same as when you use
BACKUP_URL=nfs://nfs_server_IP/path
This behaviour is a prerequirement both for using
multiple backups and incremental/differential backups
also with BACKUP_URL=usb because now one
can have all backup files in one same directory
also with BACKUP_URL=usb.
I.e. now one can make it with BACKUP_URL=usb
work the same as with BACKUP_URL=nfs where
multiple backups and incremental/differential backups
do already work.
In both cases all backups must be in one same directory,
cf. my comment regarding BACKUP_TYPE in default.conf.
When you neither use multiple backups nor
incremental/differential backups then this feature is not useful
for you - in this case leave the defaults and do not set
USB_PREFIX="rear/$HOSTNAME/fixedbackupdir"
But according to
https://github.com/rear/rear/issues/1141
I thought you like to use incremental/differential backups
with BACKUP_URL=usb, see
https://github.com/rear/rear/issues/1145#issuecomment-272129606
Summary:
Either
have separated full backups in separated timestamp directories
via the default settings for BACKUP_URL=usb
or
use multiple backups or incremental/differential backups
for BACKUP_URL=usb but then all backup files must be
in one same directory and then you need special additional
setup like USB_PREFIX="rear/$HOSTNAME/fixedbackupdir"
and USB_RETAIN_BACKUP_NR=999
dwerner1 commented at 2017-01-12 15:46:¶
I thought you like to use incremental/differential backups
with BACKUP_URL=usb, see
#1145 (comment)
Sorry, I had lost #1145 out of view! I'll change my setup and do further tests later on!
jsmeix commented at 2017-01-13 14:22:¶
With
https://github.com/rear/rear/pull/1165
merged this issue is fixed.
[Export of Github issue for rear/rear.]