#1166 Issue closed: Possibly useless backup selection when backup restore from USB¶
Labels: enhancement, bug, cleanup, fixed / solved / done
jsmeix opened issue at 2017-01-13 14:16:¶
From my experience with "rear recover" at
https://github.com/rear/rear/pull/1165#issuecomment-272449196
I use
OUTPUT=USB USB_DEVICE=/dev/disk/by-label/REAR-000 BACKUP=NETFS BACKUP_URL=usb:///dev/disk/by-label/REAR-000
I did several (5) times "rear mkbackup" and got
(with the default USB_RETAIN_BACKUP_NR=2)
on the REAR-000 medium three backup directories
rear/e205/20170113.1422 rear/e205/20170113.1422/kernel rear/e205/20170113.1422/rear-e205.log rear/e205/20170113.1422/syslinux.cfg rear/e205/20170113.1422/backup.log rear/e205/20170113.1422/initrd.cgz rear/e205/20170113.1422/backup.tar.gz rear/e205/20170113.1429 rear/e205/20170113.1429/kernel rear/e205/20170113.1429/rear-e205.log rear/e205/20170113.1429/syslinux.cfg rear/e205/20170113.1429/backup.log rear/e205/20170113.1429/initrd.cgz rear/e205/20170113.1429/backup.tar.gz rear/e205/20170113.1442 rear/e205/20170113.1442/kernel rear/e205/20170113.1442/rear-e205.log rear/e205/20170113.1442/syslinux.cfg rear/e205/20170113.1442/backup.log rear/e205/20170113.1442/initrd.cgz rear/e205/20170113.1442/backup.tar.gz
During boot I selected in the syslinux boot menue
2017-01-13 14:29 backup
Then I run "rear -d -D recover" which results (excerpts):
Running workflow recover within the ReaR rescue/recovery system Using backup archive 'backup.tar.gz' ... Select a backup archive. 1) 20170113.1422 2) 20170113.1429 3) 20170113.1442 4) Abort #? 3 ... 2017-01-13 13:52:09.209884020 Backup archive /tmp/rear.9fahNJ3x3Dm5i43/outputfs/rear/e205/20170113.1442/backup.tar.gz chosen. ... Calculating backup archive size Backup archive size is 992M /tmp/rear.9fahNJ3x3Dm5i43/outputfs/rear/e205/20170113.1429/backup.tar.gz (compressed) ... Disk layout created. Restoring from '/tmp/rear.9fahNJ3x3Dm5i43/outputfs/rear/e205/20170113.1429/backup.tar.gz'... Backup restore program 'tar' started in subshell (PID=2148) Restored 344 MiB [avg. 117665 KiB/sec] ... Restored 2465 MiB [avg. 114745 KiB/sec] OK Restored 2506 MiB in 25 seconds [avg. 102656 KiB/sec] Restoring finished.
In short:
I got the backup restored that matches what
I selected during boot in the syslinux menu
and not the backup that I selected during "rear recover".
I wonder why there is that backup selection
during "rear recover" when the right backup is
already specified during boot in the syslinux menu?
If there is a reason for the backup selection
during "rear recover" it seems it does not work.
@gdha
do you know the reason for the backup selection
during "rear recover" in case of backup on USB?
jsmeix commented at 2017-01-13 14:19:¶
FYI:
I did not change the backup selection during "rear recover" in
verify/USB/NETFS/default/540_choose_backup_archive.sh
in my recent
https://github.com/rear/rear/pull/1165
jsmeix commented at 2017-01-13 14:36:¶
I think I know why the backup selection during "rear recover" in
verify/USB/NETFS/default/540_choose_backup_archive.sh
does no longer work:
"usr/sbin/rear -s recover" shows (excerpt)
Source verify/NETFS/default/070_set_backup_archive.sh ... Source verify/USB/NETFS/default/540_choose_backup_archive.sh
The right backup is set as RESTORE_ARCHIVES in
verify/NETFS/default/070_set_backup_archive.sh
but later
verify/USB/NETFS/default/540_choose_backup_archive.sh
does not adapt RESTORE_ARCHIVES so that it has no effect.
This means the issue is a regression since my
https://github.com/rear/rear/pull/1071
https://github.com/rear/rear/issues/1069
I need to implement RESTORE_ARCHIVES in
verify/USB/NETFS/default/540_choose_backup_archive.sh
provided there is a real reason for another backup selection
during "rear recover" regardless that the right backup
should have already specified during boot in the syslinux menu.
Furthermore that backup selection during "rear recover"
makes it impossible to run "rear recover" unattended
in case of backup from USB.
If at all that backup selection during "rear recover" should
only hapen if RESTORE_ARCHIVES is empty or
its content cannot be found on the REAR-000 medium.
gozora commented at 2017-01-13 14:59:¶
Hello @jsmeix ,
I've used something like
this
but it is more an workaround....
jsmeix commented at 2017-01-13 15:08:¶
Yes, that is the basic idea but if I implement
RESTORE_ARCHIVES support in
verify/USB/NETFS/default/540_choose_backup_archive.sh
I would implement support to select multiple archives
because restore of multiple archives just works now
with RESTORE_ARCHIVES.
jsmeix commented at 2017-01-13 15:23:¶
The code in
verify/USB/NETFS/default/540_choose_backup_archive.sh
looks wrong because it sets "backuparchive"
and not "restorearchive" what it was in the old code
and what I have completely replaced by RESTORE_ARCHIVES.
But that whole old code about "backuparchive" looks so ugly
because in
restore/NETFS/default/400_restore_backup.sh
there is
# The RESTORE_ARCHIVES array contains the restore input files. # If it is not set, RESTORE_ARCHIVES is only one element which is the backup archive: test "$RESTORE_ARCHIVES" || RESTORE_ARCHIVES=( "$backuparchive" )
What a mess of badly named and badly used variables!
jsmeix commented at 2017-01-13 15:24:¶
I guess I need to rework the whole old code about "backuparchive"
and replace it with something that at least I can understand
to avoid more such regressions in the future :-(
jsmeix commented at 2017-01-16 11:43:¶
In contrast to what I wrote last week above in
https://github.com/rear/rear/issues/1166#issuecomment-272469071
I think now that the "backuparchive" usage and implementation
is actually o.k.- only the variable name should be uppercase, cf.
https://github.com/rear/rear/pull/1165#issuecomment-272814988
jsmeix commented at 2017-01-16 11:51:¶
I think meanwhile I understand why that backup selection
dialog during "rear recover" is there:
It is not needed when one uses "rear mkbackup"
where kernel, initrd, and backup are in one same directory
but it is needed when one runs separated
"rear mkrescue" and "rear mkbackuponly"
because then by default on USB kernel and initrd
are in one timestamp directory and the backup
is in another timestamp directory.
Because by default on USB there are more than one
backup (via USB_RETAIN_BACKUP_NR) there should
be a dialog during "rear recover" where the user can
select what backup he likes to restore.
But I think that backup selection dialog should behave
more user friendly and preselect the latest backup
that can be found on the USB medium so that the user
could just hit [Enter] to get the latest backup restored.
When only one backup is found on the USB medium
there should not be such a dialog - just that one backup
gets restored - because what else could the user do?
jsmeix commented at 2017-01-16 12:03:¶
For me it does not work with separated
"rear mkrescue" and "rear mkbackuponly":
I did two times
"rear mkrescue" and "rear mkbackuponly"
so that I have now on my REAR-000 medium
(cf.
https://github.com/rear/rear/issues/1166#issue-200640200
what I had initially):
rear/e205/20170116.1230 rear/e205/20170116.1230/backup.log rear/e205/20170116.1230/backup.tar.gz rear/e205/20170116.1227 rear/e205/20170116.1227/kernel rear/e205/20170116.1227/rear-e205.log rear/e205/20170116.1227/syslinux.cfg rear/e205/20170116.1227/initrd.cgz rear/e205/20170113.1442 rear/e205/20170113.1442/kernel rear/e205/20170113.1442/rear-e205.log rear/e205/20170113.1442/syslinux.cfg rear/e205/20170113.1442/backup.log rear/e205/20170113.1442/initrd.cgz rear/e205/20170113.1442/backup.tar.gz rear/e205/20170116.1219 rear/e205/20170116.1219/kernel rear/e205/20170116.1219/rear-e205.log rear/e205/20170116.1219/syslinux.cfg rear/e205/20170116.1219/initrd.cgz rear/e205/20170116.1225 rear/e205/20170116.1225/backup.log rear/e205/20170116.1225/backup.tar.gz
Summary what timestamps belong to what action:
...1442 mkbackup
...1219 mkrescue
...1225 mkbackuponly
...1227 mkrescue
...1230 mkbackuponly
The syslinux.cfg files contain those menu labels:
rear/e205/20170113.1442/syslinux.cfg: menu label 2017-01-13 14:42 backup rear/e205/20170113.1442/syslinux.cfg: menu label 2017-01-13 14:42 backup - AUTOMATIC RECOVER rear/e205/20170116.1219/syslinux.cfg: menu label 2017-01-16 12:19 rescue image rear/e205/20170116.1219/syslinux.cfg: menu label 2017-01-16 12:19 rescue image - AUTOMATIC RECOVER rear/e205/20170116.1227/syslinux.cfg: menu label 2017-01-16 12:27 rescue image rear/e205/20170116.1227/syslinux.cfg: menu label 2017-01-16 12:27 rescue image - AUTOMATIC RECOVER
For recovery I booted the syslinux menu label
"2017-01-16 12:19 rescue image"
Then in the ReaR recovery system it fails as follows:
Using backup archive 'backup.tar.gz' NOTICE: Will do driver migration Select a backup archive. 1) 20170113.1442 2) 20170116.1225 3) 20170116.1230 4) Abort #? 3 Backup archive /tmp/rear.dMag4O5WJ908yfk/outputfs/rear/e205/20170116.1230/backup.tar.gz chosen. ERROR: Backup archive 'backup.tar.gz' not found. Aborting due to an error, check /var/log/rear/rear-e205.log for details Terminated
The
ERROR: Backup archive 'backup.tar.gz' not found.
message comes from
verify/NETFS/default/550_check_backup_archive.sh
"rear -s recover" shows:
Source verify/NETFS/default/070_set_backup_archive.sh ... Source verify/USB/NETFS/default/540_choose_backup_archive.sh Source verify/NETFS/default/550_check_backup_archive.sh
I will make
verify/USB/NETFS/default/540_choose_backup_archive.sh
behave well...
jsmeix commented at 2017-01-16 12:39:¶
Initial bug is in 070_set_backup_archive.sh that blindly sets
++ backuparchive=/tmp/rear.B5LcS9D1ZfCXfJs/outputfs/rear/e205/20170116.1219/backup.tar.gz
++ RESTORE_ARCHIVES=("$backuparchive")
regardless that there is no such file
(the "1219" directory is from a "mkrescue")
which then leads to 550_check_backup_archive.sh
not setting RESTORE_ARCHIVES to the right backuparchive
that is specified during 540_choose_backup_archive
via
test "$RESTORE_ARCHIVES" || RESTORE_ARCHIVES=( "$backuparchive" )
because RESTORE_ARCHIVES is already set.
Accordingly this small change in 070_set_backup_archive.sh
makes "rear recover" works for me in this case:
@@ -38,8 +38,12 @@ local backup_directory=$BUILD_DIR/outputfs/$NETFS_PREFIX
if ! test "incremental" = "$BACKUP_TYPE" -o "differential" = "$BACKUP_TYPE" ; then
backuparchive="$backup_directory/$backup_file_name"
# In case of normal (i.e. non-incremental) backup there is only one restore archive
- # and its name is the same as the backup archive (usually 'backup.tar.gz'):
- RESTORE_ARCHIVES=( "$backuparchive" )
+ # and its name is the same as the backup archive (usually 'backup.tar.gz').
+ # Only set RESTORE_ARCHIVES for "rear recover" when the backup archive is accessible
+ # cf. https://github.com/rear/rear/issues/1166
+ test -r "$backuparchive" && RESTORE_ARCHIVES=( "$backuparchive" )
+ # Do not error out here when backup archive is not accessible because that same code
+ # is also run during "rear mkbackup" where usually the backup archive does not yet exist.
LogPrint "Using backup archive '$backup_file_name'"
return
fi
gdha commented at 2017-01-16 13:04:¶
@jsmeix USB code was designed and written by @dagwieers and @jhoekx for their special customer. It always used to work as it was designed. I cannot remember I changed lot of code in that area...
jsmeix commented at 2017-01-16 13:59:¶
@gdha
many thanks for the information that this code was made
as a special case (hack) for a special customer.
I will make it working reasonably well also for other users
because that functionality is now there in ReaR
so that it should work reasonably well in general.
jsmeix commented at 2017-01-17 10:42:¶
Only a side note (I won't fix that):
It seems the default USB_RETAIN_BACKUP_NR=2
does not strictly work this way.
Continuing from the state what I had
on my REAR-000 medium as shown in
https://github.com/rear/rear/issues/1166#issuecomment-272845326
I did one more "rear mkrescue" (timestamp "...1447")
then a "rear mkbackup" (timestamp "...1057") and
a "rear mkbackuponly" (timestamp "...1106") and
now I have those files on my REAR-000 medium
rear/e205/20170116.1230 rear/e205/20170116.1230/backup.log rear/e205/20170116.1230/backup.tar.gz rear/e205/20170116.1447 rear/e205/20170116.1447/kernel rear/e205/20170116.1447/rear-e205.log rear/e205/20170116.1447/syslinux.cfg rear/e205/20170116.1447/initrd.cgz rear/e205/20170116.1225 rear/e205/20170116.1225/backup.log rear/e205/20170116.1225/backup.tar.gz rear/e205/20170117.1106 rear/e205/20170117.1106/backup.log rear/e205/20170117.1106/backup.tar.gz rear/e205/20170117.1057 rear/e205/20170117.1057/kernel rear/e205/20170117.1057/rear-e205.log rear/e205/20170117.1057/syslinux.cfg rear/e205/20170117.1057/backup.log rear/e205/20170117.1057/initrd.cgz rear/e205/20170117.1057/backup.tar.gz
so that I have now 4 backups and two recovery systems.
jsmeix commented at 2017-01-17 15:45:¶
With
https://github.com/rear/rear/pull/1168
merged, this issue should be sufficiently fixed.
[Export of Github issue for rear/rear.]