#1066 PR merged: fixed and cleaned up some parts of incremental backup¶
Labels: enhancement, bug, cleanup, fixed / solved / done
jsmeix opened issue at 2016-11-11 13:01:¶
in particular basically rewrote most parts of
prep/NETFS/default/070_set_backup_archive.sh
and adapted messaging in case of incremental backup
(mainly avoid false information by simpler texts)
See
https://github.com/rear/rear/issues/1062
gdha commented at 2016-11-11 13:18:¶
@jsmeix did you test if the pull request works well? It is a bit too much to overview by looking at it. If you are sure it is ok you may merge the pull request. Thanks Johannes.
jsmeix commented at 2016-11-11 13:25:¶
@gdha
it does not yet "work well" but it works
already "better" than before for me.
It does not yet work o.k. initially (this is no regression, cf.
https://github.com/rear/rear/issues/1062#issuecomment-259394607
where after the initial full backup there are no timestamp.txt
and basebackup.txt files in the NFS backup directory).
I think meanwhile I know the reason:
When initially prep/NETFS/default/070_set_backup_archive.sh
is run the commands that should write timestamp.txt
and basebackup.txt files in the NFS backup directory
do not work correctly - but I will have to analyze that
next week...
Bottom line:
One might merge it right now because it should improve things
but it is not yet really solved - therefore I would wait until I got
it
sufficiently solved next week.
jsmeix commented at 2016-11-11 13:56:¶
Except the initial startup phase incremental backup
works now very well for me.
For me the only issue is that initially one gets two
full backups but that is only a minor annoyance.
On my original SLES12-SP2 test system:
# grep -v ^# etc/rear/local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://10.160.4.244/nfs
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" snapper chattr lsattr )
COPY_AS_IS=( "${COPY_AS_IS[@]}" /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
BACKUP_PROG_INCLUDE=( '/var/cache/*' '/var/lib/mailman/*' '/var/tmp/*' '/var/lib/pgsql/*' '/usr/local/*' '/opt/*' '/var/lib/libvirt/images/*' '/boot/grub2/i386/*' '/var/opt/*' '/srv/*' '/boot/grub2/x86_64/*' '/var/lib/mariadb/*' '/var/spool/*' '/var/lib/mysql/*' '/tmp/*' '/home/*' '/var/log/*' '/var/lib/named/*' '/var/lib/machines/*' )
POST_RECOVERY_SCRIPT=( 'if snapper --no-dbus -r $TARGET_FS_ROOT get-config | grep -q "^QGROUP.*[0-9]/[0-9]" ; then snapper --no-dbus -r $TARGET_FS_ROOT set-config QGROUP= ; snapper --no-dbus -r $TARGET_FS_ROOT setup-quota && echo snapper setup-quota done || echo snapper setup-quota failed ; else echo snapper setup-quota not used ; fi' )
SSH_ROOT_PASSWORD="rear"
USE_DHCLIENT="yes"
KEEP_BUILD_DIR=""
BACKUP_TYPE="incremental"
FULLBACKUPDAY="Wed"
# usr/sbin/rear -d -D mkbackup
Relax-and-Recover 1.19 / Git
Using log file: /root/rear/var/log/rear/rear-d108.log
No full backup date file (timestamp.txt) found - triggers full backup
Performing full backup (/tmp/rear.oK2UiYsMECeMUbQ/outputfs/d108/2016-11-11-1413-F.tar.gz)
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /root/rear/var/lib/rear/output/rear-d108.iso (150M)
Copying resulting files to nfs location
Encrypting disabled
Creating tar archive '/tmp/rear.oK2UiYsMECeMUbQ/outputfs/d108/2016-11-11-1413-F.tar.gz'
Archived 826 MiB [avg 7627 KiB/sec]OK
Archived 826 MiB in 112 seconds [avg 7559 KiB/sec]
# usr/sbin/rear -d -D mkbackup
Relax-and-Recover 1.19 / Git
Using log file: /root/rear/var/log/rear/rear-d108.log
No full backup date file (timestamp.txt) found - triggers full backup
Performing full backup (/tmp/rear.vgoI6le63tFhBll/outputfs/d108/2016-11-11-1437-F.tar.gz)
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /root/rear/var/lib/rear/output/rear-d108.iso (150M)
Copying resulting files to nfs location
Encrypting disabled
Creating tar archive '/tmp/rear.vgoI6le63tFhBll/outputfs/d108/2016-11-11-1437-F.tar.gz'
Archived 825 MiB [avg 7827 KiB/sec]OK
Archived 825 MiB in 109 seconds [avg 7755 KiB/sec]
# usr/sbin/rear -d -D mkbackup
Relax-and-Recover 1.19 / Git
Using log file: /root/rear/var/log/rear/rear-d108.log
Full backup files found (timestamp.txt, basebackup.txt, 2016-11-11-1437-F.tar.gz) - doing incremental backup
Performing incremental backup (/tmp/rear.Jm3sdhGUoXWRcTO/outputfs/d108/2016-11-11-1442-I.tar.gz)
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /root/rear/var/lib/rear/output/rear-d108.iso (150M)
Copying resulting files to nfs location
Encrypting disabled
Creating tar archive '/tmp/rear.Jm3sdhGUoXWRcTO/outputfs/d108/2016-11-11-1442-I.tar.gz'
Archived 0 MiB [avg 152 KiB/sec]OK
Archived 0 MiB in 5 seconds [avg 121 KiB/sec]
On the NFS server:
# ls -lh d108 total 1.8G ... 827M Nov 11 14:16 2016-11-11-1413-F.tar.gz ... 827M Nov 11 14:40 2016-11-11-1437-F.tar.gz ... 1.8M Nov 11 14:42 2016-11-11-1442-I.tar.gz ... 202 Nov 11 14:42 README ... 262 Nov 11 14:42 VERSION ... 12M Nov 11 14:42 backup.log ... 25 Nov 11 14:37 basebackup.txt ...150M Nov 11 14:42 rear-d108.iso ... 8.5M Nov 11 14:42 rear.log ... 11 Nov 11 14:37 timestamp.txt # cat d108/basebackup.txt 2016-11-11-1437-F.tar.gz # cat d108/timestamp.txt 2016-11-11
On the second system for recovery:
# rear -d -D recover
Relax-and-Recover 1.19 / Git
Using log file: /var/log/rear/rear-d108.log
Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
Started RPC portmapper 'rpcbind'.
RPC portmapper 'rpcbind' available.
Started rpc.statd.
RPC status rpc.statd available.
Full backup files found (timestamp.txt, basebackup.txt, 2016-11-11-1437-F.tar.gz) - doing incremental backup
Performing incremental backup (/tmp/rear.XChlrCUEINgTPNw/outputfs/d108/2016-11-11-1346-I.tar.gz)
NOTICE: Will do driver migration
Comparing disks.
Disk configuration is identical, proceeding with restore.
Doing SLES12 special btrfs subvolumes setup because the default subvolume path contains '@/.snapshots/'
Start system layout restoration.
Creating partitions for disk /dev/sda (msdos)
Creating LVM PV /dev/sda1
0 logical volume(s) in volume group "system" now active
Restoring LVM VG system
Sleeping 3 seconds to let udev or systemd-udevd create their devices...
Creating filesystem of type btrfs with mount point / on /dev/mapper/system-root.
/dev/mapper/system-root: 8 bytes were erased at offset 0x00010040 (btrfs): 5f 42 48 52 66 53 5f 4d
btrfs-progs v4.5.3+20160729
See http://btrfs.wiki.kernel.org for more information.
Performing full device TRIM (18.54GiB) ...
Label: (null)
UUID: 65065716-fab4-4277-8adb-d972d513610e
Node size: 16384
Sector size: 4096
Filesystem size: 18.54GiB
Block group profiles:
Data: single 8.00MiB
Metadata: DUP 1.01GiB
System: DUP 12.00MiB
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 18.54GiB /dev/mapper/system-root
Mounting filesystem /
Create subvolume '/mnt/local/@'
Create subvolume '/mnt/local//@/boot/grub2/i386-pc'
Create subvolume '/mnt/local//@/boot/grub2/x86_64-efi'
Create subvolume '/mnt/local//@/home'
Create subvolume '/mnt/local//@/opt'
Create subvolume '/mnt/local//@/srv'
Create subvolume '/mnt/local//@/tmp'
Create subvolume '/mnt/local//@/usr/local'
Create subvolume '/mnt/local//@/var/cache'
Create subvolume '/mnt/local//@/var/crash'
Create subvolume '/mnt/local//@/var/lib/libvirt/images'
Create subvolume '/mnt/local//@/var/lib/machines'
Create subvolume '/mnt/local//@/var/lib/mailman'
Create subvolume '/mnt/local//@/var/lib/mariadb'
Create subvolume '/mnt/local//@/var/lib/mysql'
Create subvolume '/mnt/local//@/var/lib/named'
Create subvolume '/mnt/local//@/var/lib/pgsql'
Create subvolume '/mnt/local//@/var/log'
Create subvolume '/mnt/local//@/var/opt'
Create subvolume '/mnt/local//@/var/spool'
Create subvolume '/mnt/local//@/var/tmp'
Running snapper/installation-helper:
step 1 device:/dev/mapper/system-root
temporarily mounting device
copying/modifying config-file
creating filesystem config
creating snapshot
setting default subvolume
done
Creating swap on /dev/mapper/system-swap
Disk layout created.
Decrypting disabled
First restoring full backup tar archive from '/tmp/rear.XChlrCUEINgTPNw/outputfs/d108/2016-11-11-1437-F.tar.gz'
Then restoring incremental backup tar archive from '/tmp/rear.XChlrCUEINgTPNw/outputfs/d108/2016-11-11-1442-I.tar.gz'
Restoring...OK
Restored full backup and incremental backup in 32 seconds
Restore the Mountpoints (with permissions) from /var/lib/rear/recovery/mountpoint_permissions
Patching file 'boot/grub2/grub.cfg'
Patching file 'boot/grub2/device.map'
Patching file 'etc/sysconfig/bootloader'
Patching file 'etc/fstab'
Patching file 'etc/mtools.conf'
Patching file 'etc/smartd.conf'
Patching file 'etc/sysconfig/smartmontools'
Patching file 'etc/security/pam_mount.conf.xml'
Installing GRUB2 boot loader
snapper setup-quota done
Finished recovering your system. You can explore it under '/mnt/local'.
jsmeix commented at 2016-11-14 13:36:¶
In prep/NETFS/default/070_set_backup_archive.sh
writing timestamp.txt and basebackup.txt fails during
the very first "rear mkbackup" because there is not yet
the HOSTNAME directory '.../outputfs/d108' that gets
created much later by output/default/200_make_prefix_dir.sh
+ source /root/rear/usr/share/rear/prep/NETFS/default/070_set_backup_archive.sh ... ++ echo 2016-11-14 /root/rear/usr/share/rear/prep/NETFS/default/070_set_backup_archive.sh: line 147: /tmp/rear.7DJp8pjbKtWca7u/outputfs/d108/timestamp.txt: No such file or directory ++ echo 2016-11-14-1346-F.tar.gz /root/rear/usr/share/rear/prep/NETFS/default/070_set_backup_archive.sh: line 148: /tmp/rear.7DJp8pjbKtWca7u/outputfs/d108/basebackup.txt: No such file or directory . . . + source /root/rear/usr/share/rear/output/default/200_make_prefix_dir.sh ... ++ mkdir -p -v -m0750 /tmp/rear.7DJp8pjbKtWca7u/outputfs/d108 mkdir: created directory '/tmp/rear.7DJp8pjbKtWca7u/outputfs/d108'
jsmeix commented at 2016-11-15 13:16:¶
I fixed that during initial incremental backup (i.e. for the first full
backup) timestamp.txt and basebackup.txt get stored in the backup
directory
by writing them in prep/NETFS/default/070_set_backup_archive.sh
only to TMP_DIR and later copy them to the backup directory
in output/default/950_copy_result_files.sh
This makes the current incremental backup implementation
even more a dirty hack (several special case handling for it
spread over several scripts) but for now (i.e. within the current
ReaR framework) it looks o.k. for me.
In its current state incremental backup works well for me.
It even works somewhat fail-safe together with an
intermixed non-incremental backup run so that
a single backup.tar.gz contains a full backup:
If a backup.tar.gz exists in the backup directory
that one is used for restore regardless of any
BACKUP_TYPE="incremental" setting and
regardless whether or not there are also
incremental backups in the backup directory
which are newer than the backup.tar.gz file.
After a simple
# mv backup.tar.gz backup.tar.gz.outdated
in the backup directory and if needed setting
BACKUP_TYPE="incremental"
in /etc/rear/local.conf in the recovery system
"rear recover" restores the latest full backup of the
incremental backups and the latest actually incremental backup.
jsmeix commented at 2016-11-15 13:17:¶
Because for me it works now sufficiently well, I will merge it...
jsmeix commented at 2016-11-15 13:19:¶
FYI:
While I adapted output/default/950_copy_result_files.sh
for incremental backup I detected
https://github.com/rear/rear/issues/1068
[Export of Github issue for rear/rear.]