#2168 Issue `closed`: BORG: Unable to exclude swapfile from backup¶

Labels: support / question, fixed / solved / done

gaia opened issue at 2019-06-29 19:36:¶

Relax-and-Recover 2.5 / 2019-05-10
Debian GNU/Linux 9.9 (stretch)
cat /etc/rear/local.conf: see below
Intel X86_64 bare metal
UEFI with Grub2
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe): see below

Issue

I cannot exclude the swapfile from the backup. I've tried different ways of setting the exclude (single/double quotes, exclude by mountpoint/components/device, relative/absolute path).

Swap is mounted as /mnt/storage/swapfile on /dev/mapper/lvmt_storage1-lvmt_storage1. The USB disk where the backups are stored is /dev/sdc on /mnt/usb/. I've tried to run ReaR with the drive mounted and unmounted.

The only item I want to backup is the root partition to the USB drive, with no mountpoints and some standard exclusions. The only item I would want to restore is also the root partition, leaving the other filesystems intact. I feel this is a good strategy of backup size vs safety, as the contents of /zfs_containers2 are on RAID 1 and will be backed up to /dev/mapper/lvmt_storage1-lvmt_storage1 (also RAID 1) and the contents of /dev/mapper/lvmt_containers1-lvmt_containers1 are also backed up to /dev/mapper/lvmt_storage1-lvmt_storage1.

What do I need to do to exclude the swapfile? I am under the impression that nothing is actually being excluded, as /dev/mapper/lvmt_storage1-lvmt_storage1 and /zfs_containers2 are still empty. The only reason I know the swapfile is being included is because it takes a minute to compress the 32GB before backing it up.

Workaround, if any: None that I am aware of
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

# cat local.conf

USB_DEVICE_PARTED_LABEL=gpt
USB_DEVICE_FILESYSTEM=ext4
USING_UEFI_BOOTLOADER=1

OUTPUT=USB
BACKUP=BORG
USB_DEVICE=/dev/disk/by-label/REAR-000
BORGBACKUP_REPO="backups"
BORGBACKUP_UMASK="0002"
BORGBACKUP_PRUNE_DAILY=7
BORGBACKUP_ENC_TYPE="keyfile"
BORGBACKUP_COMPRESSION="lz4"
BORGBACKUP_SHOW_PROGRESS="yes"
BORGBACKUP_SHOW_STATS="yes"
PROGS_BORG=( )
export BORG_PASSPHRASE="xxxxxxx"

export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

# EXCLUDE_COMPONENTS=( swap:mnt/storage/swapfile )
EXCLUDE_MOUNTPOINTS=( /zfs_containers2 )
EXCLUDE_MD=( /dev/md1 /dev/md1p1 )
EXCLUDE_VG=( /dev/mapper/lvmt_containers1-lvmt_containers1 /dev/mapper/lvmt_storage1-lvmt_storage1 /dev/disk/by-label/REAR-000 )

# Tried all of the following
COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" "home/*/.cache/*" "var/cache/*" "var/tmp/*" "dev/*" "media/*" "proc/*" "sys/*" "tmp/*" "var/run/*" "var/lock/*" "mnt/storage/swapfile" )
#COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" 'home/*/.cache/*' 'var/cache/*' 'var/tmp/*' 'dev/*' 'media/*' 'proc/*' 'sys/*' 'tmp/*' 'var/run/*' 'var/lock/*' 'mnt/storage/swapfile' )
#COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}"  '/mnt/storage/*' '/mnt/containers/*' '/mnt/usb/*' '/mnt/bindmounts/*' '/zfs_containers2/*')

COPY_AS_IS_BORG=( )

SSH_UNPROTECTED_PRIVATE_KEYS="yes"
SSH_FILES="yes"

# cat /var/lib/rear/layout/disklayout.conf

disk /dev/nvme0n1 2048408248320 unknown

disk /dev/sda 2000398934016 gpt

part /dev/sda 510656512 1048576 rear-noname boot,esp /dev/sda1
part /dev/sda 53687091200 511705088 rear-noname none /dev/sda2
part /dev/sda 161061273600 54198796288 Linux%20filesystem none /dev/sda3
part /dev/sda 1785138847232 215260069888 Linux%20RAID raid /dev/sda4

disk /dev/sdb 2000398934016 gpt
part /dev/sdb 510656512 1048576 rear-noname boot,esp /dev/sdb1
part /dev/sdb 53687091200 511705088 rear-noname raid /dev/sdb2
part /dev/sdb 161061273600 54198796288 Linux%20filesystem none /dev/sdb3
part /dev/sdb 1785138847232 215260069888 Linux%20RAID raid /dev/sdb4

raid /dev/md1 metadata=1.2 level=raid1 raid-devices=2 uuid=38538483:ffefcbb8:3cca39c8:811c0d15 name=1 devices=/dev/sda4,/dev/sdb4
part /dev/md1 1785003556352 1048576 rear-noname none /dev/md1p1
raid /dev/md0 metadata=1.2 level=raid1 raid-devices=2 uuid=ba8a642c:2deff1d1:2d4392b6:9afc60b1 name=0 devices=/dev/sda2,/dev/sdb2

lvmdev /dev/lvmt_storage1 /dev/md1p1 mWXwl0-CmLf-EbXw-2dS0-OrW2-R1Gs-50DRVa 3486335071
lvmdev /dev/lvmt_containers1 /dev/nvme0n1 VF2lWK-uLZ0-qQdt-NLXk-gdsj-Cc9E-umRtCj 4000797360

lvmgrp /dev/lvmt_storage1 4096 425577 1743163392
lvmgrp /dev/lvmt_containers1 4096 488378 2000396288

lvmvol /dev/lvmt_containers1 lvmt_containers1 2048154140672b thin,pool chunksize:1048576b
lvmvol /dev/lvmt_storage1 lvmt_storage1 1784781209600b thin,pool chunksize:1048576b

fs /dev/mapper/lvmt_containers1-lvmt_containers1 /mnt/containers ext4 uuid=b3dc563a-08ca-4cfd-9a18-3d82d6518306 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16383 default_mount_options=user_xattr,acl options=rw,relatime,stripe=256,data=ordered
fs /dev/mapper/lvmt_storage1-lvmt_storage1 /mnt/storage ext4 uuid=0f924dc4-4c1f-4722-aaed-8d48c3fc0651 label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16383 default_mount_options=user_xattr,acl options=rw,relatime,stripe=256,data=ordered
fs /dev/md0 / ext4 uuid=fe3e3b7f-e8fd-4a9e-ba7b-5f8947e62488 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16373 default_mount_options=user_xattr,acl options=rw,relatime,errors=remount-ro,data=ordered
fs /dev/sda1 /boot/efi vfat uuid=FE89-3DF9 label= options=rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro

# cat /etc/fstab

UUID=fe3e3b7f-e8fd-4a9e-ba7b-5f8947e62488 /               ext4    errors=remount-ro 0       1
UUID=FE89-3DF9  /boot/efi       vfat    umask=0077      0       1

# NVME SSD: LVM thin for containers
/dev/mapper/lvmt_containers1-lvmt_containers1 /mnt/containers ext4 defaults,nofail 0 0
#/dev/nvme0n1p1 auto mounts to /zfs_containers

# WD Red RAID 1: LVM thin for backups
/dev/mapper/lvmt_storage1-lvmt_storage1 /mnt/storage ext4 defaults,nofail 0 0

# Swapfile
/mnt/storage/swapfile swap swap defaults 0 0


# cat /proc/mounts

sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,nosuid,relatime,size=16400444k,nr_inodes=4100111,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=3284084k,mode=755 0 0
/dev/md0 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17913 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
/dev/mapper/lvmt_containers1-lvmt_containers1 /mnt/containers ext4 rw,relatime,stripe=256,data=ordered 0 0
/dev/mapper/lvmt_storage1-lvmt_storage1 /mnt/storage ext4 rw,relatime,stripe=256,data=ordered 0 0
/dev/sda1 /boot/efi vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 0
zfs_containers2 /zfs_containers2 zfs rw,xattr,noacl 0 0
lxcfs /var/lib/lxcfs fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
/dev/fuse /etc/pve fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3284080k,mode=700,uid=1000,gid=1000 0 0

gozora commented at 2019-06-29 20:16:¶

Since Borg is using same exclude mechanism as e.g. NETFS, can you try some of https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L2387, especially EXCLUDE_BACKUP can be usefull.

V.

gaia commented at 2019-06-30 19:30:¶

thanks.

what is the syntax for EXCLUDE_BACKUP ?

should I use only EXCLUDE_BACKUP instead of all the below variables?
EXCLUDE_MOUNTPOINTS
EXCLUDE_VG
EXCLUDE_WD
COPY_AS_IS_EXCLUDE

gaia commented at 2019-07-01 00:32:¶

tried

EXCLUDE_BACKUP=( ${EXCLUDE_BACKUP[@]} fs:/home/*/.cache/* fs:/var/cache/* fs:/var/tmp/* fs:/dev/* fs:/media/* fs:/proc/* fs:/sys/* fs:/tmp/* fs:/var/run/* fs:/var/lock/* fs:/mnt/storage/swapfile )

and

EXCLUDE_BACKUP=( ${EXCLUDE_BACKUP[@]} fs:/mnt/storage/swapfile )

still backed up swapfile

jsmeix commented at 2019-07-01 12:25:¶

@gaia
I do not use Borg backup but as far as I see in
usr/share/rear/backup/BORG/default/500_make_backup.sh
it uses the same $TMP_DIR/backup-include.txt mechanism via
usr/share/rear/backup/BORG/default/400_create_include_exclude_files.sh
that is a symlink to
usr/share/rear/backup/NETFS/default/400_create_include_exclude_files.sh
and therein is

# Implicitly also backup all local filesystems as defined in mountpoint_device
...
        # Add the mountpoints that will be recovered to the backup include list
        # unless a mountpoint is excluded:
        while read mountpoint device junk ; do
            if ! IsInArray "$mountpoint" "${EXCLUDE_MOUNTPOINTS[@]}" ; then
                echo "$mountpoint"
            fi
        done <"$VAR_DIR/recovery/mountpoint_device" >> $TMP_DIR/backup-include.txt

I think you should run "rear -D mkbackup" in debug mode
and inspect the log file what exactly hapens in your case
how $TMP_DIR/backup-include.txt is filled up.

Additionally you may use KEEP_BUILD_DIR so that you can see what
in $TMP_DIR/backup-include.txt actually is in your particular case.
That file is located in /tmp/rear.XXX/tmp/backup-include.txt.
See the KEEP_BUILD_DIR description in default.conf e.g. online at
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L128

I guess you may need BACKUP_ONLY_INCLUDE to switch off
that automatism, see its description in default.conf e.g. online at
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L976
but then you must explicitly specify all you want to have in your backup.

gaia commented at 2019-07-03 21:15:¶

Thank you, I used -d to keep the tmp dir.

I assumed the swapfile is being backed up because I see 13.85 GB O 54.40 MB C 65.95 kB D 0 N mnt/storage/swapfile during the backup process (and it goes up to 32GB, size of swap). But ll /tmp/rear*/rootfs/mnt/ shows only cdrom local, and none of the other folders in /mnt. if the swap is not being backed up, why does the backup process spend most of its time on compressing the swapfile? just curious, as I've given up. Let the swapfile be included.

What I can't give up on is /home and other dirs (although some not important) not being included in the backup:

lsblk:

PS: I can't use INCLUDE_ONLY at all. this is a recipe for disaster when you are trying to backup a root partition. the only possibility would be using borg directly to backup the root FS and having rear backup only the resulting borg repo.

here's the current local.conf:

BACKUP=BORG
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
###
USB_DEVICE_PARTED_LABEL=gpt
USB_DEVICE_FILESYSTEM=ext4
USING_UEFI_BOOTLOADER=1
###
SSH_UNPROTECTED_PRIVATE_KEYS="yes"
SSH_FILES="yes"
###
EXCLUDE_MOUNTPOINTS=( 'zfs_containers2' 'mnt/storage' 'mnt/containers' 'mnt/usb' 'mnt/bindmounts' 'mnt/storage/swapfile' )
EXCLUDE_MD=( /dev/md1 )
EXCLUDE_VG=( /dev/mapper/lvmt_containers1-lvmt_containers1 /dev/mapper/lvm_storage1-lvm_storage1 )
###
BORGBACKUP_REPO="/node2_borg"
BORGBACKUP_UMASK="0002"
BORGBACKUP_PRUNE_DAILY=7
BORGBACKUP_ENC_TYPE="keyfile"
BORGBACKUP_COMPRESSION="lz4"
BORGBACKUP_SHOW_PROGRESS="yes"
BORGBACKUP_SHOW_STATS="yes"
BORGBACKUP_REPO="/node2_borg_backup"
export BORG_PASSPHRASE="xxxxxxxxxxxxxx"
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"
###
PROGS_BORG=( )
COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" 'home/*/.cache/*' 'var/cache/*' 'var/tmp/*' 'dev/*' 'media/*' 'proc/*' 'sys/*' 'tmp/*' 'var/run/*' 'var/lock/*' '/mnt/storage/swapfile' )
COPY_AS_IS_BORG=(  )
COPY_AS_IS_BORG=( "${COPY_AS_IS[@]}" "/usr/src/linux-headers-4.15.18-16-pve" "/usr/share/file/magic" )

backup-include.txt (why are the first two included?)

/mnt/storage
/mnt/containers
/
/boot/efi

backup-exclude.txt:

/tmp/*
/dev/shm/*
/var/lib/rear/output/*
/tmp/rear.8zBQExtwAOKXB07
zfs_containers2/
mnt/storage/
mnt/containers/
mnt/usb/
mnt/bindmounts/
mnt/storage/swapfile/

jsmeix commented at 2019-07-04 12:27:¶

usr/share/rear/layout/save/default/340_generate_mountpoint_device.sh
generates
/tmp/rear.XXX/rootfs/...var/lib/rear/recovery/mountpoint_device
which is used by
usr/share/rear/backup/BORG/default/400_create_include_exclude_files.sh
that is a symlink to
usr/share/rear/backup/NETFS/default/400_create_include_exclude_files.sh
to generate
/tmp/rear.XXX/tmp/backup-include.txt
and
/tmp/rear.XXX/tmp/backup-exclude.txt
which are used for 'tar' by
usr/share/rear/backup/NETFS/default/500_make_backup.sh
and for Borg by
usr/share/rear/backup/BORG/default/500_make_backup.sh
to actually make the backup.

I do not use Borg but plain 'tar'
and for me BACKUP_ONLY_INCLUDE "just works".

I compared in etc/rear/local.conf
the default behaviour without BACKUP_ONLY_INCLUDE and BACKUP_PROG_INCLUDE
with BACKUP_ONLY_INCLUDE and an explicit BACKUP_PROG_INCLUDE listing

BACKUP_ONLY_INCLUDE="yes"
BACKUP_PROG_INCLUDE=( / /boot/efi )

and I got exactly the same files in my backup in both cases.

I have those real disk filesystems mounted

# mount | grep dev/sd
/dev/sda2 on / type ext4 ...
/dev/sda1 on /boot/efi type vfat ...

accordingly I need to specify / and /boot/efi in BACKUP_PROG_INCLUDE.

I got a /tmp/rear.XXX/tmp/backup-include.txt file that contains

/
/boot/efi

All what BACKUP_ONLY_INCLUDE requires for me with 'tar'
is explicit listing of those mounted real disk filesystems
that should be included in the backup in BACKUP_PROG_INCLUDE.

This is because 'tar' is called in
usr/share/rear/backup/NETFS/default/500_make_backup.sh
and 'borg create' is called in
usr/share/rear/backup/BORG/default/500_make_backup.sh
both with the '--one-file-system' option
so from plain loooking at the code it looks to me as if
BACKUP_ONLY_INCLUDE work same for Borg and 'tar'.

But I don't know if things may actually behave different with Borg
because I do not use Borg.

gaia commented at 2019-07-04 16:28:¶

thanks. I tried with

BACKUP_ONLY_INCLUDE="yes"
BACKUP_PROG_INCLUDE=( / /boot/efi )

with no exclusions. it still did not include /home (and others)

could i use borg separately, and use rear to backup only the borg backup to the USB? I'd imagine I'd first restore with rear a minimal system, then restore the actual system with borg on a 2nd pass.

adatum commented at 2019-08-10 19:37:¶

@gaia What about using only rear mkrescue and including the borg executable in the available programs? Use rear to recover the system layout and then handle backup restore in the usual ways with borg, hopefully all from the rescue environment.

gaia commented at 2019-08-11 14:55:¶

@adatum that's what I ended up doing. thanks.

jsmeix commented at 2019-08-12 13:17:¶

@gaia
and addedum FYI:

When you use

BACKUP_ONLY_INCLUDE="yes"
BACKUP_PROG_INCLUDE=( / /boot/efi )

the whole point of BACKUP_ONLY_INCLUDE="yes" is
that only / and /boot/efi are in the backup (and not including /home and others)
so to get also e.g. /home in the backup you would have to use

BACKUP_ONLY_INCLUDE="yes"
BACKUP_PROG_INCLUDE=( / /boot/efi /home )

when /home is mounted separately or behaves this way.

Strange - it seems in your case /home in not mounted separately.
Could it be that in your case /home is not just a normal directory in the
root filesystem but something special so that Borg backup treats it
as if it is not on the same '--one-file-system' as / is?
E.g. btrfs subvolumes behave this way.

[Export of Github issue for rear/rear.]

#2168 Issue closed: BORG: Unable to exclude swapfile from backup¶