#1036 Issue closed: SLES12 SP1 recovery fails with btrfs if snapper is not enabled¶
Labels: bug, fixed / solved / done
gozora opened issue at 2016-10-15 12:42:¶
- rear version (/usr/sbin/rear -V): Relax-and-Recover 1.18 / Git
- OS version (cat /etc/rear/os.conf or lsb_release -a): SUSE Linux Enterprise Server 12 SP1
- rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):
BACKUP=BORG
OUTPUT=ISO
BORG_HOST=beta.virtual.sk
BORG_USERNAME=root
BORG_REPO=/mnt/rear/borg/$HOSTNAME
BORG_PRUNE_HOURLY=5
BORG_PRUNE_WEEKLY=2
OUTPUT_URL=nfs://beta.virtual.sk/mnt/rear/iso
EXCLUDE_RECREATE=( fs:/srv/www/htdocs/glpi_data )
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} /mnt /test/exclude )
GRUB_RESCUE=n
ISO_MKISOFS_BIN=/usr/bin/ebiso
#BTRFS stuff
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" snapper chattr lsattr )
COPY_AS_IS=( "${COPY_AS_IS[@]}" /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
- Brief description of the issue
I've installed my SLES12 SP1 from scratch, my btrfs was not managed by
snapper (for whatever reason).
Btrfs config looks like this:
suse12:~ # btrfs subvol list -a /
ID 257 gen 3982 top level 5 path <FS_TREE>/@
ID 258 gen 3817 top level 257 path @/boot/grub2/i386-pc
ID 259 gen 3910 top level 257 path @/boot/grub2/x86_64-efi
ID 260 gen 3817 top level 257 path @/home
ID 261 gen 3817 top level 257 path @/opt
ID 262 gen 3817 top level 257 path @/srv
ID 263 gen 3978 top level 257 path @/tmp
ID 264 gen 3892 top level 257 path @/usr/local
ID 265 gen 3817 top level 257 path @/var/crash
ID 266 gen 3817 top level 257 path @/var/lib/libvirt/images
ID 267 gen 3817 top level 257 path @/var/lib/mailman
ID 268 gen 3817 top level 257 path @/var/lib/mariadb
ID 269 gen 3817 top level 257 path @/var/lib/mysql
ID 270 gen 3817 top level 257 path @/var/lib/named
ID 271 gen 3817 top level 257 path @/var/lib/pgsql
ID 272 gen 3978 top level 257 path @/var/log
ID 273 gen 3817 top level 257 path @/var/opt
ID 274 gen 3982 top level 257 path @/var/spool
ID 275 gen 3972 top level 257 path @/var/tmp
When running rear recover, session ended up with following trace in
log:
2016-10-15 11:58:12 Mounting filesystem /
+++ Print 'Mounting filesystem /'
+++ test -n 1
+++ echo -e 'Mounting filesystem /'
+++ mkdir -p /mnt/local/
+++ mount -t btrfs -o rw,relatime,space_cache /dev/sda3 /mnt/local/
++++ sed -e 's/<FS_TREE>\///'
++++ grep ' @$'
++++ tr -s '[:blank:]' ' '
++++ cut -d ' ' -f 2
++++ btrfs subvolume list -a /mnt/local/
+++ subvolumeID=
+++ btrfs subvolume set-default /mnt/local/
btrfs subvolume set-default: too few arguments
usage: btrfs subvolume set-default <subvolid> <path>
Missing argument is subvolid which seems to be obtained from
13_include_mount_subvolumes_code.sh by btrfs subvolume list ...
which as empty at time of execution. I'm really not sure if this is a
new clone of #944 resp #963, or something completely new.
- Work-around, if any
Configure snapper, create one snapper snapshot and recreate
Relax-and-Recover rescue/recovery system:
snapper -c root create-config /
snapper create
rear mkrescue
With snapper in place rear recover runs just fine.
gozora commented at 2016-10-15 12:51:¶
I've installed my SLES12 SP1 from scratch, my btrfs was not managed by snapper (for whatever reason).
Hmm, this "whatever reason" looks to be mentioned
here:
"Taking snapshots is automatically enabled if the root partition (/) is
big enough (approximately more than 8GB). Taking snapshots on partitions
other than / is not enabled by default."
So maybe snapper is not mandatory if your root FS (/) is not large enough?
jsmeix commented at 2016-10-17 08:26:¶
@gozora
how big is your harddisk?
Can you also show me your disklayout.conf file?
I will reproduce it.
Because my QEMU/KVM default virtual harddisk is 20GB
I had recently always tested btrfs with snapper enabled
but I think in the past it had also worked without snapper...?
gozora commented at 2016-10-17 08:33:¶
Hello @jsmeix,
I use 8GB disk (VirtualBox), OS sees disk ask follows:
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 7.2G 5.3G 1.7G 77% /
devtmpfs 110M 0 110M 0% /dev
tmpfs 119M 0 119M 0% /dev/shm
tmpfs 119M 5.7M 113M 5% /run
tmpfs 119M 0 119M 0% /sys/fs/cgroup
/dev/sda3 7.2G 5.3G 1.7G 77% /var/tmp
/dev/sda3 7.2G 5.3G 1.7G 77% /var/spool
/dev/sda3 7.2G 5.3G 1.7G 77% /var/opt
/dev/sda3 7.2G 5.3G 1.7G 77% /var/lib/mysql
/dev/sda3 7.2G 5.3G 1.7G 77% /var/lib/pgsql
/dev/sda3 7.2G 5.3G 1.7G 77% /var/log
/dev/sda3 7.2G 5.3G 1.7G 77% /var/lib/named
/dev/sda3 7.2G 5.3G 1.7G 77% /var/lib/mariadb
/dev/sda3 7.2G 5.3G 1.7G 77% /var/lib/mailman
/dev/sda3 7.2G 5.3G 1.7G 77% /var/crash
/dev/sda3 7.2G 5.3G 1.7G 77% /usr/local
/dev/sda3 7.2G 5.3G 1.7G 77% /var/lib/libvirt/images
/dev/sda3 7.2G 5.3G 1.7G 77% /tmp
/dev/sda3 7.2G 5.3G 1.7G 77% /srv
/dev/sda3 7.2G 5.3G 1.7G 77% /opt
/dev/sda3 7.2G 5.3G 1.7G 77% /boot/grub2/x86_64-efi
/dev/sda3 7.2G 5.3G 1.7G 77% /home
/dev/sda3 7.2G 5.3G 1.7G 77% /boot/grub2/i386-pc
/dev/sda1 156M 4.9M 151M 4% /boot/efi
Here is my disklayout.conf:
# Disk /dev/sda
# Format: disk <devname> <size(bytes)> <partition label type>
disk /dev/sda 8589934592 gpt
# Partitions on /dev/sda
# Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
part /dev/sda 163577856 1048576 primary boot /dev/sda1
part /dev/sda 772800512 164626432 primary none /dev/sda2
part /dev/sda 7651459072 937426944 primary none /dev/sda3
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/sda1 /boot/efi vfat uuid=2FA0-92D0 label= options=rw,relatime,fmask=0002,dmask=0002,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro
fs /dev/sda3 / btrfs uuid=054c83f3-ff96-4342-b7de-a2bd64dce69e label= options=rw,relatime,space_cache,subvolid=257,subvol=/@
# Btrfs default subvolume for /dev/sda3 at /
# Format: btrfsdefaultsubvol <device> <mountpoint> <btrfs_subvolume_ID> <btrfs_subvolume_path>
btrfsdefaultsubvol /dev/sda3 / 257 @
# Btrfs normal subvolumes for /dev/sda3 at /
# Format: btrfsnormalsubvol <device> <mountpoint> <btrfs_subvolume_ID> <btrfs_subvolume_path>
# All mounted btrfs subvolumes (including mounted btrfs default subvolumes and mounted btrfs snapshot subvolumes).
# Determined by the findmnt command that shows the mounted btrfs_subvolume_path.
# Format: btrfsmountedsubvol <device> <subvolume_mountpoint> <mount_options> <btrfs_subvolume_path>
btrfsmountedsubvol /dev/sda3 / rw,relatime,space_cache,subvolid=257,subvol=/@ @
btrfsmountedsubvol /dev/sda3 /var/tmp rw,relatime,space_cache,subvolid=275,subvol=/@/var/tmp @/var/tmp
btrfsmountedsubvol /dev/sda3 /var/spool rw,relatime,space_cache,subvolid=274,subvol=/@/var/spool @/var/spool
btrfsmountedsubvol /dev/sda3 /var/opt rw,relatime,space_cache,subvolid=273,subvol=/@/var/opt @/var/opt
btrfsmountedsubvol /dev/sda3 /var/log rw,relatime,space_cache,subvolid=272,subvol=/@/var/log @/var/log
btrfsmountedsubvol /dev/sda3 /var/lib/pgsql rw,relatime,space_cache,subvolid=271,subvol=/@/var/lib/pgsql @/var/lib/pgsql
btrfsmountedsubvol /dev/sda3 /var/lib/named rw,relatime,space_cache,subvolid=270,subvol=/@/var/lib/named @/var/lib/named
btrfsmountedsubvol /dev/sda3 /var/lib/mysql rw,relatime,space_cache,subvolid=269,subvol=/@/var/lib/mysql @/var/lib/mysql
btrfsmountedsubvol /dev/sda3 /var/lib/mariadb rw,relatime,space_cache,subvolid=268,subvol=/@/var/lib/mariadb @/var/lib/mariadb
btrfsmountedsubvol /dev/sda3 /var/lib/mailman rw,relatime,space_cache,subvolid=267,subvol=/@/var/lib/mailman @/var/lib/mailman
btrfsmountedsubvol /dev/sda3 /var/lib/libvirt/images rw,relatime,space_cache,subvolid=266,subvol=/@/var/lib/libvirt/images @/var/lib/libvirt/images
btrfsmountedsubvol /dev/sda3 /var/crash rw,relatime,space_cache,subvolid=265,subvol=/@/var/crash @/var/crash
btrfsmountedsubvol /dev/sda3 /usr/local rw,relatime,space_cache,subvolid=264,subvol=/@/usr/local @/usr/local
btrfsmountedsubvol /dev/sda3 /tmp rw,relatime,space_cache,subvolid=263,subvol=/@/tmp @/tmp
btrfsmountedsubvol /dev/sda3 /srv rw,relatime,space_cache,subvolid=262,subvol=/@/srv @/srv
btrfsmountedsubvol /dev/sda3 /opt rw,relatime,space_cache,subvolid=261,subvol=/@/opt @/opt
btrfsmountedsubvol /dev/sda3 /home rw,relatime,space_cache,subvolid=260,subvol=/@/home @/home
btrfsmountedsubvol /dev/sda3 /boot/grub2/x86_64-efi rw,relatime,space_cache,subvolid=259,subvol=/@/boot/grub2/x86_64-efi @/boot/grub2/x86_64-efi
btrfsmountedsubvol /dev/sda3 /boot/grub2/i386-pc rw,relatime,space_cache,subvolid=258,subvol=/@/boot/grub2/i386-pc @/boot/grub2/i386-pc
# Mounted btrfs subvolumes that have the 'no copy on write' attribute set.
# Format: btrfsnocopyonwrite <btrfs_subvolume_path>
btrfsnocopyonwrite @/var/lib/pgsql
btrfsnocopyonwrite @/var/lib/mysql
btrfsnocopyonwrite @/var/lib/mariadb
btrfsnocopyonwrite @/var/lib/libvirt/images
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/sda2 uuid=cfefd884-82c7-414c-a1b7-0beb0ced4170 label=
gdha commented at 2017-01-02 16:40:¶
@gozora @jsmeix Is this issue resolved for ReaR v2.0? Or, do we need to post-pone it for the next release?
jsmeix commented at 2017-01-03 07:41:¶
This issue is not resolved.
I will work on it when the first user is actually hit by it
(i.e. when it happens on a real used system) because
I assume it does not happen in practice - otherwise I should
have got already at least one SUSE customer bug report.
aussendorf commented at 2017-01-16 15:23:¶
Trying to run rear 2 with my patches for Bareos in our CI environment using openSuse Leap 42.2 produces the same behavior. From the log:
+++ mount -t btrfs -o rw,relatime,ssd,space_cache /dev/xvda2
/mnt/local/
mount: /dev/xvda2 is already mounted or /mnt/local busy
/dev/xvda2 is already mounted on /mnt/local
2017-01-16 12:21:01 An error occurred during layout recreation.
2017-01-16 12:21:01 User selected: ) Abort Relax-and-Recover
I send you the full log by PM
jsmeix commented at 2017-01-17 08:22:¶
@aussendorf
from your full log by PM (excerpts)
+++ parted -s /dev/xvda mkpart '"primary"' 1048576B 1035993087B ... +++ parted -s /dev/xvda mkpart '"primary"' 1035993088B 8589934591B ... +++ btrfs subvolume set-default /mnt/local/ btrfs subvolume set-default: too few arguments usage: btrfs subvolume set-default
Your disk is only about 8GiB with
one partition with size
(1035993087-1048576)/1024/1024/1024 = 0.96 GiB
and one partition with size
(8589934591-1035993088)/1024/1024/1024 = 7.0 Gib
It is known that current ReaR does not work on
small disks with SUSE's special btrfs structure,
see the above comments.
I guess a workaround hack could be in
usr/share/rear/layout/prepare/GNU/Linux/130_include_mount_subvolumes_code.sh
to change therein the code that is related to
"btrfs subvolume set-default".
jsmeix commented at 2017-07-31 12:10:¶
With
https://github.com/rear/rear/pull/1435
merged
this issue should be fixed.
Many thanks to @schabrolles
for his careful analysis of the actual root issue in
layout/save/GNU/Linux/230_filesystem_layout.sh
and for his fix and
many thanks to @gozora
for his verification and careful code review.
[Export of Github issue for rear/rear.]