#944 Issue closed: "rear recover" fails with default btrfs on SLES12-SP2-beta5¶
Labels: enhancement, cleanup, fixed / solved / done
jsmeix opened issue at 2016-07-27 08:12:¶
Current rear GitHub master
on a from scratch installed SLES12-SP2-beta5
with its default btrfs structure using
usr/share/rear/conf/examples/SLE12-SP1-btrfs-example.conf
as template for etc/rear/local.conf
"rear mkbackup" just works but
"rear recover" fails with:
... Create subvolume '/mnt/local//@/var/spool' Create subvolume '/mnt/local//@/var/tmp' Create subvolume '/mnt/local//@/.snapshots/1/snapshot/var/lib/machines' step 1 device:/dev/sda2 temporarily mounting device copying/modifying config-file creating filesystem config An error occurred during layout recreation.
and in the rear log file there is:
+++ echo '2016-07-27 07:44:37.924058935 Mounting filesystem /' 2016-07-27 07:44:37.924058935 Mounting filesystem / +++ Print 'Mounting filesystem /' +++ test -n 1 +++ echo -e 'Mounting filesystem /' +++ mkdir -p /mnt/local/ +++ mount -t btrfs -o rw,relatime,space_cache /dev/sda2 /mnt/local/ +++ btrfs subvolume create /mnt/local//@ +++ test -d /mnt/local//@/boot/grub2 ... +++ btrfs subvolume create /mnt/local//@/var/spool +++ test -d /mnt/local//@/var +++ btrfs subvolume create /mnt/local//@/var/tmp +++ test -d /mnt/local//@/.snapshots/1/snapshot/var/lib +++ mkdir -p /mnt/local//@/.snapshots/1/snapshot/var/lib +++ btrfs subvolume create /mnt/local//@/.snapshots/1/snapshot/var/lib/machines ++++ sed -e 's/\///' ++++ grep ' @$' ++++ tr -s '[:blank:]' ' ' ++++ cut -d ' ' -f 2 ++++ btrfs subvolume list -a /mnt/local/ +++ subvolumeID=257 +++ btrfs subvolume set-default 257 /mnt/local/ +++ umount /mnt/local/ +++ test -x /usr/lib/snapper/installation-helper +++ /usr/lib/snapper/installation-helper --step 1 --device /dev/sda2 --description 'first root filesystem' create subvolume failed, ioctl(BTRFS_IOC_SUBVOL_CREATE) failed, errno:17 (File exists) terminate called after throwing an instance of 'snapper::CreateConfigFailedException' what(): creating btrfs subvolume .snapshots failed since it already exists /var/lib/rear/layout/diskrestore.sh: line 390: 2205 Aborted /usr/lib/snapper/installation-helper --step 1 --device /dev/sda2 --description 'first root filesystem' ++ (( 1 == 0 )) ++ LogPrint 'An error occurred during layout recreation.' ++ Log 'An error occurred during layout recreation.'
It seems as always in SLE12 they changed something
in the default btrfs setup on SLE12 from SP1 to SP2
(before they had changed it from SLE12 GA to SP1)
so that as always I need to adapt and enhance rear
to make it work (hopefully in a generic way)
for all the various different kind of btrfs structures.
FYI:
With ext4 it "just works" also on SLES12-SP2-beta5
(using usr/share/rear/conf/examples/SLE11-ext3-example.conf
as template for etc/rear/local.conf).
jsmeix commented at 2016-07-28 07:50:¶
As a reference to what they had changed
in the btrfs default setup from SLE12 GA to SP1
see
https://github.com/rear/rear/issues/556
jsmeix commented at 2016-08-03 13:30:¶
Since SLES12-SP2 there is by default another '.snapshots'
normal btrfs subvolume that appears in disklayout.conf as
btrfsnormalsubvol /dev/sda2 / 281 @/.snapshots/1/snapshot/var/lib/machines
When one manually disables that in the rear recovery system
in var/lib/rear/layout/disklayout.conf as
#btrfsnormalsubvol /dev/sda2 / 281 @/.snapshots/1/snapshot/var/lib/machines
then "rear recover" succeeds on SLES12-SP2.
The code that needs to be enhanced is in
layout/prepare/GNU/Linux/13_include_mount_subvolumes_code.sh
# In case of SLES 12 SP1 special btrfs subvolumes setup
# skip setup of the normal btrfs subvolume '@/.snapshots' because
# that one will be created by "snapper/installation-helper --step 1"
# which fails if it already exists:
if test -n "$SLES12SP1_btrfs_subvolumes_setup" ; then
test "$subvolume_path" = "@/.snapshots" && continue
fi
so that it also excludes @/.snapshots/1/snapshot/var/lib/machines
But then that subvolume is missing after "rear recover".
In the original system it is there:
# btrfs subvolume list -a / ... ID 281 gen 139 top level 259 path @/.snapshots/1/snapshot/var/lib/machines ...
After "rear recover" it is missing.
The .snapshots/1/snapshot/var/lib/machines subvolume
is not created during initial system installation by YaST
via usual "btrfs subvolume create".
This is expected because usually all those 'snapshot' stuff
is somehow magically done by 'snapper'.
Now I need to find out what 'snapper' stuff is missing
during "rear recover" to also get that
.snapshots/1/snapshot/var/lib/machines
subvolume back.
jsmeix commented at 2016-08-03 14:17:¶
On the original system that .snapshots/1/snapshot/var/lib/machines
subvolume is not mounted so that it is perhaps not really needed.
At least for me it seems 'snapper' still works normally on the
recreated system.
jsmeix commented at 2016-08-03 14:21:¶
I remove 'bug' from that issue because it can never ever be a bug
when 'rear recover' fails because of special changes in newest
Linux distribution's fancy default setup.
jsmeix commented at 2016-08-03 14:23:¶
With
https://github.com/rear/rear/pull/950
merged
'rear recover' seems to work sufficiently well for me
so that I set it to be 'fixed'.
Nevertheless I keep that issue open until I better know
about that .snapshots/1/snapshot/var/lib/machines
subvolume.
jsmeix commented at 2016-08-08 13:40:¶
Now I know about that .snapshots/1/snapshot/var/lib/machines
subvolume:
That ".../var/lib/machines" subvolume is falsely created
in SLES12 SP2 beta5 - probably by systemd, cf.
https://bbs.archlinux.org/viewtopic.php?id=196541
and
https://cgit.freedesktop.org/systemd/systemd/commit/?id=113b3fc1a8061f4a24dd0db74e9a3cd0083b2251
Therefore there was nothing wrong in rear - even more:
It was "rear recover" that detected that wrong subvolume!
FYI:
Regarding how to create a btrfs subvolume in compliance
with the SLES12 default brtfs structure (i.e. how to create a
btrfs subvolume as the other SLES12 subvolumes are created):
Creating btrfs subvolumes in the SLES12 default brtfs structure
results falsely created btrfs subvolumes when one uses plain
"btrfs subvolume create /path/to/new/subvolume":
# btrfs subvolume create /var/lib/mysubvol Create subvolume '/var/lib/mysubvol' # btrfs subvolume list -a / | grep mysubvol ID 304 gen 1453 top level 259 path @/.snapshots/1/snapshot/var/lib/mysubvol
This way one gets /path/to/new/subvolume below that particular
snapshot subvolume that is currently mounted at '/'.
Therefore such a falsely created subvolume must be removed:
# btrfs subvolume delete /var/lib/mysubvol Delete subvolume (no-commit): '/var/lib/mysubvol'
To create a btrfs subvolume in compliance with the SLES12 default
brtfs structure one must create it below the '/@/' directory
but that directory is not accessible unless one mounts
the whole btrfs at its root subvolume somewhere:
# mkdir /tmp/btrfsroot # mount -t btrfs -o subvolid=0 /dev/sda2 /tmp/btrfsroot # ls /tmp/btrfsroot @ # btrfs subvolume create /tmp/btrfsroot/@/var/lib/mysubvol Create subvolume '/tmp/btrfsroot/@/var/lib/mysubvol' # btrfs subvolume list -a / | grep mysubvol ID 305 gen 1460 top level 257 path/@/var/lib/mysubvol # mkdir /var/lib/mysubvol # mount -t btrfs -o subvolid=305 /dev/sda2 /var/lib/mysubvol # findmnt -t btrfs -o TARGET,SOURCE TARGET SOURCE / /dev/sda2[/@/.snapshots/1/snapshot] ... |-/tmp /dev/sda2[/@/tmp] | `-/tmp/btrfsroot /dev/sda2 ... `-/var/lib/mysubvol /dev/sda2[/@/var/lib/mysubvol]
Now it is correctly created and accessible:
# echo hello1 >/tmp/btrfsroot/@/var/lib/mysubvol/hello # cat /var/lib/mysubvol/hello hello1 # echo hello2 >/var/lib/mysubvol/hello # cat /tmp/btrfsroot/@/var/lib/mysubvol/hello hello2 # umount /tmp/btrfsroot # rm -v /var/lib/mysubvol/hello removed '/var/lib/mysubvol/hello'
jsmeix commented at 2016-08-16 15:42:¶
With
https://github.com/rear/rear/pull/966
merged
this issue should now be solved in a more generic
and a more fail-safe way.
At least for me it now runs like clockwork.
[Export of Github issue for rear/rear.]