#1435 PR merged: SLES12 with no snapshots failed to recreate/mount btrfs FS during recovery.¶
Labels: bug, fixed / solved / done
schabrolles opened issue at 2017-07-28 18:55:¶
If you install SLES12 with btrfs support but without snapshots you should face some issue during recovery:
During layout creation, ReaR will fail to mount btrfs FS (because subvolumeID is empty)
+++ subvolumeID=
+++ btrfs subvolume set-default /mnt/local/
btrfs subvolume set-default: too few arguments
usage: btrfs subvolume set-default <subvolid> <path>
Set the default subvolume of a filesystem
2017-07-25 17:33:28.564682686 An error occurred during layout recreation.
Looking at disklayout.conf shows that NO btrfsnormalsubvol are
present (=> no subvol creation)
# Disk /dev/sda
# Format: disk <devname> <size(bytes)> <partition label type>
disk /dev/sda 53687091200 msdos
# Partitions on /dev/sda
# Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
part /dev/sda 7340032 1048576 primary boot,prep /dev/sda1
part /dev/sda 53678702592 8388608 primary lvm /dev/sda2
lvmdev /dev/system /dev/sda2 m5HZEO-DWBB-ShHr-YZgm-sloR-8V9Q-aRnFm1 104841216
lvmgrp /dev/system 4096 12797 52416512
lvmvol /dev/system root 10240 83886080
lvmvol /dev/system swap 512 4194304
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/mapper/system-root / btrfs uuid=681502da-fcba-4da8-8925-b538f4c09049 label= options=rw,relatime,space_cache,subvolid=257,subvol=/@
# Btrfs default subvolume for /dev/mapper/system-root at /
# Format: btrfsdefaultsubvol <device> <mountpoint> <btrfs_subvolume_ID> <btrfs_subvolume_path>
btrfsdefaultsubvol /dev/mapper/system-root / 257 @
# Btrfs normal subvolumes for /dev/mapper/system-root at /
# Format: btrfsnormalsubvol <device> <mountpoint> <btrfs_subvolume_ID> <btrfs_subvolume_path>
# All mounted btrfs subvolumes (including mounted btrfs default subvolumes and mounted btrfs snapshot subvolumes).
# Determined by the findmnt command that shows the mounted btrfs_subvolume_path.
# Format: btrfsmountedsubvol <device> <subvolume_mountpoint> <mount_options> <btrfs_subvolume_path>
btrfsmountedsubvol /dev/mapper/system-root / rw,relatime,space_cache,subvolid=257,subvol=/@ @
btrfsmountedsubvol /dev/mapper/system-root /boot/grub2/powerpc-ieee1275 rw,relatime,space_cache,subvolid=258,subvol=/@/boot/grub2/powerpc-ieee1275 @/boot/grub2/powerpc-ieee1275
btrfsmountedsubvol /dev/mapper/system-root /home rw,relatime,space_cache,subvolid=259,subvol=/@/home @/home
btrfsmountedsubvol /dev/mapper/system-root /var/lib/mailman rw,relatime,space_cache,subvolid=268,subvol=/@/var/lib/mailman @/var/lib/mailman
btrfsmountedsubvol /dev/mapper/system-root /var/crash rw,relatime,space_cache,subvolid=265,subvol=/@/var/crash @/var/crash
btrfsmountedsubvol /dev/mapper/system-root /opt rw,relatime,space_cache,subvolid=260,subvol=/@/opt @/opt
btrfsmountedsubvol /dev/mapper/system-root /var/cache rw,relatime,space_cache,subvolid=264,subvol=/@/var/cache @/var/cache
btrfsmountedsubvol /dev/mapper/system-root /var/spool rw,relatime,space_cache,subvolid=275,subvol=/@/var/spool @/var/spool
btrfsmountedsubvol /dev/mapper/system-root /var/lib/libvirt/images rw,relatime,space_cache,subvolid=266,subvol=/@/var/lib/libvirt/images @/var/lib/libvirt/images
btrfsmountedsubvol /dev/mapper/system-root /srv rw,relatime,space_cache,subvolid=261,subvol=/@/srv @/srv
btrfsmountedsubvol /dev/mapper/system-root /var/log rw,relatime,space_cache,subvolid=273,subvol=/@/var/log @/var/log
btrfsmountedsubvol /dev/mapper/system-root /var/lib/mysql rw,relatime,space_cache,subvolid=270,subvol=/@/var/lib/mysql @/var/lib/mysql
btrfsmountedsubvol /dev/mapper/system-root /tmp rw,relatime,space_cache,subvolid=262,subvol=/@/tmp @/tmp
btrfsmountedsubvol /dev/mapper/system-root /var/lib/named rw,relatime,space_cache,subvolid=271,subvol=/@/var/lib/named @/var/lib/named
btrfsmountedsubvol /dev/mapper/system-root /var/lib/mariadb rw,relatime,space_cache,subvolid=269,subvol=/@/var/lib/mariadb @/var/lib/mariadb
btrfsmountedsubvol /dev/mapper/system-root /var/lib/machines rw,relatime,space_cache,subvolid=267,subvol=/@/var/lib/machines @/var/lib/machines
btrfsmountedsubvol /dev/mapper/system-root /usr/local rw,relatime,space_cache,subvolid=263,subvol=/@/usr/local @/usr/local
btrfsmountedsubvol /dev/mapper/system-root /var/lib/pgsql rw,relatime,space_cache,subvolid=272,subvol=/@/var/lib/pgsql @/var/lib/pgsql
btrfsmountedsubvol /dev/mapper/system-root /var/opt rw,relatime,space_cache,subvolid=274,subvol=/@/var/opt @/var/opt
btrfsmountedsubvol /dev/mapper/system-root /var/tmp rw,relatime,space_cache,subvolid=276,subvol=/@/var/tmp @/var/tmp
# Mounted btrfs subvolumes that have the 'no copy on write' attribute set.
# Format: btrfsnocopyonwrite <btrfs_subvolume_path>
btrfsnocopyonwrite @/var/lib/libvirt/images
btrfsnocopyonwrite @/var/lib/mysql
btrfsnocopyonwrite @/var/lib/mariadb
btrfsnocopyonwrite @/var/lib/pgsql
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/mapper/system-swap uuid=890ff8c3-e4ca-4189-8117-f128e72e1a80 label=
This seems to come from
usr/share/rear/layout/save/GNU/Linux/230_filesystem_layout.sh
SLES12 without snapshot controlled by snapper will generate an empty
$snapshot_subvolumes_pattern variable.
snapshot_subvolumes_pattern=$( btrfs subvolume list -as $btrfs_mountpoint | tr -s '[:blank:]' ' ' | cut -d ' ' -f 2 | sed -e 's/^/^/' -e 's/$/ |/' | tr -d '\n' | sed -e 's/|$//' )
because of that, $subvolumes_exclude_pattern variable generated looks
like this:
subvolumes_exclude_pattern="|@.snapshots"
=> This will filter ALL the subvolumes listed by
btrfs subvolume list -a /
+++ btrfs subvolume list -as /
+++ tr -s '[:blank:]' ' '
+++ cut -d ' ' -f 2
+++ sed -e 's/^/^/' -e 's/$/ |/'
+++ tr -d '\n'
+++ sed -e 's/|$//'
++ snapshot_subvolumes_pattern=
++ snapper_base_subvolume=@/.snapshots
++ subvolumes_exclude_pattern='|@/.snapshots'
++ echo '# Btrfs normal subvolumes for /dev/mapper/system-root at /'
++ echo '# Format: btrfsnormalsubvol <device> <mountpoint> <btrfs_subvolume_ID> <btrfs_subvolume_path>'
++ btrfs subvolume list -a /
++ grep -q @/.snapshots
++ test -z '|@/.snapshots'
++ echo '257 @
258 @/boot/grub2/powerpc-ieee1275
259 @/home
260 @/opt
261 @/srv
262 @/tmp
263 @/usr/local
264 @/var/cache
265 @/var/crash
266 @/var/lib/libvirt/images
267 @/var/lib/machines
268 @/var/lib/mailman
269 @/var/lib/mariadb
270 @/var/lib/mysql
271 @/var/lib/named
272 @/var/lib/pgsql
273 @/var/log
274 @/var/opt
275 @/var/spool
276 @/var/tmp'
++ egrep -v '|@/.snapshots'
++ sed -e 's/^/btrfsnormalsubvol \/dev\/mapper\/system-root \/ /'
++ read_mounted_btrfs_subvolumes_command='mount -t btrfs | cut -d '\'' '\'' -f 1,3,6'
++ test -x /usr/bin/findmnt
++ /usr/bin/findmnt -mnrv -o FSROOT -t btrfs
We just need to test if the $snapshot_subvolumes_pattern is empty
before setting the $subvolumes_exclude_pattern
Results after patch: (disklayout.conf) => (btrfsnormalsubvol are
present before btrfsmountedsubvol)
# Disk /dev/sda
# Format: disk <devname> <size(bytes)> <partition label type>
disk /dev/sda 53687091200 msdos
# Partitions on /dev/sda
# Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
part /dev/sda 7340032 1048576 primary boot,prep /dev/sda1
part /dev/sda 53678702592 8388608 primary lvm /dev/sda2
lvmdev /dev/system /dev/sda2 m5HZEO-DWBB-ShHr-YZgm-sloR-8V9Q-aRnFm1 104841216
lvmgrp /dev/system 4096 12797 52416512
lvmvol /dev/system root 10240 83886080
lvmvol /dev/system swap 512 4194304
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/mapper/system-root / btrfs uuid=681502da-fcba-4da8-8925-b538f4c09049 label= options=rw,relatime,space_cache,subvolid=257,subvol=/@
# Btrfs default subvolume for /dev/mapper/system-root at /
# Format: btrfsdefaultsubvol <device> <mountpoint> <btrfs_subvolume_ID> <btrfs_subvolume_path>
btrfsdefaultsubvol /dev/mapper/system-root / 257 @
# Btrfs normal subvolumes for /dev/mapper/system-root at /
# Format: btrfsnormalsubvol <device> <mountpoint> <btrfs_subvolume_ID> <btrfs_subvolume_path>
btrfsnormalsubvol /dev/mapper/system-root / 257 @
btrfsnormalsubvol /dev/mapper/system-root / 258 @/boot/grub2/powerpc-ieee1275
btrfsnormalsubvol /dev/mapper/system-root / 259 @/home
btrfsnormalsubvol /dev/mapper/system-root / 260 @/opt
btrfsnormalsubvol /dev/mapper/system-root / 261 @/srv
btrfsnormalsubvol /dev/mapper/system-root / 262 @/tmp
btrfsnormalsubvol /dev/mapper/system-root / 263 @/usr/local
btrfsnormalsubvol /dev/mapper/system-root / 264 @/var/cache
btrfsnormalsubvol /dev/mapper/system-root / 265 @/var/crash
btrfsnormalsubvol /dev/mapper/system-root / 266 @/var/lib/libvirt/images
btrfsnormalsubvol /dev/mapper/system-root / 267 @/var/lib/machines
btrfsnormalsubvol /dev/mapper/system-root / 268 @/var/lib/mailman
btrfsnormalsubvol /dev/mapper/system-root / 269 @/var/lib/mariadb
btrfsnormalsubvol /dev/mapper/system-root / 270 @/var/lib/mysql
btrfsnormalsubvol /dev/mapper/system-root / 271 @/var/lib/named
btrfsnormalsubvol /dev/mapper/system-root / 272 @/var/lib/pgsql
btrfsnormalsubvol /dev/mapper/system-root / 273 @/var/log
btrfsnormalsubvol /dev/mapper/system-root / 274 @/var/opt
btrfsnormalsubvol /dev/mapper/system-root / 275 @/var/spool
btrfsnormalsubvol /dev/mapper/system-root / 276 @/var/tmp
# All mounted btrfs subvolumes (including mounted btrfs default subvolumes and mounted btrfs snapshot subvolumes).
# Determined by the findmnt command that shows the mounted btrfs_subvolume_path.
# Format: btrfsmountedsubvol <device> <subvolume_mountpoint> <mount_options> <btrfs_subvolume_path>
btrfsmountedsubvol /dev/mapper/system-root / rw,relatime,space_cache,subvolid=257,subvol=/@ @
btrfsmountedsubvol /dev/mapper/system-root /boot/grub2/powerpc-ieee1275 rw,relatime,space_cache,subvolid=258,subvol=/@/boot/grub2/powerpc-ieee1275 @/boot/grub2/powerpc-ieee1275
btrfsmountedsubvol /dev/mapper/system-root /home rw,relatime,space_cache,subvolid=259,subvol=/@/home @/home
btrfsmountedsubvol /dev/mapper/system-root /var/lib/mailman rw,relatime,space_cache,subvolid=268,subvol=/@/var/lib/mailman @/var/lib/mailman
btrfsmountedsubvol /dev/mapper/system-root /var/crash rw,relatime,space_cache,subvolid=265,subvol=/@/var/crash @/var/crash
btrfsmountedsubvol /dev/mapper/system-root /opt rw,relatime,space_cache,subvolid=260,subvol=/@/opt @/opt
btrfsmountedsubvol /dev/mapper/system-root /var/cache rw,relatime,space_cache,subvolid=264,subvol=/@/var/cache @/var/cache
btrfsmountedsubvol /dev/mapper/system-root /var/spool rw,relatime,space_cache,subvolid=275,subvol=/@/var/spool @/var/spool
btrfsmountedsubvol /dev/mapper/system-root /var/lib/libvirt/images rw,relatime,space_cache,subvolid=266,subvol=/@/var/lib/libvirt/images @/var/lib/libvirt/images
btrfsmountedsubvol /dev/mapper/system-root /srv rw,relatime,space_cache,subvolid=261,subvol=/@/srv @/srv
btrfsmountedsubvol /dev/mapper/system-root /var/log rw,relatime,space_cache,subvolid=273,subvol=/@/var/log @/var/log
btrfsmountedsubvol /dev/mapper/system-root /var/lib/mysql rw,relatime,space_cache,subvolid=270,subvol=/@/var/lib/mysql @/var/lib/mysql
btrfsmountedsubvol /dev/mapper/system-root /tmp rw,relatime,space_cache,subvolid=262,subvol=/@/tmp @/tmp
btrfsmountedsubvol /dev/mapper/system-root /var/lib/named rw,relatime,space_cache,subvolid=271,subvol=/@/var/lib/named @/var/lib/named
btrfsmountedsubvol /dev/mapper/system-root /var/lib/mariadb rw,relatime,space_cache,subvolid=269,subvol=/@/var/lib/mariadb @/var/lib/mariadb
btrfsmountedsubvol /dev/mapper/system-root /var/lib/machines rw,relatime,space_cache,subvolid=267,subvol=/@/var/lib/machines @/var/lib/machines
btrfsmountedsubvol /dev/mapper/system-root /usr/local rw,relatime,space_cache,subvolid=263,subvol=/@/usr/local @/usr/local
btrfsmountedsubvol /dev/mapper/system-root /var/lib/pgsql rw,relatime,space_cache,subvolid=272,subvol=/@/var/lib/pgsql @/var/lib/pgsql
btrfsmountedsubvol /dev/mapper/system-root /var/opt rw,relatime,space_cache,subvolid=274,subvol=/@/var/opt @/var/opt
btrfsmountedsubvol /dev/mapper/system-root /var/tmp rw,relatime,space_cache,subvolid=276,subvol=/@/var/tmp @/var/tmp
# Mounted btrfs subvolumes that have the 'no copy on write' attribute set.
# Format: btrfsnocopyonwrite <btrfs_subvolume_path>
btrfsnocopyonwrite @/var/lib/libvirt/images
btrfsnocopyonwrite @/var/lib/mysql
btrfsnocopyonwrite @/var/lib/mariadb
btrfsnocopyonwrite @/var/lib/pgsql
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/mapper/system-swap uuid=890ff8c3-e4ca-4189-8117-f128e72e1a80 label=
gozora commented at 2017-07-28 19:05:¶
Sweet, if this works (and I really believe it does) it should solve #1036 as well...
V.
schabrolles commented at 2017-07-28 19:43:¶
@gozora, Yes, it's really looks like the same bug....
I've only tested it on SLE12 SP2, it could be great if you can confirm it solve your issue #1036 with SLE12 SP1.
schabrolles commented at 2017-07-29 07:45:¶
@jsmeix, your feedback is really welcomed as it is related to SLES with btrfs.
gozora commented at 2017-07-30 10:31:¶
Hello @schabrolles,
I just successfully run test restore of SLES12 SP1 with disabled snapper and no snapshots with your patch.
V.
jsmeix commented at 2017-07-31 08:53:¶
@schabrolles @gozora
I commited two changes:
First:
https://github.com/rear/rear/pull/1435/commits/ab5a3c86fbc9fa5c1cc6dda87337206fe9fcb3f7
is only cosmetical (fixed wrong issue number in comment).
Second:
https://github.com/rear/rear/pull/1435/commits/b90a3819d21f5eef3d08b77f646354f30959f282
mainly for you to have a closer look because
now the code looks duplicated because I already do test
in the code below for
if test -z "$snapshot_subvolumes_pattern" ; then
so that the test above from @schabrolles should
not be needed but somehow things go wrong
without the test above from @schabrolles
and currently I do not understand what actually
goes wrong here.
@schabrolles @gozora
could you have a closer look and help me to find out
what actually goes wrong here - somehow my test
if test -z "$snapshot_subvolumes_pattern" ; then
seems to not work as intended?
gozora commented at 2017-07-31 09:08:¶
Hi @jsmeix
Just from plain look on your patch
if test -z "$snapshot_subvolumes_pattern " ; then
Is the space in the end of "$snapshot_subvolumes_pattern " used on purpose?
V.
jsmeix commented at 2017-07-31 09:09:¶
Always - really always - when I have to work with
that oversophisticated SUSE btrfs default structure
my poor mathematician's mind gets "knotted" :-(
Meanwhile even I can see how it had failed before:
It is the subsequent test
# Output btrfs normal subvolumes: if test -z "$subvolumes_exclude_pattern" ; then
that does not work as intended in case of
subvolumes_exclude_pattern='|@/.snapshots'
so that the test above from @schabrolles
is needed to make that subsequent test
work as intended.
Accordingly I think everything is right with this pull request
but I will test it on SLES12-SP2 with the btrfs default structure
to verify that it still works there (which is the main use-case).
jsmeix commented at 2017-07-31 09:12:¶
Aaaaaaaaaaaarrrrrrrrrrrrrrggggggggggggghhhhhhhhhhhhh!
One space too much issues also hit me!
@gozora
many thanks for your careful looking
even at the emptiness in my code!
schabrolles commented at 2017-07-31 09:23:¶
@jsmeix HeHeHe .... Glad to see I'm not the only one ;)
jsmeix commented at 2017-07-31 09:40:¶
@schabrolles
many thanks for your careful analysis of the root issue
of
https://github.com/rear/rear/issues/1036
and for your fix here!
@schabrolles @gozora
from my point of view everything looks right now
(I even understand my own code a bit better now ;-)
and if you agree I would be happy to merge it soon.
[Export of Github issue for rear/rear.]