#598 Issue closed
: checklayout.conf/disklayout.conf sets wrong blocksize=swap, grep -oi in subshell does not return valid values¶
Labels: bug
, fixed / solved / done
abbbi opened issue at 2015-06-10 10:45:¶
hi,
the following situation: the created disklayout.conf misses newlines so
recovery fails, version
is 1.17.0 on sles11 sp1
the created file looks like this:
#disk /dev/sda 144955146240 gpt #part /dev/sda 205615104 17408 primary none /dev/sda1 #part /dev/sda 144749496832 205632512 primary lvm /dev/sda2 disk /dev/sdb 290984034304 lvmdev /dev/oracle /dev/sdb Ds6R3n-AUZ6-RzTn-7z7s-P8YK-Jexe-vtBlTc 568328192 #lvmdev /dev/system /dev/sda2 DW97hv-Aylg-eOKa-vup2-xUkX-7tOE-hy1Q7b 282713861 lvmgrp /dev/oracle 4096 69375 284160000 #lvmgrp /dev/system 4096 34510 141352960 lvmvol /dev/oracle disppl 25600 209715200 lvmvol /dev/oracle orabackup 12800 104857600 lvmvol /dev/oracle cos 5120 41943040 #lvmvol /dev/system home 1024 8388608 #lvmvol /dev/system opt 2560 20971520 #lvmvol /dev/system root 1024 8388608 #lvmvol /dev/system swap 5120 41943040 #lvmvol /dev/system tmp 512 4194304 #lvmvol /dev/system var 512 4194304 # Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported). # Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>] fs /dev/mapper/oracle-cos /oracle/COS ext3 uuid=fe070599-2e4c-4242-a357-755ba182e787 label= blocksize=swap /dev/mapper/system-swap uuid=845d7b2f-a09d-4566-8ce6-814c913db056 label=
note the blocksize=swap option which does not make any sense, this is
the point where
the recovery fails.
Im currently investigating this and i really have no idea what is going
wrong. First of all i can
see the following error from the checklayout logfile, which i think is
the cause of the problem:
2015-06-10 12:34:48.385146993 Saving filesystem layout (using the traditional mount command). /var/opt/sesam/var/lib/rear/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh: line 104: reserved_blocks * 100 / nr_blocks : division by 0 (error token is " ")
i have set -x in the appropriate section in 12_filesystem_layout.sh and it looks like this:
2015-06-10 12:34:48.385146993 Saving filesystem layout (using the traditional mount command). ++ tunefs=tune2fs ++ '[' ext3 = ext4 ']' ++ echo -n 'DEVICE: /dev/mapper/oracle-cos' +++ tune2fs -l /dev/mapper/oracle-cos +++ tr -d '[:blank:]' +++ grep -i UUID: +++ cut -d : -f 2 ++ uuid=fe070599-2e4c-4242-a357-755ba182e787 ++ echo -n ' uuid=fe070599-2e4c-4242-a357-755ba182e787' +++ e2label /dev/mapper/oracle-cos ++ label= ++ echo -n ' label=' +++ tune2fs -l /dev/mapper/oracle-cos +++ tr -d '[:blank:]' +++ grep -oi 'Blocksize:[0-9]*' +++ cut -d : -f 2 ++ blocksize= ++ echo -n ' blocksize=' +++ tune2fs -l /dev/mapper/oracle-cos +++ tr -d '[:blank:]' +++ grep -iv reserved +++ grep -oi 'Blockcount:[0-9]*' +++ cut -d : -f 2 ++ nr_blocks= +++ tune2fs -l /dev/mapper/oracle-cos +++ tr -d '[:blank:]' +++ grep -oi 'Reservedblockcount:[0-9]*' +++ cut -d : -f 2 ++ reserved_blocks=
as you can see it fails to read any info besides the UUID of the device!
It however works nicely
on the command line:
# echo $( tune2fs -l /dev/oracle/cos | tr -d '[:blank:]' | grep -iv reserved | grep -oi 'Blockcount:[0-9]*' | cut -d ':' -f 2 ) 5242880 # echo $( tune2fs -l /dev/oracle/cos | tr -d '[:blank:]' | grep -oi 'Blocksize:[0-9]*' | cut -d ':' -f 2 ) 4096
any idea how it can be that it fails to read the appropriate values if
the commands are executed
from REAR? If i add a static tune2fs command within this section i dont
get back any value from the command. /bin/sh points to bash, rear uses
bash aswell. There is only one tune2fs within
the path. fail to see where the problem could be.
abbbi commented at 2015-06-10 11:35:¶
hi,
ok this i very weird, after changing:
blocksize=$( $tunefs -l $device | tr -d '[:blank:]' | grep -oi 'Blocksize:[0-9]*' | cut -d ':' -f 2 )�������������������������������������������������������������������������������������������������������
to
blocksize=$( $tunefs -l $device | tr -d '[:blank:]' | grep -i 'Blocksize:[0-9]*' | cut -d ':' -f 2 )�������������������������������������������������������������������������������������������������������
it can succesfully get the blocksize and the other values:
+++ tune2fs -l /dev/mapper/oracle-cos
+++ tr -d '[:blank:]'
+++ grep -i 'Blocksize:[0-9]'
+++ cut -d : -f 2
++ blocksize=4096
++ echo -n ' blocksize=4096'
+++ tune2fs -l /dev/mapper/oracle-cos
+++ tr -d '[:blank:]'
+++ grep -iv reserved
+++ grep -i 'Blockcount:[0-9]'
+++ cut -d : -f 2
++ nr_blocks=5242880
+++ tune2fs -l /dev/mapper/oracle-cos
+++ tr -d '[:blank:]'
+++ grep -i 'Reservedblockcount:[0-9]'
+++ cut -d : -f 2
++ reserved_blocks=262144
++ reserved_percentage=5
++ echo -n ' reserved_blocks=5%'
+++ tune2fs -l /dev/mapper/oracle-cos
+++ tr -d '[:blank:]'
+++ grep -i 'Maximummountcount:[0-9]'
+++ cut -d : -f 2
++ max_mounts=-1
++ echo -n ' max_mounts=-1'
+++ tune2fs -l /dev/mapper/oracle-cos
+++ tr -d '[:blank:]'
+++ grep -oi 'Checkinterval:[0-9]*'
+++ cut -d : -f 2
++ check_interval=
++ let check_interval=/86400
/var/opt/sesam/var/lib/rear/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh:
line 111: let: check_interval=/86400: syntax error: operand expected
(error token is "/86400")
����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������
i stumbled about the followig comment regarding this:
# FIXME: I (jsmeix@suse.de) have no idea what the reason for the following is:
# On Fedora | grep -oi 'Maximummountcount:[0-9]*' | does not work but | grep -i 'Maximummountcount:[0-9]*' | works.
# If someone knows the reason replace this comment with a description of the actual root cause.
seems to be of the same nature ..
abbbi commented at 2015-06-10 11:40:¶
The other problem is with checkinterval wich contains not only a integer:
nosudb01:/var/opt/sesam/var/lib/rear/usr/sbin # tune2fs -l
/dev/oracle/cos | tr -d '[:blank:]' | grep -i
'Checkinterval:[0-9]*' | cut -d ':' -f 2
0(
����������������������������������������������������������������������������������������������������������������������������������������������������������������
this makes it bail out a few lines later:
++ let 'check_interval=0(
/var/opt/sesam/var/lib/rear/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh:
line 111: let: check_interval=0(
++ echo -n ' check_interval=0d'
�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������
jsmeix commented at 2015-06-10 12:33:¶
Again an issue with that tunefs output parsing code.
I think the whole tune2fs output parsing code in 23_filesystem_layout.sh is fragile and should be reworked to be more fail-safe. But currently I have no good idea how to actually improve it.
@abbbi
please run the commands
tune2fs -l /dev/mapper/oracle-cos tune2fs -l /dev/mapper/oracle-cos | tr '<>' '][' tune2fs -l /dev/mapper/oracle-cos | tr '<>' '][' | od -a
on your original system and paste their exact output here so that I could get a better understanding what exactly your particular tune2fs output is.
jsmeix commented at 2015-06-10 13:02:¶
@schlomo @gdha
it seems 23_filesystem_layout.sh proceeds regardless of errors in some of its commands and that results a broken disklayout.conf file.
I have a bad feeling with that error-ignoring behaviour.
I think all scripts that are run during "rear mkbackup" should by default exit in case of any error (i.e. "set -e" and "set -o pipefail") to avoid that what "rear mkbackup" produces could be silently wrong.
In contrast for scripts that are run during "rear recover" it might be a better default to ignore errors - but personally I even would prefer to also exit by default in case of errors for "rear recover".
A rear command line option should be added to specify the rear error-exit behaviour so that the admin could enfore on his own risk that rear proceeds regardless of errors.
abbbi commented at 2015-06-10 13:43:¶
hi,
it looks like the system in question has two grep versions, one
/bin/grep
/usr/bin/grep
probably this is cause of the issue, will wait for customer to
respond.
I have attached the output of your requested commands here:
http://download.sep.de/.sep/out.txt
jsmeix commented at 2015-06-10 13:47:¶
No - I think it is the other way round:
23_filesystem_layout.sh exits when one of the tune2fs suff fails e.g. at
reserved_percentage=$(( reserved_blocks * 100 / nr_blocks ))
with
/var/opt/sesam/var/lib/rear/usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh: line 104: reserved_blocks * 100 / nr_blocks : division by 0 (error token is " ")
The last thing that was written to disklayout.conf before is from
echo -n " blocksize=$blocksize"
When 23_filesystem_layout.sh exits as described here it causes that the final
# Finish the current filesystem layout line with a newline character: echo
is not run so that the newline is missing i.e. disklayout.conf looks now
... fs /dev/mapper/oracle-cos /oracle/COS ext3 uuid=fe070599-2e4c-4242-a357-755ba182e787 label= blocksize=
without a newline after "blocksize="
Afterwards 24_swaps_layout.sh is run and that appends to disklayout.conf
swap /dev/mapper/system-swap uuid=845d7b2f-a09d-4566-8ce6-814c913db056 label=
which results in the end
fs /dev/mapper/oracle-cos /oracle/COS ext3 uuid=fe070599-2e4c-4242-a357-755ba182e787 label= blocksize=swap /dev/mapper/system-swap uuid=845d7b2f-a09d-4566-8ce6-814c913db056 label=
Therefore - from my current point of view - the question is:
When 23_filesystem_layout.sh exits as described above, why does then rear proceed with 24_swaps_laxout.sh ?
jsmeix commented at 2015-06-11 09:48:¶
The content from http://download.sep.de/.sep/out.txt plus added commands:
# tune2fs -l /dev/mapper/oracle-cos tune2fs 1.41.9 (22-Aug-2009) Filesystem volume name: <none> Last mounted on: <not available> Filesystem UUID: fe070599-2e4c-4242-a357-755ba182e787 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file Filesystem flags: signed_directory_hash Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 1310720 Block count: 5242880 Reserved block count: 262144 Free blocks: 2991817 Free inodes: 1310693 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1022 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Filesystem created: Thu Jul 7 15:53:17 2011 Last mount time: Thu Apr 23 14:26:58 2015 Last write time: Thu Apr 23 14:26:58 2015 Mount count: 18 Maximum mount count: -1 Last checked: Thu Jul 7 15:53:17 2011 Check interval: 0 (<none>) Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: 6eb517c2-7af4-4131-82eb-fbe21d9a930c Journal backup: inode blocks # tune2fs -l /dev/mapper/oracle-cos | tr '<>' '][' tune2fs 1.41.9 (22-Aug-2009) Filesystem volume name: ]none[ Last mounted on: ]not available[ Filesystem UUID: fe070599-2e4c-4242-a357-755ba182e787 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file Filesystem flags: signed_directory_hash Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 1310720 Block count: 5242880 Reserved block count: 262144 Free blocks: 2991817 Free inodes: 1310693 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1022 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Filesystem created: Thu Jul 7 15:53:17 2011 Last mount time: Thu Apr 23 14:26:58 2015 Last write time: Thu Apr 23 14:26:58 2015 Mount count: 18 Maximum mount count: -1 Last checked: Thu Jul 7 15:53:17 2011 Check interval: 0 (]none[) Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: 6eb517c2-7af4-4131-82eb-fbe21d9a930c Journal backup: inode blocks # tune2fs -l /dev/mapper/oracle-cos | tr '<>' '][' | od -a 0000000 t u n e 2 f s sp 1 . 4 1 . 9 sp ( 0000020 2 2 - A u g - 2 0 0 9 ) nl F i l 0000040 e s y s t e m sp v o l u m e sp n 0000060 a m e : sp sp sp ] n o n e [ nl L a 0000100 s t sp m o u n t e d sp o n : sp sp 0000120 sp sp sp sp sp sp sp sp ] n o t sp a v a 0000140 i l a b l e [ nl F i l e s y s t 0000160 e m sp U U I D : sp sp sp sp sp sp sp sp 0000200 sp sp f e 0 7 0 5 9 9 - 2 e 4 c - 0000220 4 2 4 2 - a 3 5 7 - 7 5 5 b a 1 0000240 8 2 e 7 8 7 nl F i l e s y s t e 0000260 m sp m a g i c sp n u m b e r : sp 0000300 sp 0 x E F 5 3 nl F i l e s y s t 0000320 e m sp r e v i s i o n sp # : sp sp 0000340 sp sp 1 sp ( d y n a m i c ) nl F i 0000360 l e s y s t e m sp f e a t u r e 0000400 s : sp sp sp sp sp sp h a s _ j o u r 0000420 n a l sp e x t _ a t t r sp r e s 0000440 i z e _ i n o d e sp d i r _ i n 0000460 d e x sp f i l e t y p e sp n e e 0000500 d s _ r e c o v e r y sp s p a r 0000520 s e _ s u p e r sp l a r g e _ f 0000540 i l e nl F i l e s y s t e m sp f 0000560 l a g s : sp sp sp sp sp sp sp sp sp s i 0000600 g n e d _ d i r e c t o r y _ h 0000620 a s h sp nl D e f a u l t sp m o u 0000640 n t sp o p t i o n s : sp sp sp sp ( 0000660 n o n e ) nl F i l e s y s t e m 0000700 sp s t a t e : sp sp sp sp sp sp sp sp sp 0000720 c l e a n nl E r r o r s sp b e h 0000740 a v i o r : sp sp sp sp sp sp sp sp sp sp 0000760 C o n t i n u e nl F i l e s y s 0001000 t e m sp O S sp t y p e : sp sp sp sp 0001020 sp sp sp L i n u x nl I n o d e sp c 0001040 o u n t : sp sp sp sp sp sp sp sp sp sp sp 0001060 sp sp sp 1 3 1 0 7 2 0 nl B l o c k 0001100 sp c o u n t : sp sp sp sp sp sp sp sp sp 0001120 sp sp sp sp sp 5 2 4 2 8 8 0 nl R e s 0001140 e r v e d sp b l o c k sp c o u n 0001160 t : sp sp sp sp sp 2 6 2 1 4 4 nl F r 0001200 e e sp b l o c k s : sp sp sp sp sp sp 0001220 sp sp sp sp sp sp sp sp 2 9 9 1 8 1 7 nl 0001240 F r e e sp i n o d e s : sp sp sp sp 0001260 sp sp sp sp sp sp sp sp sp sp 1 3 1 0 6 9 0001300 3 nl F i r s t sp b l o c k : sp sp 0001320 sp sp sp sp sp sp sp sp sp sp sp sp 0 nl B l 0001340 o c k sp s i z e : sp sp sp sp sp sp sp 0001360 sp sp sp sp sp sp sp sp 4 0 9 6 nl F r a 0001400 g m e n t sp s i z e : sp sp sp sp sp 0001420 sp sp sp sp sp sp sp 4 0 9 6 nl R e s e 0001440 r v e d sp G D T sp b l o c k s : 0001460 sp sp sp sp sp sp 1 0 2 2 nl B l o c k 0001500 s sp p e r sp g r o u p : sp sp sp sp 0001520 sp sp sp sp sp 3 2 7 6 8 nl F r a g m 0001540 e n t s sp p e r sp g r o u p : sp 0001560 sp sp sp sp sp 3 2 7 6 8 nl I n o d e 0001600 s sp p e r sp g r o u p : sp sp sp sp 0001620 sp sp sp sp sp 8 1 9 2 nl I n o d e sp 0001640 b l o c k s sp p e r sp g r o u p 0001660 : sp sp sp 5 1 2 nl F i l e s y s t 0001700 e m sp c r e a t e d : sp sp sp sp sp 0001720 sp sp T h u sp J u l sp sp 7 sp 1 5 : 0001740 5 3 : 1 7 sp 2 0 1 1 nl L a s t sp 0001760 m o u n t sp t i m e : sp sp sp sp sp 0002000 sp sp sp sp sp T h u sp A p r sp 2 3 sp 0002020 1 4 : 2 6 : 5 8 sp 2 0 1 5 nl L a 0002040 s t sp w r i t e sp t i m e : sp sp 0002060 sp sp sp sp sp sp sp sp T h u sp A p r sp 0002100 2 3 sp 1 4 : 2 6 : 5 8 sp 2 0 1 5 0002120 nl M o u n t sp c o u n t : sp sp sp 0002140 sp sp sp sp sp sp sp sp sp sp sp 1 8 nl M a 0002160 x i m u m sp m o u n t sp c o u n 0002200 t : sp sp sp sp sp sp - 1 nl L a s t sp 0002220 c h e c k e d : sp sp sp sp sp sp sp sp 0002240 sp sp sp sp sp T h u sp J u l sp sp 7 sp 0002260 1 5 : 5 3 : 1 7 sp 2 0 1 1 nl C h 0002300 e c k sp i n t e r v a l : sp sp sp 0002320 sp sp sp sp sp sp sp sp 0 sp ( ] n o n e 0002340 [ ) nl R e s e r v e d sp b l o c 0002360 k s sp u i d : sp sp sp sp sp sp 0 sp ( 0002400 u s e r sp r o o t ) nl R e s e r 0002420 v e d sp b l o c k s sp g i d : sp 0002440 sp sp sp sp sp 0 sp ( g r o u p sp r o 0002460 o t ) nl F i r s t sp i n o d e : 0002500 sp sp sp sp sp sp sp sp sp sp sp sp sp sp 1 1 0002520 nl I n o d e sp s i z e : ht sp sp sp 0002540 sp sp sp sp sp sp sp 2 5 6 nl R e q u i 0002560 r e d sp e x t r a sp i s i z e : 0002600 sp sp sp sp sp 2 8 nl D e s i r e d sp 0002620 e x t r a sp i s i z e : sp sp sp sp 0002640 sp sp 2 8 nl J o u r n a l sp i n o 0002660 d e : sp sp sp sp sp sp sp sp sp sp sp sp 8 0002700 nl D e f a u l t sp d i r e c t o 0002720 r y sp h a s h : sp sp sp h a l f _ 0002740 m d 4 nl D i r e c t o r y sp H a 0002760 s h sp S e e d : sp sp sp sp sp sp 6 e 0003000 b 5 1 7 c 2 - 7 a f 4 - 4 1 3 1 0003020 - 8 2 e b - f b e 2 1 d 9 a 9 3 0003040 0 c nl J o u r n a l sp b a c k u 0003060 p : sp sp sp sp sp sp sp sp sp sp sp i n o 0003100 d e sp b l o c k s nl 0003112
Note that the "Filesystem flags: signed_directory_hash " entry ends with a trailing space character.
gdha commented at 2015-06-11 14:52:¶
@abbbi are the 2 grep executables the same?
abbbi commented at 2015-06-11 15:12:¶
hi,
yes, its been only a symlink.
gdha commented at 2015-06-12 10:38:¶
@abbbi @jsmeix I've updated/sanitized the /usr/share/rear/layout/save/GNU/Linux/23_filesystem_layout.sh script. Your feedback is more then welcome.
gdha commented at 2015-06-30 10:29:¶
move the milestone, however, it should be fixed in the meantime. Lacking good feedback to close the issue.
[Export of Github issue for rear/rear.]