#558 Issue closed: Error recreating file systems (mkfs.ext4: invalid option -- 'f')¶
Labels: bug, documentation, support / question
cocampbe opened issue at 2015-03-04 19:16:¶
OS: OEL 6.6
REAR: 1.16.1-git201503031650
Kernel: 3.8.13-55.1.2.el6uek.x86_64
I recently updated from a 1.13 release because mounting nfs shares would not work. That issue was resolved in the update. Now the issue is in creating ext4 file systems. The diskrestore.sh script uses a -f for the fragment size, but the copy of mkfs.ext4 does not support fragment size and halts. The version f e2fsprogs on the system is e2fsprogs-1.42.8-1.0.2.el6.x86_64.
error from log file:
+++ mkfs.ext4 -b 4096 -f 4096 -i 16384 /dev/mapper/vg00-lvol3
mkfs.ext4: invalid option -- 'f'
Usage: mkfs.ext4 [-c|-l filename] [-b block-size] [-C cluster-size]
[-i bytes-per-inode] [-I inode-size] [-J journal-options]
[-G flex-group-size] [-N number-of-inodes]
[-m reserved-blocks-percentage] [-o creator-os]
[-g blocks-per-group] [-L volume-label] [-M last-mounted-directory]
[-O feature[,...]] [-r fs-revision] [-E extended-option[,...]]
[-t fs-type] [-T usage-type ] [-U UUID] [-jnqvDFKSV] device [blocks-count]
2015-03-04 12:46:17 An error occurred during layout recreation.
I removed the fragment size and was able to restore. The server came back online without issue.
gdha commented at 2015-03-05 08:23:¶
Thank you - I'll add it to the release notes to watch out for - will not
change the code as far I could verify all other distro's are OK with the
-foption
gdha commented at 2015-03-05 08:25:¶
@cocampbe could you post the /var/lib/rear/layout/disklayout.conf file
please. The fragmentsize must have been listed via e2label -
personally I think this is a defect in OEL 6.6 (not being consistent in
options)
cocampbe commented at 2015-03-05 15:01:¶
disk /dev/sda 450064605184 msdos
part /dev/sda 524288000 1048576 primary boot /dev/sda1
part /dev/sda 449539211264 525336576 primary lvm /dev/sda2
#disk /dev/sdb 53687091200
#disk /dev/sdc 26843545600
#disk /dev/sdd 53687091200
#disk /dev/sde 26843545600
#disk /dev/sdf 53687091200
#disk /dev/sdg 26843545600
#disk /dev/sdh 53687091200
#disk /dev/sdi 26843545600
#lvmdev /dev/vgtest /dev/mapper/test_lun uJOvjW-AvMq-bnYk-X3XY-bI5h-BWbj-5zW0oT 52428800
lvmdev /dev/vg00 /dev/sda2 8evCPt-9SzP-wgYP-VgBL-1JUV-mGAH-z4aoFU 878006272
#lvmgrp /dev/vgtest 4096 6396 26198016
lvmgrp /dev/vg00 4096 107178 439001088
#lvmvol /dev/vgtest data 1280 10485760
#lvmvol /dev/vgtest ctl 1280 10485760
#lvmvol /dev/vgtest arch 1280 10485760
lvmvol /dev/vg00 lv_swap 992 8126464
lvmvol /dev/vg00 lvol3 512 4194304
lvmvol /dev/vg00 lvol4 1280 10485760
lvmvol /dev/vg00 lvol5 2560 20971520
lvmvol /dev/vg00 lvol6 1280 10485760
lvmvol /dev/vg00 lvol7 2560 20971520
lvmvol /dev/vg00 lvol8 2560 20971520
lvmvol /dev/vg00 lvol9 1280 10485760
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/mapper/vg00-lvol3 / ext4 uuid=1081d79d-59d8-42ca-b39d-5a624021228c label= blocksize=4096 fragmentsize=4096 reserved_blocks=2% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=64,data=ordered
fs /dev/mapper/vg00-lvol4 /home ext4 uuid=385e252e-d2bc-4a14-b1d7-928a0828e160 label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=64,data=ordered
fs /dev/mapper/vg00-lvol5 /opt ext4 uuid=f56fc2dd-d4b9-442c-a5e7-bb7b030a367f label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=64,data=ordered
fs /dev/mapper/vg00-lvol6 /tmp ext4 uuid=1d6472ac-df06-427f-a71e-e27c310b608d label= blocksize=4096 fragmentsize=4096 reserved_blocks=2% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=64,data=ordered
fs /dev/mapper/vg00-lvol7 /usr ext4 uuid=7cf0269c-977c-4371-a42d-e1c74cbea3a7 label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=64,data=ordered
fs /dev/mapper/vg00-lvol8 /var ext4 uuid=f3f52bdb-c5c8-49f6-b33b-098ae02d8cb4 label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=64,data=ordered
fs /dev/mapper/vg00-lvol9 /oma ext4 uuid=e301a61d-1a5b-4162-b0c9-6564c26bc6f1 label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=64,data=ordered
#fs /dev/mapper/vgtest-arch /test/arch ext4 uuid=98c70e58-01cf-4ee9-afda-68ee30373b24 label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=4096,data=ordere
#fs /dev/mapper/vgtest-ctl /test/oractl ext4 uuid=dff9e845-6c78-403b-8393-0ec95f486fdb label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=4096,data=ordere
#fs /dev/mapper/vgtest-data /test/oradata ext4 uuid=302313c2-e6cc-44d6-822e-e23771ee169d label= blocksize=4096 fragmentsize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime,stripe=4096,data=ordere
fs /dev/sda1 /boot ext4 uuid=8be4a07a-5436-4ed7-b57c-35b82c0d94a6 label= blocksize=1024 fragmentsize=1024 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=4063 default_mount_options=user_xattr,acl options=rw,relatime,stripe=256,data=ordered
swap /dev/mapper/vg00-lv_swap uuid=f6001d7e-c83e-4606-aad5-c1f4ce538dff label=
logicaldrive /dev/sda 0|A|1 raid=1 drives=1I:1:1,1I:1:2, spares= sectors=32 stripesize=256
smartarray 0
#multipath /dev/mapper/mpathe /dev/sdb,/dev/sdd,/dev/sdf,/dev/sdh
#multipath /dev/mapper/test_lun /dev/sdc,/dev/sde,/dev/sdg,/dev/sdi
cocampbe commented at 2015-03-05 15:03:¶
@gdha I would think that this would affect RHEL instances as well. OEL is, for the most part, a binary equivalent of RHEL. And I would also see CENTOS as having this issue. I don't have a RHEL or CENTOS instance to test with.
cocampbe commented at 2015-03-05 21:33:¶
Argh. I am not sure why this is so. The man page for mkfs.ext4 shows the fragment size option, but compiled binary throws an error. The package is e2fsprogs-1.42.8-1.0.2.el6.x86_64. Again, I am going to assume that anyone with this package installed will see the issue. Balls.
schlomo commented at 2015-03-05 21:41:¶
A couple of years ago @dagwieers brought the concept of capabilities (with regard to syslinux). Maybe this approach would help here as well? Query which features mkfs.ext4 supports and include only compatible options?
cocampbe commented at 2015-03-05 21:59:¶
@schlomo I think that makes sense. But I feel the fragment size is not necessary. I cannot speak for all cases, but most people do not specify a block size or fragment size when creating file systems. It seems to me that needing those for a recovery is more a closet case, than the norm. But alas, I am not a developer. And you guys have done a great job maintaining this project. So I don't want to be too critical.
schlomo commented at 2015-03-05 22:09:¶
Well, a recovered system will always have some little differences from the original. But we try our best. I personally could also live with emitting a warning about the filesystem features that could not be recovered.
Probably same like sheep cloning is not yet perfect.
However, your initial problem was that recovery failed so there I see a bug which is caused by trying to be too perfect, at least for your version of mkfs.ext4. I guess that since a successful recovery is the main objective we either have to use that feature only if it is available or we have to remove support for setting the fragment size till we can do that.
@gdha what do you think?
gdha commented at 2015-03-06 07:07:¶
@schlomo @cocampbe @jsmeix Indeed Centos 7 has the same bug with
mkfs.ext4: invalid option -- 'f'
Centos 7 has rpm e2fsprogs-1.42.9-4.el7.x86_64 containing the
executable and man-page (which do describe the feature). The
fragmentsize was introduced recently via issue #549. I think it is
better that we just make the variable empty for the moment as it is an
unkown fact when e2fprogs will enable the fragmentsize feauture. Any
objections?
schlomo commented at 2015-03-06 07:49:¶
No. Can you make it transparent to the end user that the recreated filesystem is not 100% identical?
Also, I could imagine the recovery to fail on very large file systems and the recovered system to be inefficient on 4k hard disks. Or are these unrelated?
jsmeix commented at 2015-03-06 09:32:¶
I think rear should remove the newly added actual support for fragmentsize everywhere which makes rear again work as it actually did all the time before (i.e. all the time before rear did not actually support fragmentsize).
Reasoning:
In https://github.com/rear/rear/issues/549 I wrote
It seems ${fragmentsize} in usr/share/rear/layout/prepare/GNU/Linux/13_include_filesystem_code.sh is used but nowhere defined.
Because in the past ${fragmentsize} was nowhere defined, the value of ${fragmentsize} was empty so that the mkfs command was always done without any kind of fragmentsize support.
Because I had found out that ${fragmentsize} is used but nowhere defined, @gdha had added actual support for it by defining it in 13_include_filesystem_code.sh and 23_filesystem_layout.sh see https://github.com/rear/rear/commit/ca41fe5046729fd30a7832ad2985adb18896d631
But this issue here now shows that the actually right way how to deal with fragmentsize is to not use it because it does not work.
Therefore I suggest to remove both defining it and using it from 13_include_filesystem_code.sh and 23_filesystem_layout.sh
Any time later when the tools not only describe stuff in their man pages but even actually support it, then rear might even also support it ;-)
I think the rear code should not contain "fragmentsize" when it does not actually support it (i.e. I think having a variable ${fragmentsize} but with empty value because actually it does not work is misleading).
jsmeix commented at 2015-03-06 10:15:¶
@schlomo
I think it is not possible with reasonable effort to determine the cases
when the recreated filesystem is not 100% identical.
I think in gereral after a system was recreated with rear it is no longer 100% identical but there are zillions of individual possible cases what exactly is no longer 100% identical.
Therefore I suggest rear should make it transparent to its end users (by the way rear is not used by usual "end users" but by experienced admins) that in general the recreated system will not be 100% identical.
For example after recreation the system performance might unexpectedly speed up because tons of fragmented files are now contiguous on the disk and we should really warn the admin to be prepared against possible dumb questions from his big boss why the system wasn't already as fast before the recreation ;-)
And don't forget unexpectedly more free disk space after recreation because this or that big file that was removed but still in use by some process that runs "since ever" is now really gone after recreation :-)
schlomo commented at 2015-03-06 11:16:¶
+1 for removing it.
Thanks!
gdha commented at 2015-03-06 14:08:¶
Fine in script layout/save/GNU/Linux/23_filesystem_layout.sh I
disabled fragmentsize to be written into the disklayout.conf file.
I will not change the script
layout/prepare/GNU/Linux/13_include_filesystem_code.sh as
fragmentsize will not be found anymore in the disklayout.conf and is
therefore completely harmless.
cocampbe commented at 2015-03-09 12:24:¶
I just installed rear-1.16.1-151.git201503061713.el6.noarch.rpm. I'll let you know what happens.
cocampbe commented at 2015-03-09 13:10:¶
OK. It worked like a champ. Thanks so much.
cocampbe commented at 2015-03-17 18:33:¶
I just wanted to update with some info I got from oracle support
I got more information about this SR.
Fragment size was a feature of ext2/ext3 which was removed in ext4 as
part of implementing the 'bigalloc' feature:https://ext4.wiki.kernel.org/index.php/Design_for_Large_Allocation_Blocks
So, the information related with '-f' from manpage is wrong.
schlomo commented at 2015-03-18 10:48:¶
@cocampbe thanks a lot for finding this information, it explains a lot. I think that the man page is the same for mkfs.ext4 and mkfs.ext3, but of course they should be more specific.
Do you think that ReaR should do -f for ext3? Of course provided that
mkfs.ext3? actually does support it.
cocampbe commented at 2015-03-18 11:47:¶
@schlomo Personally, I would leave -f out. Usually the default settings for mkfs are sufficient. I would think most people are only using rear for the OS disk.
[Export of Github issue for rear/rear.]