#2575 Issue closed: Possibly obsolete '-F 16' for mkfs.vfat and 'fat=16' mount option in 700_create_efibootimg.sh and 200_mount_bootimg.sh¶
Labels: enhancement, fixed / solved / done, minor bug
jsmeix opened issue at 2021-02-25 11:56:¶
In current GitHub master code there is
# find usr/share/rear/ -type f | xargs grep 'fat=16'
usr/share/rear/output/ISO/Linux-i386/700_create_efibootimg.sh:
mount $v -o loop -t vfat -o fat=16 $TMP_DIR/efiboot.img $TMP_DIR/efi_virt >&2
usr/share/rear/output/ISO/Linux-ia64/200_mount_bootimg.sh:
mount $v -o loop -t vfat -o fat=16 $TMP_DIR/boot.img $TMP_DIR/mnt >&2
On my openSUSE Leap 15.2 system 'man 8 mount' reads (excerpts):
Mount options for fat
(Note: fat is not a separate filesystem, but a common part
of the msdos, umsdos and vfat filesystems.)
...
fat={12|16|32}
Specify a 12, 16 or 32 bit fat.
This overrides the automatic FAT type detection routine.
Use with caution!
This indicates that -o fat=16 is not actually needed
and could be even wrong in some cases.
I think relying on "the automatic FAT type detection"
seems to be the more fail-save way
so I suggest to remove the -o fat=16 option setting
in the above mount commands.
We at SUSE got an issue report that reads (excerpts):
noticed when running "rear mkbackup" the line
"FAT-fs (loop0): "fat=16" option is obsolete, not supported now"
is shown in /var/log/messages
/var/log/messages
2021-02-18T11:54:42.061371+01:00 ... kernel: [ 2415.490068] FAT-fs (loop0): "fat=16" option is obsolete, not supported now
2021-02-18T11:59:32.535396+01:00 ... rear[3641]: rear mkbackup finished with zero exit code
Removing -o fat=16 makes the message disappear.
So far no negative impact found by doing that.
jsmeix commented at 2021-02-25 13:39:¶
On my SLES11 test system
'man 8 mount' also reads
Mount options for fat
(Note: fat is not a separate filesystem,
but a common part of the msdos, umsdos and vfat filesystems.)
...
fat={12|16|32}
Specify a 12, 16 or 32 bit fat.
This overrides the automatic FAT type detection routine.
Use with caution!
Regarding "I wonder if -F 16 is actually needed for mkfs.vfat ?" in
https://github.com/rear/rear/pull/2576#issuecomment-785893981
on my SLES11 test system
'man mkfs.vfat' also reads
-F FAT-size
Specifies the type of file allocation tables used (12, 16 or 32 bit).
If nothing is specified, mkfs.fat will automatically select between
12, 16 and 32 bit, whatever fits better for the filesystem size.
So it seems it is safe at least down to SLES11
to remove both -F 16 for mkfs.vfat
and -o fat=16 for mount -t vfat
OliverO2 commented at 2021-02-26 11:56:¶
Specifying a mount option for parameters which can be read from the file
system itself (like -o fat=16) seems always questionable, if it makes
sense at all.
Regarding mkfs.vfat there is this comment section in
usr/share/rear/output/RAWDISK/Linux-i386/280_create_bootable_disk_image.sh
summarizing my research on the issue:
# Note: Having a small EFI System Partition (ESP) might introduce problems:
# - The UEFI spec seems to require a FAT32 EFI System Partition (ESP).
# - syslinux/Legacy BIOS fails to install on small FAT32 partitions with "syslinux: zero FAT sectors (FAT12/16)".
# - Some firmwares fail to boot from small FAT32 partitions.
# - Some firmwares fail to boot from FAT16 partitions.
# See:
# - http://www.rodsbooks.com/efi-bootloaders/principles.html
# - http://lists.openembedded.org/pipermail/openembedded-core/2012-January/055999.html
# As there seems to be no silver bullet, let mkfs.vfat choose the 'right' FAT partition type based on partition size
# (i.e. do not use the '-F 32' option) and hope for the best...
jsmeix commented at 2021-02-26 12:31:¶
@OliverO2
thank you for your explanatory comment (in your source code)!
It helps a lot.
jsmeix commented at 2021-02-26 14:06:¶
I think relying on mkfs.vfat to choose the 'right' FAT partition type
(based on whatever mksf.vfat implements for that e.g. based on size)
is currently also the 'right' thing in ReaR.
With
https://github.com/rear/rear/pull/2576
the code gets explanatory comments
so users could see why things are currently as is and easily adapt it if
needed
for exceptional cases.
When more issues appear because of this (i.e. the FAT partition type of
mkfs.vfat
was not the 'right' one for the UEFI firmware) we can introduce
something like a
ESP_FAT_TYPE config variable so that the user can specify what he
needs.
jsmeix commented at 2021-03-01 12:40:¶
Mainly for direct documenation here what the links in
https://github.com/rear/rear/issues/2575#issuecomment-786602931
contain:
http://www.rodsbooks.com/efi-bootloaders/principles.html
reads (excerpt)
The ESP should officially use a FAT32 filesystem,
although some (especially older) Linux distributions
place a FAT16 filesystem on this partition.
(This choice can cause problems for Windows 7,
which will accept only a FAT32 ESP.
At least one EFI implementation, Gigabyte's Hybrid EFI,
also seems to be allergic to FAT16 ESPs.)
I.e. the ESP should officially use a FAT32
and it seems for "Linux" (perhaps actually GRUB is meant) also a FAT16
ESP works
but some EFI firmware may have issues with a FAT16 ESP
so having a FAT32 ESP should be right in any case (in theory)
BUT:
https://www.openembedded.org/pipermail/openembedded-core/2012-January/055999.html
reads (excerpts):
Do not force FAT32 on all images, it violates the FAT specification
...
forcing the use of FAT32 with "-F 32" to mkdosfs.
The FAT specification is clear on cluster count being the determining
factor for FAT size (even if the fs string is set to FAT32, go figure).
Syslinux follows this spec, and rightly so, resulting in a failure
on core-image-minimal:
syslinux: zero FAT sectors (FAT12/16)
Drop the "-F 32" from mkdosfs to allow it to select the appropriate
FAT size based on cluster count.
...
- mkdosfs -F 32 -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${IMG} ${BLOCKS}
+ mkdosfs -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${IMG} ${BLOCKS}
so in the end not enforcing FAT32 for the ESP seems to be
"the right thing" in practice.
FYI when mkfs.vfat automatically switches from FAT16 to FAT32:
On my openSUSE Leap 15.2 system
with mkfs.vfat from dosfstools-4.1
mkfs.vfat automatically uses FAT16 up to 511 MiB
and automatically switches to FAT32 at 512 MiB:
Details (long lines shown wrapped here):
# for size in 511 512 ; \
do echo Creating FAT image with $size MiB size ; echo ; \
dd if=/dev/zero of=fat.$size.img count=$size bs=1M ; echo ; \
ls -lh fat.$size.img ; echo ; \
mkfs.vfat -v fat.$size.img ; echo ; \
file -s fat.$size.img ; \
echo ============================================== ; done
Creating FAT image with 511 MiB size
511+0 records in
511+0 records out
535822336 bytes (536 MB, 511 MiB) copied, 1.55126 s, 345 MB/s
-rw-r--r-- 1 root root 511M Mar 1 13:19 fat.511.img
mkfs.fat 4.1 (2017-01-24)
fat.511.img has 64 heads and 32 sectors per track,
hidden sectors 0x0000;
logical sector size is 512,
using 0xf8 media descriptor, with 1046528 sectors;
drive number 0x80;
filesystem has 2 16-bit FATs and 16 sectors per cluster.
FAT size is 256 sectors, and provides 65373 clusters.
There are 16 reserved sectors.
Root directory contains 512 slots and uses 32 sectors.
Volume ID is be72868f, no volume label.
fat.511.img: DOS/MBR boot sector, code offset 0x3c+2,
OEM-ID "mkfs.fat", sectors/cluster 16, reserved sectors 16,
root entries 512, Media descriptor 0xf8, sectors/FAT 256,
sectors/track 32, heads 64, sectors 1046528 (volumes > 32 MB),
serial number 0xbe72868f, unlabeled,
FAT (16 bit)
==============================================
Creating FAT image with 512 MiB size
512+0 records in
512+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 1.31168 s, 409 MB/s
-rw-r--r-- 1 root root 512M Mar 1 13:19 fat.512.img
mkfs.fat 4.1 (2017-01-24)
Auto-selecting FAT32 for large filesystem
fat.512.img has 255 heads and 63 sectors per track,
hidden sectors 0x0000;
logical sector size is 512,
using 0xf8 media descriptor, with 1048576 sectors;
drive number 0x80;
filesystem has 2 32-bit FATs and 8 sectors per cluster.
FAT size is 1024 sectors, and provides 130812 clusters.
There are 32 reserved sectors.
Volume ID is be87fee4, no volume label.
fat.512.img: DOS/MBR boot sector, code offset 0x58+2,
OEM-ID "mkfs.fat", sectors/cluster 8, Media descriptor 0xf8,
sectors/track 63, heads 255, sectors 1048576 (volumes > 32 MB),
FAT (32 bit),
sectors/FAT 1024, serial number 0xbe87fee4, unlabeled
==============================================
jsmeix commented at 2021-03-01 12:46:¶
Because mkfs.vfat uses FAT16 up to 511 MiB and FAT32 starting at 512
MiB
our current default
USB_UEFI_PART_SIZE="400"
results a FAT16 ESP by default which may cause issues with certain UEFI firmware.
So I think we should increase our default ESP size to
USB_UEFI_PART_SIZE="512"
to get by default a FAT32 ESP to be in compliance that
"the ESP should officially use a FAT32 filesystem"
to avoid possible issues with certain UEFI firmware.
jsmeix commented at 2021-03-02 08:50:¶
I increased USB_UEFI_PART_SIZE from 400 MiB to 512 MiB via
https://github.com/rear/rear/pull/2576/commits/c38e61db066196c90e6118cab8887b76df58b20a
On my openSUSE Leap 15.2 x86_64 system
I get a 336M ISO that contains a 326M initrd with the defaults
MODULES=( 'all_modules' ) and FIRMWARE_FILES=()
i.e. including all kernel modules and all firmware files:
# usr/sbin/rear -D mkrescue
...
Copying all kernel modules in /lib/modules/5.3.18-lp152.63-default (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
...
Created initrd.cgz with gzip default compression (341002623 bytes) in 55 seconds
Making ISO image
Wrote ISO image: /root/rear.github.master/var/lib/rear/output/rear-linux-h9wr.iso (336M)
so the current default USB_UEFI_PART_SIZE="400" is rather limited
when in particular the firmware files get more and more
(on my openSUSE Leap 15.2 system I have 496M in /lib/firmware/).
jsmeix commented at 2021-03-04 12:36:¶
With
https://github.com/rear/rear/pull/2576
merged
this issue should be done.
[Export of Github issue for rear/rear.]