#1216 Issue closed: Implement FIRMWARE_FILES support

Labels: enhancement, fixed / solved / done

jsmeix opened issue at 2017-03-03 14:43:

I would like to implement a new variable
FIRMWARE_FILES
where the user can specify which firmware files
get included in the ReaR recovery system.

This enhancement is based on the experience in
https://github.com/rear/rear/issues/1142#issuecomment-276015345

It seems more than the half of ReaR's initrd
on my x86_64 system is used for firmware.

and
https://github.com/rear/rear/issues/1142#issuecomment-283324917

After removing /usr/lib/firmware contents
except for /usb/lib/firmware/rtl_nic ...
I ended up with ...
initrd.xz with xz lzma compression (48327540 bytes)

In particular an explicit FIRMWARE_FILES variable
that is documented in default.conf gives the user the final power
to decide what gets included in the ReaR recovery system.
Regarding "give the user the final power" cf.
"too much secretly working 'magic automatisms' in ReaR" in
https://github.com/rear/rear/pull/1204#issuecomment-282252947

Furthermore the FIRMWARE_FILES variable could be
a good template case for variables have ternary semantics, cf.
https://github.com/rear/rear/pull/1212#issuecomment-283698753

My currently planned ternary semantics for
FIRMWARE_FILES will be:

FIRMWARE_FILES=""
means "whatever reasonable automated behaviour"

FIRMWARE_FILES="yes" (or any explicit 'true' value)
means "include all firmware files in the ReaR recovery system"

FIRMWARE_FILES="no" (or any explicit 'false' value)
means "include no firmware file in the ReaR recovery system"

Finally the FIRMWARE_FILES variable might become
also a template case for variables that have multiple semantics:

For example in addition to the ternary semantics above
the FIRMWARE_FILES could also be an array
of explicitly listed firmware files that will be included in the
ReaR recovery system and/or filename globbing patterns
that are used as '-iname' arguments for 'find' calls to find
firmware files in the /lib/firmware/ directory that will get
included in the ReaR recovery system.

Regarding variables that can have multiple semantics
cf. how UEFI_BOOTLOADER can be specified since
https://github.com/rear/rear/pull/1212
is merged (see the current default.conf).

jsmeix commented at 2017-03-03 14:48:

With a FIRMWARE_FILES variable
that gives the user the final power to decide
what gets included in the ReaR recovery system
it may then even work on ppc64le in BareMetal Mode (PowerNV)
with yaboot, cf.
https://github.com/rear/rear/issues/1142#issuecomment-276050900

ProBackup-nl commented at 2017-03-03 15:48:

I did clean up my firmware directory to just over 92K.

I think I'd rather see a warning message during rescue creation, in case firmware is relative large, as a reminder to clean up the firmware folder on that (new) machine too.

ProBackup-nl commented at 2017-03-03 21:32:

On second thought, this feature is actually quite usable: I don't need networking to restore my system from the USB drive. So I don't need any firmware at all copied into the initrd file.

jsmeix commented at 2017-03-06 09:20:

ReaR will not do things like "clean up the firmware folder".

In general ReaR must not change any existing file
of the original system.
The only thing what happens during "rear mkbackup/mkrescue"
on the original system is that ReaR creates its working directory
where ReaR builds its rescue/recovery system ISO image
(cf. TMPDIR in default.conf).

The only exception that I know about where ReaR can change
even crucial files of the original system is GRUB_RESCUE
but that functionality is disabled by default (cf. default.conf).

During "rear recover" ReaR should not change the recreated system.
During "rear recover" ReaR should recreate the system as much as
possible as it was before (except in case of bigger new harddisk
where ReaR will try to adapt the partitioning to the new harddisk
by its so called "migration mode").
During "rear recover" ReaR will restore the files of the original
system as they were saved in the backup during "rear mkbackup"
and there is the special BACKUP_RESTORE_MOVE_AWAY
functionality that does nothing by default (see default.conf).

jsmeix commented at 2017-03-06 11:35:

I will use the variable name FIRMWARE_FILES
(instead of my initial idea REAR_INITRD_FIRMWARE)
to have this variable name in compliance with
the existing variable name KERNEL_FILE.

ProBackup-nl commented at 2017-03-09 23:14:

Some more experience is shared. Even after cleaning up the host machine its firmware directory,

Finished running 'build' stage in 21 seconds

of running mkrescue workflow, which finishes in 31 seconds in total.

With 68% of time in the build stage, that phase is longing for speed improvements.

Like excluding certain copy paths in build/GNU/Linux/400_copy_modules.sh, a variable where a more advanced user can specify exclusions like for example:

/lib/modules/*-ARCH/kernel/drivers/net/
/lib/modules/*-ARCH/kernel/fs/cifs/
/lib/modules/*-ARCH/kernel/net/

When not needing networking in the rescue system, exclude SSH permissions adjustment from build/GNU/Linux/150_adjust_permissions.sh

jsmeix commented at 2017-03-10 08:22:

@ProBackup-nl
regarding what kernel modules get included in the recovery system see
https://github.com/rear/rear/issues/1202
and regarding what I intend to improve there see in particular
https://github.com/rear/rear/issues/1202#issuecomment-284377443
but all this happens "as time permits" for me which currently means
it has to wait because currently I have to work on other areas
which is currently mainly printing related stuff, cf.
https://en.opensuse.org/User:Jsmeix

jsmeix commented at 2017-03-27 09:06:

With https://github.com/rear/rear/pull/1256 merged,
this issue should be fixed.


[Export of Github issue for rear/rear.]