#1202 Issue `closed`: recovery system does not contain all kernel modules¶

Labels: enhancement, documentation, cleanup, fixed / solved / done

jsmeix opened issue at 2017-02-22 15:31:¶

I noticed that the ReaR recovery system does not
contain all kernel modules of the original system.

I think by default the ReaR recovery system should
contain all kernel modules of the original system.

This way even system migration could be much easier
because the ReaR recovery system would be prepared
by default for "all hardware" (via manual module loading).
In contrast when needed kernel modules are missing in the
ReaR recovery system it is a dead end for "rear recover".

Missing kernel modules can result any kind of
awkward unexpected failures like:

RESCUE e205:~ # mkdir /mp

RESCUE e205:~ # ls -l /dev/disk/by-label/
... SLE-12-SP2-Server-DVD-x86_642192 -> ../../sr0

RESCUE e205:~ # mount /dev/sr0 /mp
mount: unknown filesystem type 'iso9660'

In the recovery system I have

RESCUE e205:~ # find /lib/modules/ | wc -l
861

in contrast on the original system I have much more kernel modules:

# find /lib/modules/ | wc -l
2479

The easy workaround is

COPY_AS_IS=( "${COPY_AS_IS[@]:-}" /lib/modules )

With that

RESCUE e205:~ # mount /dev/sr0 /mp

just works because the needed kernel modules are
loaded full automagically (provided they exist ;-)

schlomo commented at 2017-02-22 16:24:¶

IMHO we copy all storage and network modules but skip all the other stuff
to keep the image size smaller.

On 22 February 2017 at 16:31, Johannes Meixner notifications@github.com
wrote:

I noticed that the ReaR recovery system does not
contain all kernel modules of the original system.

I think by default the ReaR recovery system should
contain all kernel modules of the original system.

This way even system migration could be much easier
because the ReaR recovery system would be prepared
by default for "all hardware" (via manual module loading).
In contrast when needed kernel modules are missing in the
ReaR recovery system it is a dead end for "rear recover".

Missing kernel modules can result any kind of
awkward unexpected failures like:

RESCUE e205:~ # mkdir /mp

RESCUE e205:~ # ls -l /dev/disk/by-label/
... SLE-12-SP2-Server-DVD-x86_642192 -> ../../sr0

RESCUE e205:~ # mount /dev/sr0 /mp
mount: unknown filesystem type 'iso9660'

In the recovery system I have

RESCUE e205:~ # find /lib/modules/ | wc -l
861

in contrast on the original system I have much more kernel modules:

find /lib/modules/ | wc -l¶

2479

The easy workaround is

COPY_AS_IS=( "${COPY_AS_IS[@]:-}" /lib/modules )

With that

RESCUE e205:~ # mount /dev/sr0 /mp

just works because the needed kernel modules are
loaded full automagically (provided they exist ;-)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rear/rear/issues/1202, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGMCG_nLrwlZHxuEVbTlptCPkBQh3itks5rfFTLgaJpZM4MIyrO
.

didacog commented at 2017-02-22 17:05:¶

Hi @jsmeix @schlomo

I agree with @schlomo about keeping smaller image size as possible, but on the other way I also understand the concern of @jsmeix.

Maybe something like SYSTEM_MIGRATION=no to keep the current behaviour and in case of a system migration or cloning, ... set to SYSTEM_MIGRATION=yes and in that case set:

COPY_AS_IS=( "${COPY_AS_IS[@]:-}" /lib/modules )

Maybe a better var name could be:

RESCUE_IMAGE_TYPE=[targeted|template] ???

Is just food for thoughts... :-P

regards,

jsmeix commented at 2017-02-23 08:38:¶

As a first step I will better document the current behaviour
in default.conf which contains currently only

# files/dirs to copy as-is (with tar)
COPY_AS_IS=( $SHARE_DIR $VAR_DIR )

jsmeix commented at 2017-03-06 11:58:¶

Currently in default.conf there is

# kernel modules to include on the rescue media in addition to the ones
# present at mkrescue time
MODULES=()

I think it should be relatively easy to add support
for values with special meaning something like

MODULES="all"

or

MODULES="none"

or

MODULES="currently_loaded"

plus support for find '-iwholename' patterns like

MODULES=( 'currently_loaded' '*btrfs*' '*/net/ethernet/*' )

This will not change the current default behaviour but
it will give the user some better means to specify if needed
what exact kernel modules get included in his recovery system.

ProBackup-nl commented at 2017-03-10 11:21:¶

IMHO Not even all storage and all networking needs to be copied onto the rescue system.

To "Relax" it will be nice when the average user needs as little configuration as possible, and a rescue system that only carries what is needed.

The user choice:

Do you wish to build a:
Minimal rescue system for fastest backup and restore on similar hardware
or
Hardware migration rescue system?

Further along the road OUTPUT, BACKUP_URL, and the host system itself do give clues what be needed on the rescue system.

Building blocks:

bare minimum
+MD (software RAID)
+LVM
+LUKS (disk encryption)
+Multipath disks
+Btrfs (or include in bare minimum?)
+DRBD (enables networking block too?)
+Hardware RAID controllers
+Tape drivers (?)
+networking: BACKUP_URL=(nfs|cifs|sshfs) or OUTPUT=PXE

~~Although "hardware migration" will require most, if not all blocks.~~ For hardware migration via USB, I still wouldn't need MD, LVM, LUKS, multipath, DRBD, Hardware RAID, Tape drivers or networking.

The building blocks inclusion is only needed for the included source disk(s) not the excluded source disk(s). For example it is pointless (from bare minimum rescue system point of view) to include HP SmartArray controllers when the machine has got such a controller, but the disk(s) that are being back-upped, are not attached to that controller.

Second user choice:

Based on your host system your rescue system will be made including and excluding the following building blocks:
Included: BTRFS, ...
Excluded: MD, LVM, ...
Do you wish to:
(A)ccept auto-selection
(M)anually change this selection

Note: these user choices can be interactive questions, or configuration file options.

jsmeix commented at 2017-03-17 12:55:¶

I like to enhance my proposal in
https://github.com/rear/rear/issues/1202#issuecomment-284377443
in a (hopefully) very useful way according to
what I learned in
https://github.com/rear/rear/pull/1244
in particular according to
https://github.com/rear/rear/pull/1244#issuecomment-286718903
USB keyboard function on any replacement hardware is a must
so that a plain

MODULES="none"

is probably never really useful.

Therefore I think it should be possible to specify something like

MODULES=( 'none' 'currently_loaded' '*/usb/*' )

which includes no modules except those that are currently loaded
plus all USB modules - i.e. which includes only those that are
currently loaded plus all USB modules.

The logical counterpart would be something like

MODULES=( 'all' '*/bluetooth/*' '*/firewire/*' )

which includes all modules except the modules
for bluetooth and firewire.

schlomo commented at 2017-03-17 14:27:¶

Do you maybe mean this?

MODULES=( 'all' '!*/bluetooth/*' '!*/firewire/*' )

jsmeix commented at 2017-05-10 08:57:¶

With https://github.com/rear/rear/pull/1352 merged
there is a major cleanup of linux-functions.sh
therein in particular the ResolveModules function
and the ModulesCopyTo function that are called
only by build/GNU/Linux/400_copy_modules.sh
as a preparation step that is intended to solve
this issue here.

jsmeix commented at 2017-05-12 13:01:¶

With https://github.com/rear/rear/pull/1359 merged
this issue should be solved.

Now there is support for some new special MODULES values
MODULES=( 'all_modules' )
MODULES=( 'loaded_modules' )
MODULES=( 'no_modules' )
This way users can better specify what kernel modules
get included in the rescue/recovery system.
See the explanation in default.conf.

This is not yet the full featured detailled MODULES support as in
https://github.com/rear/rear/issues/1202#issuecomment-287346514
but it is (hopefully) a first step forward in the right direction.

Because of what I noticed in
https://github.com/rear/rear/issues/1355
I think it may perhaps fail more often than not
if only certain kernel modules directories
or kernel modules file name globbing stuff
gets copied into the recovery system because
there could be so subtle kernel module dependencies
that in practice only simple things like
MODULES=( 'all_modules' )
MODULES=( 'loaded_modules' )
would actually work sufficiently well
(see what I described in in default.conf).

[Export of Github issue for rear/rear.]

#1202 Issue closed: recovery system does not contain all kernel modules¶