#1510 PR `merged`: Improve ReaR network migration¶

Labels: enhancement, fixed / solved / done

schabrolles opened issue at 2017-09-21 15:01:¶

The objective of this PR is to update the network migration process in ReaR.

Before this PR, network migration was based on udev rules files to detect and update mac or interface name.

A) The problem is that newer Linux distro (rhel7, ubuntu) does not automatically update anymore the udev network rule files.

This prevent ReaR :

to propose new interface in case of migration
to rename interface and properly activate network in recovery mode
to perform Network Migration on the restored image.

=> Migration is also currently impossible with biosdevname network interface name format (which is the new default for ubuntu/redhat.)

B) Network parameter migration (ip/routes via /etc/rear/mapping/ip_addresses or /etc/rear/mapping/routes) are only compatible with RedHat/Suse sysconfig file format. (not debian/ubuntu)

This PR propose a migration compatible with latest RH / ubuntu (not based in udev rules but biosdevname) while keeping compatibility with udev rules for older distro.

always propose new interface (even when udev rules is not used or empty)
keep udev rules renaming as default (for compatibility)
perform a direct migration of network system-setup scripts (only if udev migration cannot be done.) => to activate network in rescue mode
Allow a migration of debian/ubuntu network configuration files (currently only redhat/suse)

Tested in migration on POWER with rhel 7.3 / suse 11 / suse 12 / ubuntu 16.04 (with and without net.ifnames=0)

jsmeix commented at 2017-09-22 09:12:¶

Only for the fun of it:
Perhaps the initial comment in
finalize/GNU/Linux/320_migrate_network_configuration_files.sh
is no longer fully right:
"rewrite all the network configuration files for SUSE LINUX"
;-)

schabrolles commented at 2017-09-22 09:49:¶

@jsmeix thanks for the quick review and trust :-) ... But I will wait the other to test this one with vagrant to cross check and hunt any regression / issues.
Especially @gdha as I think it should solves some of the issues he had with ubuntu and biosdevames (#1400, #951 and #1020)

jsmeix commented at 2017-09-22 10:04:¶

@schabrolles
don't overestimate my trust here ;-)
Basically all I can do is to "bindly trust" you here
because I am mostly clueless in this area
so better do not trust my review here.

jsmeix commented at 2017-09-22 11:34:¶

@schabrolles
because you added your functions to lib/network-functions.sh
I had a first look at that script (I never had something to do with it)
and I saw its strange looking functions (some of them even
just call 'exit' which is totally unusual in "normal" ReaR scripts).

Now I am wondering (in particular after reading its initial comment)
if lib/network-functions.sh was ever meant to be sourced
by usr/sbin/rear to make its strange functions avaiable
to all ReaR scripts?

In current (i.e. without this pull request) code
lib/network-functions.sh is explicitly sourced by
skel/default/bin/dhclient-script and
skel/default/etc/scripts/system-setup.d/58-start-dhclient.sh
which matches the initial comment in lib/network-functions.sh

Currently the only "normal" ReaR script that calls
a function from lib/network-functions.sh is
finalize/GNU/Linux/320_migrate_network_configuration_files.sh
which calls only the prefix2netmask function and that call
was added by @gdha via
https://github.com/rear/rear/commit/8472c46b31de2b748a6f948325101a970c452e61

From my current point of view lib/network-functions.sh
was never meant to be sourced by usr/sbin/rear to have
its functions available in all "normal" ReaR scripts.

From my current point of view lib/network-functions.sh
is only meant to be sourced and used to setup dhclient
during recovery system initialization via the 'skel' scripts.

Simply put:
I think some general cleanup is needed here.
I think this should be done as a separated issue:
I think network-functions.sh should be renamed into
somethinmg like dhclient-setup-functions.sh
and moved away from usr/share/rear/lib
probably into usr/share/rear/skel and the two 'skel' scritps
that source it should be adapted.

@schabrolles
accordingly I think that your new networking functions
should be implemented in a new separated file
so that later we can do the dhclient setup functions
cleanup.

The problem is that lib/network-functions.sh is the exact right name
for generic networking functions for normal ReaR scripts
so that the current lib/network-functions.sh should be right now
renamed e.g. into dhclient-setup-functions.sh but then the two
'skel' scripts need to be adapted right now which means
that general cleanup needs to be done right now :-(

@schabrolles
what do you think?

jsmeix commented at 2017-09-22 12:14:¶

@gdha
I see now that lib/network-functions.sh was added via your
https://github.com/rear/rear/commit/38d5bd280654dd4e05a8a408daad8e08925c3ab0

Can you explain if that strange looking functions therein
are really meant to be always available in all normal
ReaR scripts?

What I mean is:
None of the functions in lib/network-functions.sh
uses any of the standard ReaR functions that we have
but instead those functions in lib/network-functions.sh
implement all on their own so that I think those functions
basically come from a foreign world and live in that world
which makes them not well suitable to be used by normal
ReaR scripts.

jsmeix commented at 2017-09-22 12:19:¶

@schabrolles
I really dislike it when a specific issue like this one
out of a sudden "explodes" into a monstrous task
to overhaul major parts of ReaR.
I do want to Keep Separated Issues Separated (KSIS).
Therefore it should be best when you first merge
this pull request as soon as possible.
Afterwards we can do the lib/network-functions.sh
cleanup as time permits.

jsmeix commented at 2017-09-22 12:30:¶

I missed two functions in lib/network-functions.sh
that are currently called by a normal ReaR script:
is_ip and get_ip_from_fqdn are called in lib/bootloader-functions.sh
which was implemented by @schabrolles via
https://github.com/rear/rear/commit/ef0256256d38e8c52d7f3e17797ea9378f3727b4

schabrolles commented at 2017-09-25 13:27:¶

@jsmeix
I've source lib/network-functions.sh in the skel 00-functions.sh to have access to get_device_by_hwaddr() function in 55-migrate-network-devices.sh (also part of skel).
=> The purpose is to have to IP set to the good interface (recognized by its IP) during boot in rescue mode. (when /etc/scripts/system-setup and /etc/scripts/system-setup.d/* scripts are executed.)

I'm not sure that usr/sbin/rear will execute /etc/scripts/system-setup.d/00-functions.sh.... But I may be wrong.

jsmeix commented at 2017-09-26 09:22:¶

@schabrolles
just proceed here as you like and as it works for you
because as I wrote above we should do any
"grand cleanup" via a separated issue/pull-request
later at any time as time permits because currently
nothing goes wrong so that there is no need to do
such a "grand cleanup" right now.

gdha commented at 2017-09-26 09:24:¶

@schabrolles @jsmeix I agree that network-functions.sh is collection I grabbed from an old RHEL system to get the job done as easy as possible (at that time - many, many years ago).
I agree we better split it apart (network piece required for DHCP in skel and network functions which are required by main rear script). That will make it easier afterwards to clean it up.
I also agree to merge this PR asap to get things going as the PR is a good thing to have.
@schabrolles rear will not execute /etc/scripts/system-setup.d/00-functions.sh or source it as it is part of the booting up part of the rescue image (it is sourced by /etc/scripts/system-setup script)

schabrolles commented at 2017-09-26 09:39:¶

@gdha thanks for your review.
Did you try it on your ubuntu 16.04 systems (where you have biosdevname) ? It solves the issues on my side; I just wanted to know if it solves yours.

jsmeix commented at 2017-09-26 11:15:¶

Regarding the lib/network-functions.sh cleanup I submitted
https://github.com/rear/rear/issues/1517

gdha commented at 2017-09-26 15:54:¶

@schabrolles No, I did not test it yet with ubuntu16 yet as I was struggling with ubuntu14 provisioning (just got it working without errors). Pff - what people need to do to earn some money...
I would say just merge it and I'll test it upcoming Friday with Ubuntu16 (if I did not break the Ubuntu16 provisioning now as everything is now one cookbook).

[Export of Github issue for rear/rear.]

#1510 PR merged: Improve ReaR network migration¶