#1400 Issue closed: RFC: biosdevnames auto detection

Labels: enhancement, fixed / solved / done, special hardware or VM

gdha opened issue at 2017-06-30 17:49:

  • rear version (/usr/sbin/rear -V): 2.1.x
  • OS version (cat /etc/rear/os.conf or lsb_release -a): ubuntu16
  • rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):
    In my rear-automated-testing environment (in centos7) I use the following entry to keep the network alias names with the kernel option:
KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"

See also the issue at https://github.com/gdha/rear-automated-testing/issues/24

  • Are you using legacy BIOS or UEFI boot? BIOS
  • Brief description of the issue: ubuntu vagrant box uses biosdevnames for network interfaces and not the alias eth names
root@client:~# ip addr show enp0s8
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:e0:e7:04 brd ff:ff:ff:ff:ff:ff
    inet 192.168.33.10/24 brd 192.168.33.255 scope global enp0s8
  • Work-around, if any:
    I used another local.conf file without the KERNEL_CMDLINE line, so we keep using the biosdevnames in client/server and recover VMs.

That is tedious and stupid, therefore, IMHO we better do some auto-detection of biosdevnames and

  • add the KERNEL_CMDLINE lines automatically, or
  • propose the user to add the KERNEL_CMDLINE line himself

PS: this is also related to previous issues #951 and #1020

schabrolles commented at 2017-08-26 13:10:

@gdha, I think adding KERNEL_CMDLINE with net.ifname=0won't help if the original system is using biosdevname.
When booting on ReaR rescue, network configuration script will look at interface name enp0s8 (Name on the original system) But because of net.ifname=0, Interfaces name will be named eth0.
So even when trying to recover on the same machine (same HW, no migration) ReaR recover cannot find the interface.

I usually don't like when an application force the user to modify the current system.
We should add net.ifname to KERNEL_CMDLINE only if it is present on the KERNEL CMDLINE option of the system to backup. (Checking /proc/cmdline).

Moreover, as I explain in #1020 there is several way to activate biosdevname. One is a script managed by systemd (which override KERNEL net.ifname parameter if I remember well).

IMHO we should rethink the way we are assigning IP address in the recovery image (based on MAC).
I think creating some function that mimic ip command but with MAC instead of device name could help.

ip_mac a add 10.1.1.1/24 dev 00:01:02:03:04:05

This wrapper function will look for interface name with 00:01:02:03:04:05 MAC and then run the real ip command with the good interface name (could be eth0 eth1 enp1s2 etc...)

If the MAC address cannot be found, then we should ask the user to choose the interface from a list.

This can be also useful for new system like ubuntu, rhel7 ... which has drop undev/70-persistent-net rules. Today, when you migrate a "multi inet" rhel7 to a NEW system (means new MAC)`, inet migration is not detected and IP is "blindly" assign to the same inet name (eth0 for exemple.. You need to be sure to get the inet in the same order (easy for VM, less for Bare metal system with 1GB for admin, 10Gb or even 40Gb for production.... the order will be given by the kernel module load order.)

My final thought in one sentence: It is time to rethink the network configuration / migration part. (Today mostly based on device name, based on udev persitent-net rules which are now dropped in distro like ubunu & redhat.)

gdha commented at 2017-08-28 07:38:

@schabrolles I fully understands the business concerns, however, we should not rush and change blindly our network scheme to a new model as we may potentially break more then we fix.
For example, in a SAP HANA environment we can have 6 or more network interfaces - in case we do a migration to another HW then it could be a nightmare for the administrator to guess which is the proper interface based on the MAC addresses to choose from to perform the recovery. IMOH asking which interface is not always useful as he/she just does not know the answer on front. He/she probably have to test it manually or on sight by investigating which interface should get the IP address assigned to.

I do agree with have to prepare ourselves for new challenges that come along with network layer.

schabrolles commented at 2017-08-28 10:27:

@gdha, don't worry, I don't want to change blindly everything in the network scheme but may be propose some evolution in order to use the latest distro.

I don't understand your point regarding SAP HANA migration with 6 adapters:

  • Currently, during a migration, some question are ask to the user to choose the new interface (devname mac are prompt to the user). This mean that the user needs to know on which new adapter he has to place the old network adapter. On the target eth0 will not necessary have the same name (order is not guaranteed ), and the only meaning of eth0 is the fact to be the first interface detected by the kernel (which has no interest for the user). So the only possibility for the user to choose the good interface is to use MAC or biosdevname (PCI location).
  • Worse: if the OS doesn't have an udev-net-rules there is no migration and network configuration will be blindly applied :

extract from 55-migrate-network-devices.sh

# get the rule files (though it should be only one)
RULE_FILES=( /etc/udev/rules.d/*persistent*{names,net}.rules )
ORIG_MACS_FILE=/etc/mac-addresses
MAC_MAPPING_FILE=/etc/rear/mappings/mac
MANUAL_MAC_MAPPING=

test "$RULE_FILES" || return 0 # skip this process if we don't have any udev rule files

But let this migration point aside for the moment. We still have an issue during recovery on the same system when a user disable biosdevname on the original OS. Unfortunately, because KERNEL option are not automatically applied on the Rescue image (#1420), the network interfaces are renamed following the default biosdevname, and because no migration occurred (no udev-rules-file), ReaR cannot setup the network (ip command with device name hardcoded in 60-network-devices.sh).

Solutions / workaround:

  • use KERNEL_CDMLINE to add 'net.ifname=0'. This could be automated by ReaR only if the option is detected in /proc/cmdline (IMHO)
  • Create a 60-network-devices.sh which is not linked to device name but HW mac. (MAC to dev and dev to MAC should be transparent for the user and automatically done by ReaR).

I made a first prototype which work now on the following scenario (ubuntu 16.04):

  • use net.ifnames=0on the original system (interface name: eth0)
  • run rear mkbackup
  • boot on the ISO, net.ifnames not set => interface name enp0s1. But IP and route are well setup due to the new ip_mac wrapper.
  • run rear recover
  • reboot on the restored system net.ifnames=0 (interface name backed to eth0)

example of new 60-network-devices.sh generated

# Network devices setup:
# Skip network devices setup if the kernel command line parameter 'noip' is specifiled:
grep -q '\<noip\>' /proc/cmdline && return
# When DHCP was used via 58-start-dhclient.sh do not change the existing networking setup here:
[[ ! -z "$USE_DHCLIENT" && -z "$USE_STATIC_NETWORKING" ]] && return
# If IPADDR=1.2.3.4 has been defined at boot time via ip=1.2.3.4 setup network devices this way:
if [[ "$IPADDR" ]] && [[ "$NETMASK" ]] ; then
    device=${NETDEV:-eth0}
    ip link set dev "$device" up
    ip addr add "$IPADDR"/"$NETMASK" dev "$device"
    if [[ "$GATEWAY" ]] ; then
        ip route add default via "$GATEWAY"
    fi
    return
fi
ip_mac addr add 10.7.19.141/24 dev 00:1a:4a:16:01:e3
ip_mac link set dev 00:1a:4a:16:01:e3 up
ip_mac link set dev 00:1a:4a:16:01:e3 mtu 1500

ip_mac() function

# ip wrapper command which use mac addresses instead of device name.
ip_mac(){
    ip_args=$*
    dev_mac=$(echo $ip_args | awk -F"dev " '{ split($2,TAB," "); print TAB[1] }')
    dev_name=$(get_device_by_hwaddr $dev_mac)
    ip_args=$( echo $ip_args | sed "s/dev $dev_mac/dev $dev_name/g" )
    ip  $ip_args
}

full commit changes available here: https://github.com/schabrolles/rear/commit/813e9d24642646ec53bda69bcba67e7757b44658
feel free to test it during your ubuntu 16.04 automated test.

There is still a lot of work/test before thinking to merge it into master.

  • checking variable
  • comments
  • vlan and bonding function need also to be modified.
  • Add Migration when udev-rules is not available. (the most difficult part)

gdha commented at 2017-10-11 12:06:

@schabrolles Are these changes already merged? Or, are you waiting on my test results? I'm sorry, but I was too busy with other customer activities (cannot remember it anymore what I did).

schabrolles commented at 2017-10-11 14:52:

@gdha, everything is now merged. Just test it when you have time on your Ubuntu 16.04. It should work with or without set.ifnames=0 parameter.

gdha commented at 2017-12-12 14:58:

@schabrolles are you sure it is merged? I cannot find ip_mac keywords in our code.
I just test it on ubuntu16.04 and the current code did not recognize biosdevnames automatically.

gdha commented at 2018-08-29 15:42:

https://gist.github.com/gdha/408f266e446ee51bfb758e2ed9982d8d#file-rear-automated-test-sh-log-L117 proofs that the latest PR does its job properly - we're good to close this case

[Export of Github issue for rear/rear.]