#3374 Issue closed: When performing rear recover, Unable to mount the NFS

Labels: support / question, fixed / solved / done

shaikdba opened issue at 2024-12-26 22:38:

  • ReaR version ("/usr/sbin/rear -V"):
    /usr/sbin/rear -V
    Relax-and-Recover 2.6 / 2020-06-17

  • If your ReaR version is not the current version, explain why you can't upgrade:

  • OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
    "cat /etc/os-release"

NAME="Oracle Linux Server"
VERSION="8.9"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="8.9"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Oracle Linux Server 8.9"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:8:9:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://github.com/oracle/oracle-linux"

ORACLE_BUGZILLA_PRODUCT="Oracle Linux 8"
ORACLE_BUGZILLA_PRODUCT_VERSION=8.9
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=8.9
  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
    "cat /etc/rear/local.conf"
OUTPUT=ISO
OUTPUT_URL="nfs://X.X.X.X/export/bmrtest"
ISO_DEFAULT=manual
BACKUP=NETFS
BACKUP_URL="nfs://X.X.X.X/export/bmrtest"
BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/media' '/var/tmp' '/var/crash')
NETFS_KEEP_OLD_BACKUP_COPY=y
USE_DHCLIENT=y
PROGS=( "${PROGS[@]}" showmount mount.nfs umount.nfs )
MODULES=( "${MODULES[@]}" nfs )
PRE_RECOVERY_SCRIPT="systemctl start rpcbind.target || rpcbind &"
GRUB_RESCUE=1

  • Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
    KVM Guest

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
    x86 compatible

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
    BIOS

  • Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
    Local disk

  • Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):
    Please check log file

  • Description of the issue (ideally so that others can reproduce it):

When performing rear recover, Unable to mount the NFS,
NFS mount can be mounted on this Vm when not in recovery mode.
But when in recovery the mount fails

  • Workaround, if any:
    None

  • Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

RESCUE vmhost2902xm-vm02:~ # rear recover
Relax-and-Recover 2.6 / 2020-06-17
Running rear recover (PID 1654)
Using log file: /var/log/rear/rear-vmhost2902xm-vm02.log
Running workflow recover within the ReaR rescue/recovery system
Running PRE_RECOVERY_SCRIPT 'systemctl start rpcbind.target || rpcbind &'
Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
Started RPC portmapper 'rpcbind'.
RPC portmapper 'rpcbind' available.
Started rpc.statd.
RPC status rpc.statd available.
Started rpc.idmapd.
ERROR: Mount command 'mount -v -t nfs -o rw,noatime X.X.X.X:/export/bmrtest /tmp/rear.p0bowR6I9nPFN3S/outputfs' failed.
Some latest log messages since the last called script 060_mount_NETFS_path.sh:
  2024-12-26 10:36:05.664142296 Including verify/NETFS/default/060_mount_NETFS_path.sh
  mkdir: created directory '/tmp/rear.p0bowR6I9nPFN3S/outputfs'
  2024-12-26 10:36:05.668335375 Mounting with 'mount -v -t nfs -o rw,noatime X.X.X.X:/export/bmrtest /tmp/rear.p0bowR6I9nPFN3S/outputfs'
  mount.nfs: mount(2): No such device
  mount.nfs: No such device
  mount.nfs: timeout set for Thu Dec 26 10:38:05 2024
  mount.nfs: trying text-based options 'vers=4.2,addr=X.X.X.X,clientaddr=0.0.0.0'
Aborting due to an error, check /var/log/rear/rear-vmhost2902xm-vm02.log for details
Exiting rear recover (PID 1654) and its descendant processes ...
Running exit tasks
Terminated
RESCUE vmhost2902xm-vm02:~ #

backup.log
rear-vmhost2902xm-vm02-1.txt
FS layout.txt

abbbi commented at 2025-01-05 11:46:

lsmod | grep nfs

in the recovery image shows?

can you load the nfsd module within the bootet recovery image via

modprobe nfsd

?

It seems your bootet recovery image is missing the required kernel modules for mounting an
NFS share:

/sbin/modprobe: line 52: flock: command not found
/sbin/modprobe: line 60: /sbin/modprobe.ksplice-orig: No such file or directory
/sbin/modprobe: line 52: flock: command not found
/sbin/modprobe: line 60: /sbin/modprobe.ksplice-orig: No such file or directory
/sbin/modprobe: line 52: flock: command not found
/sbin/modprobe: line 60: /sbin/modprobe.ksplice-orig: No such file or directory
mount: /var/lib/nfs/rpc_pipefs: unknown filesystem type 'rpc_pipefs'.
mount: /proc/fs/nfsd: unknown filesystem type 'nfsd'.

or the modules have not been loaded during boot.

What does "dmesg" show while you attempt to load the nfsd module, if it fails?

jsmeix commented at 2025-01-07 09:36:

So it seems

MODULES=( "${MODULES[@]}" nfs )

is the root cause why some needed kernel modules
are missing in the ReaR recovery system.

@shaikdba
see the MODULES description in default.conf
e.g. for ReaR 2.6 online at
https://github.com/rear/rear/blob/rear-2.6/usr/share/rear/conf/default.conf#L1249

jsmeix commented at 2025-01-07 09:39:

@abbbi
thank you for analyzing what the actual root cause was!

[Export of Github issue for rear/rear.]