#355 Issue closed: rear tries to perform Multiple NFS mount

Labels: support / question

avinash4any1 opened issue at 2014-01-09 15:07:

I have Suse 11 running on a VM. I have installed rear-1.15-5. I have made necessary changes to /etc/rear/local.conf

cat local.conf
# Create ReaR rescue media as ISO image
OUTPUT=ISO

# optionally define (non-default) backup software, e.g. TSM, NBU, DP, BACULA
BACKUP=NETFS
NETFS_URL=nfs://10.45.209.30/var/opt/ignite/recovery/archives/globiz101
NETFS_KEEP_OLD_BACKUP_COPY=yes
USE_LAYOUT=y
#if there is a need to exclude  mountpoints
EXCLUDE_MOUNTPOINTS=(
/mnt
/sapcd
/tmp
/usr/sap/CTP
/sapmnt/CTP
/sapdb/CCP
/sapdb/data
/sapdb/programs
/sapdb/CCP/sapdata1
/sapdb/CCP/saplog
/sapdb/CCP/sapdb_backup
/sapmnt/CCP
)
BACKUP_PROG_EXCLUDE=(
'/tmp/*'
'/dev/shm/*'
)

When i starting the backup, i come across below error

rear mkbackup -v
Relax-and-Recover 1.15 / Git
Using log file: /var/log/rear/rear-globiz101.log
The preparation phaseOK
Creating disk layout
Creating root filesystem layout
TIP: To login as root via ssh you need to set up /root/.ssh/authorized_keys or SSH_ROOT_PASSWORD in your configuration file
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
ERROR: Mount command 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.DsHGgsi6hKNy43F/outputfs' failed.
Aborting due to an error, check /var/log/rear/rear-globiz101.log for details
Terminated

Please suggest what changes i need to make to have it corrected.

Please find below rear logs from the point i get this error

mkdir: created directory `/tmp/rear.u2dPrxc7SDXdCnY/outputfs'
2014-01-09 15:36:04 Mounting with 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.u2dPrxc7SDXd
CnY/outputfs'
mount.nfs: trying 10.45.209.30 prog 100003 vers 3 prot TCP port 2049
mount.nfs: trying 10.45.209.30 prog 100005 vers 3 prot UDP port 49280
mount.nfs: timeout set for Thu Jan  9 15:38:04 2014
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying text-based options 'addr=10.45.209.30,vers=3,proto=tcp,mountvers=3,mountproto=udp,mountport=49280'
10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 on /tmp/rear.u2dPrxc7SDXdCnY/outputfs type nfs (rw,noatime)
2014-01-09 15:36:04 Including output/default/10_mount_output_path.sh
2014-01-09 15:36:04 Mounting with 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.u2dPrxc7SDXd
CnY/outputfs'
mount.nfs: trying 10.45.209.30 prog 100003 vers 3 prot TCP port 2049
mount.nfs: trying 10.45.209.30 prog 100005 vers 3 prot UDP port 49280
mount.nfs: mount(2): Device or resource busy
mount.nfs: /tmp/rear.u2dPrxc7SDXdCnY/outputfs is busy or already mounted
mount.nfs: timeout set for Thu Jan  9 15:38:04 2014
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying text-based options 'addr=10.45.209.30,vers=3,proto=tcp,mountvers=3,mountproto=udp,mountport=49280'
2014-01-09 15:36:04 ERROR: Mount command 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.u2dPr
xc7SDXdCnY/outputfs' failed.
=== Stack trace ===
Trace 0: /usr/sbin/rear:249 main
Trace 1: /usr/share/rear/lib/mkbackup-workflow.sh:30 WORKFLOW_mkbackup
Trace 2: /usr/share/rear/lib/framework-functions.sh:79 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:40 Source
Trace 4: /usr/share/rear/output/default/10_mount_output_path.sh:15 source
Trace 5: /usr/share/rear/lib/global-functions.sh:150 mount_url
Trace 6: /usr/share/rear/lib/_input-output-functions.sh:131 StopIfError
Message: Mount command 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.u2dPrxc7SDXdCnY/outputf
s' failed.
===================
2014-01-09 15:36:04 Running exit tasks.
rmdir: removing directory, `/tmp/rear.u2dPrxc7SDXdCnY/outputfs'
rmdir: failed to remove `/tmp/rear.u2dPrxc7SDXdCnY/outputfs': Device or resource busy
umount.nfs: trying 10.45.209.30 prog 100005 vers 3 prot UDP port 49280
umount.nfs: prog 100005, trying vers=3, prot=17
10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 umounted
rmdir: removing directory, `/tmp/rear.u2dPrxc7SDXdCnY/outputfs'
2014-01-09 15:36:04 Finished in 27 seconds
2014-01-09 15:36:04 Removing build area /tmp/rear.u2dPrxc7SDXdCnY
rmdir: removing directory, `/tmp/rear.u2dPrxc7SDXdCnY'
2014-01-09 15:36:04 End of program reached

gdha commented at 2014-01-09 15:55:

I noticed the following rmdir: failed to remove /tmp/rear.u2dPrxc7SDXdCnY/outputfs: Device or resource busy
Does your root user have the necessary privileges on 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 ?
Small advise, I would not mix HP-UX ignite archives with Linux rear archives.

avinash4any1 commented at 2014-01-09 16:20:

we have installed rear with similar configuration on other suse Host. They perfectly work there. We see this NFS mount error on few of them. I have given complete permission to NFS mount from Unix host.

avinash4any1 commented at 2014-01-09 16:21:

What i see from the logs is, the NFS was mounted at first. Rear is trying to remount it and then getting failed

mkdir: created directory `/tmp/rear.q4rI5vNiGYTeNiy/outputfs'
2014-01-09 17:18:12 Mounting with 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.q4rI5vNiGYTeNiy/outputfs'
mount.nfs: trying 10.45.209.30 prog 100003 vers 3 prot TCP port 2049
mount.nfs: trying 10.45.209.30 prog 100005 vers 3 prot UDP port 49280
mount.nfs: timeout set for Thu Jan 9 17:20:12 2014
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying text-based options 'addr=10.45.209.30,vers=3,proto=tcp,mountvers=3,mountproto=udp,mountport=49280'
10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 on /tmp/rear.q4rI5vNiGYTeNiy/outputfs type nfs (rw,noatime)
2014-01-09 17:18:12 Including output/default/10_mount_output_path.sh
2014-01-09 17:18:12 Mounting with 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.q4rI5vNiGYTeNiy/outputfs'
mount.nfs: trying 10.45.209.30 prog 100003 vers 3 prot TCP port 2049
mount.nfs: trying 10.45.209.30 prog 100005 vers 3 prot UDP port 49280
mount.nfs: mount(2): Device or resource busy
mount.nfs: /tmp/rear.q4rI5vNiGYTeNiy/outputfs is busy or already mounted
mount.nfs: timeout set for Thu Jan 9 17:20:12 2014
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying text-based options 'addr=10.45.209.30,vers=3,proto=tcp,mountvers=3,mountproto=udp,mountport=49280'
2014-01-09 17:18:12 ERROR: Mount command 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.q4rI5vNiGYTeNiy/outputfs' failed.

gdha commented at 2014-01-10 07:28:

perhaps run rear -dD mkrecsue to see more messages (incl. all code output). If the output is too big make a gist entry and paste the link. There must be an umount command failing somehow.

avinash4any1 commented at 2014-01-10 08:08:

globiz101:~ # rear -dD mkrescue
Relax-and-Recover 1.15 / Git
Using log file: /var/log/rear/rear-globiz101.log
The preparation phaseOK
Creating disk layout
Creating root filesystem layout
TIP: To login as root via ssh you need to set up /root/.ssh/authorized_keys or SSH_ROOT_PASSWORD in your configuration file
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
ERROR: Mount command 'mount -v -t nfs -o rw,noatime 10.45.209.30:/var/opt/ignite/recovery/archives/globiz101 /tmp/rear.hqdasUy6dcGDjZx/outputfs failed.
Aborting due to an error, check /var/log/rear/rear-globiz101.log for details
You should also rm -Rf /tmp/rear.hqdasUy6dcGDjZx
Terminated

The rear logs is too big to paste here or in GIST. Please help me with a FTP location or your Email, wherein i can send the logs.

Thank you

avinash4any1 commented at 2014-01-10 08:13:

I have sent you Gmail id. Please review it.

gdha commented at 2014-01-10 10:02:

In your debug logging found the following lines explaining why the mount operation was executed twice:

2014-01-10 08:37:07.319612769 Including output/NETFS/default/10_mount_NETFS_path.sh
2014-01-10 08:37:07.366684479 Including output/default/10_mount_output_path.sh

However, the current rear version should not have a NETFS sub-directory under output/ anymore:

[gdha@fedora20 ~]$ find /usr/share/rear -name 10_mount_output_path.sh
/usr/share/rear/output/default/10_mount_output_path.sh
[gdha@fedora20 ~]$ sudo rear -s mkbackup | grep mount
Source layout/save/default/34_generate_mountpoint_device.sh
Source output/default/10_mount_output_path.sh
Source output/ISO/Linux-i386/20_mount_efibootimg.sh
Source output/ISO/Linux-i386/70_umount_efibootimg.sh
Source output/default/98_umount_output_dir.sh
[gdha@fedora20 ~]$ cd /usr/share/rear/output/
[gdha@fedora20 output]$ ls
default  DP  ISO  NSR  OBDR  PXE  RAMDISK  RSYNC  TSM  USB

Did you ever upgraded rear? If yes, then the NETFS sub-dir is probably a left-over from an older release.
To fix it, rpm -e rear and re-install it I would say.

avinash4any1 commented at 2014-01-10 12:15:

Thanks a lot. Removing rear worked here after deleting the directory /usr/share/rear.

I installed a fresh version, rear backup is successfull.

dheeraj86mathur commented at 2016-08-30 06:00:

Hi, I am trying to configure BACKUP_URL=file:///localDirectorypath/../ for some firewalled servers. After running /usr/sbin/rear -v mkbackup sucessfully with this configuration, I copied the rear folder that got generated with ISO and backup to a jumphost with nfs share. Now, at time of recovery, when I have booted the server is ISO image, how should I point the backup_url to new location of backup and ISO, assuming server's localDirectory path is already destroyed.

unix1adm commented at 2017-01-05 12:47:

GM I am new to gethub and i am having a similar issue.
The issue I am facing is that I am trying to mount an NFS share from an AIX system to use this as my backup storage location.

ReaR works fine if I go to a Red Hat Linux system.

The issue appears to be with NFS 4 vs NFS 3 for AIX.

Is there a command line option or a file I need to change to tell ReaR to use NFS3 not 4.

I did find this posting in Red Hat site
mount.nfs: Remote I/O error
https://access.redhat.com/solutions/61602

When I try the mount bu hand it mounts fine.
mount -o vers=3 server_ip_addr:dir local_mount_point

The errors in the ReaR log show it to be trying to use NFS4

Any help on how to make this work with NFS3 would be appreciated.

Thank you in advance for your time.

Sorry I just realized this is an old posting.
I am still learning ho to use this site. My apologies for opening an old case.

jsmeix commented at 2017-01-05 13:20:

FYI: The last comment is
https://github.com/rear/rear/issues/1154


[Export of Github issue for rear/rear.]