#870 Issue `closed`: Recover problems with rpc.statd¶

Labels: enhancement, support / question, fixed / solved / done

GCChelp opened issue at 2016-06-09 09:47:¶

rear version (/usr/sbin/rear -V): Relax-and-Recover 1.18 / Git
OS version (cat /etc/rear/os.conf or lsb_release -a):
OS_VENDOR=SUSE_LINUX
OS_VERSION=13.1

ARCH='Linux-i386'
OS='GNU/Linux'
OS_VERSION='13.1'
OS_VENDOR='SUSE_LINUX'
OS_VENDOR_VERSION='SUSE_LINUX/13.1'
OS_VENDOR_ARCH='SUSE_LINUX/i386'
rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP=NETFS
BACKUP_URL=nfs://nfsserver/backups/rear
EXCLUDE_MOUNTPOINTS=( /home /scratch )
AUTOEXCLUDE_PATH=( /media /mnt )
AUTOEXCLUDE_AUTOFS=..
AUTOEXCLUDE_DISKS=y
SSH_ROOT_PASSWORD=XXX
Brief description of the issue
Experimenting with REAR on openSUSE 13.1, 64-bit.
Restore fails with
ERROR: Could not start rpc.statd !

In fact, it even can't be started manually:
rpc.statd -F
rpc.statd: failed to run /usr/sbin/sm-notify

BTW: Adding
BACKUP_OPTIONS="nfsvers=3,nolock"
to site.conf does change nothing. (And further: prevents the creation of the live rescue system on the USB device, because the backup options are used there as well.)

Work-around, if any

ln -s /bin/true /bin/rpc.statd

jsmeix commented at 2016-06-09 10:31:¶

I assign it to me because it is about SUSE
regardless that I am not at all a NFS expert.

Regarding "failed to run /usr/sbin/sm-notify":

@GCChelp
please check if /usr/sbin/sm-notify exists in the
rear recovery system.

You can add missing things for rpc.statd to the rear
recovery system via things like

REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" prog1 prog2 )
COPY_AS_IS=( "${COPY_AS_IS[@]}" /path1/file1 /path2/file2 )

in /etc/rear/local conf - cf.
usr/share/rear/conf/examples/SLE12-SP1-btrfs-example.conf

This way it should be possible to make rpc.statd
working in the rear recovery system.

Regarding "BACKUP_OPTIONS ... prevents the creation of the live rescue system on the USB device":

I submitted a speparated issue
https://github.com/rear/rear/issues/872

GCChelp commented at 2016-06-09 12:05:¶

@jsmeix
Thanks for the quick response!

Yes, sm-notify is missing in the recovery system. I manually fumbled it into the initrd with cpio, but your advice using the configuration setting is much more straight forward.

I will try the REQUIRED_PROGS variable and let you know.

jsmeix commented at 2016-06-09 14:07:¶

@GCChelp
see https://github.com/rear/rear/issues/872#issuecomment-224904226

I like that you test if it also works for you when you
set a non-empty dummy OUTPUT_OPTIONS
in your case something like

OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP=NETFS
BACKUP_URL=nfs://nfsserver/backups/rear
BACKUP_OPTIONS="nfsvers=3,nolock"
OUTPUT_OPTIONS="nodiratime"

To see only how things are mounted run

rear -d -D mkbackup

and afterwards grep for 'mount_url' in the rear log file.

GCChelp commented at 2016-06-10 08:05:¶

@jsmeix
Manually adding sm-notify with the REQUIRED_PROGS variable worked. At least it worked so far that sm-notify is now contained in the rescue system file system.

Nevertheless, recover still fails with

ERROR: Could not start rpc.statd !

Debug output of a manual start of rpc.statd :

rpc.statd -F -d
rpc.statd: Version 1.2.8 starting
rpc.statd: Flags: No-Daemon Log-STDERR TI-RPC
sm-notify: Version 1.2.8 starting
sm-notify: Already notifying clients; Exiting!
rpc.statd: Failed to open directory sm: No such file or directory
rpc.statd: Local NSM state number: 3
rpc.statd: Failed to open /proc/sys/fs/nfs/nsm_local_state: No such file or directory
rpc.statd: Failed to unregister program 100024, version 1
rpc.statd: Running as root. chown /var/lib/nfs to choose different user
rpc.statd: Failed to register (statd, 1, udp)
rpc.statd: Failed to register (statd, 1, tcp)
rpc.statd: Failed to register (statd, 1, udp6)
rpc.statd: Failed to register (statd, 1, tcp6)
rpc.statd: failed to create RPC listeners, exiting

Adding
BACKUP_OPTIONS="nfsvers=3,nolock"
to site.conf changes nothing.

Regarding your dummy OUTPUT_OPTIONS suggestion, I will try this too and post the results.

GCChelp commented at 2016-06-10 09:22:¶

@jsmeix
I did the OUTPUT_OPTIONS experiment and it looks good to me:

grep mount_url /var/log/rear/rear-testupd.log¶

++ mount_url nfs://rhea/backups/rear /tmp/rear.IOZzd2lGEK6Wai9/outputfs nfsvers=3,nolock
++ umount_url nfs://rhea/backups/rear /tmp/rear.IOZzd2lGEK6Wai9/outputfs
++ mount_url usb:///dev/disk/by-label/REAR-000 /tmp/rear.IOZzd2lGEK6Wai9/outputfs nodiratime
++ umount_url usb:///dev/disk/by-label/REAR-000 /tmp/rear.IOZzd2lGEK6Wai9/outputfs
++ mount_url nfs://rhea/backups/rear /tmp/rear.IOZzd2lGEK6Wai9/outputfs nfsvers=3,nolock
++ umount_url nfs://rhea/backups/rear /tmp/rear.IOZzd2lGEK6Wai9/outputfs

Hope this helps!

jsmeix commented at 2016-06-10 09:25:¶

It could become a lengthy step by step process
with some trial an error until you have all you need
in the recovery system so that rpc.statd can run there.

The "No such file or directory" messages indicate that
you need to add more programs and/or files to
REQUIRED_PROGS and/or COPY_AS_IS
to get all needed pieces for rpc.statd into the
recovery system.

Additionally and/or alternatively you might have to adapt
the rpc.statd setup (i.e. its configuration files and/or the
command line options how it is started) to get it running
within the limited possibilities of the recovery system.

Bottom line:
Currently the default rear recovery system
does not well support running rpc.statd therein
(sometimes it works sometimes not)
cf. https://github.com/rear/rear/issues/532

FYI:
In general regarding advanced NFS features see
https://github.com/rear/rear/issues/754

jsmeix commented at 2016-06-10 09:27:¶

@GCChelp
does https://github.com/rear/rear/issues/870#issuecomment-225134937 mean that with

BACKUP_OPTIONS="nfsvers=3,nolock"
OUTPUT_OPTIONS="nodiratime"

both "rear mkbackup" and then "rear recover" work for you?

GCChelp commented at 2016-06-10 09:37:¶

@jsmeix

It could become a lengthy step by step process with some trial an error until you have all you need in the recovery system so that rpc.statd can run there.

I thought, that openSUSE already was supported by REAR...

does #870 (comment) mean that with

BACKUP_OPTIONS="nfsvers=3,nolock"
OUTPUT_OPTIONS="nodiratime"

both "rear mkbackup" and then "rear recover" work for you?

Unchanged: mkbackup works and recover fails with

ERROR: Could not start rpc.statd !

jsmeix commented at 2016-06-10 10:15:¶

On my SUSE test systems I never had an NFS issue when I use

BACKUP_OPTIONS="nfsvers=3,nolock"

The only NFS issue that I had was https://github.com/rear/rear/issues/532

In general regarding "Support" see
http://relax-and-recover.org/support/

In general regarding "Disaster Recovery" see
https://en.opensuse.org/SDB:Disaster_Recovery
therein see in particular the section
"Disaster recovery with Relax-and-Recover (rear)"

GCChelp commented at 2016-06-10 11:31:¶

@jsmeix
I just noticed that on my workstations with openSUSE 13.1 rpc.statd is running with the --no-notify option:

ps -elf | grep rpc.statd | grep -v grep¶

5 S statd 1733 1 0 80 0 - 9020 core_s 11:45 ? 00:00:00 /usr/sbin/rpc.statd --no-notify

Unfortunately, this option is not derived from a configuration file, but is hardcoded in the startup script...

Could this be useful in ReaR rescue system as well? How could this be done?

GCChelp commented at 2016-06-10 11:36:¶

@jsmeix

In general regarding "Support" see
http://relax-and-recover.org/support/

This was not the kind of 'support' I meant. In fact, this page led me to this GitHub repo and I signed up just for Rear.
I was rather referring to e.g. http://relax-and-recover.org/download/ , where an official stable release is offered for openSUSE 13.1, amongst others...

In general regarding "Disaster Recovery" see
https://en.opensuse.org/SDB:Disaster_Recovery
therein see in particular the section
"Disaster recovery with Relax-and-Recover (rear)"

Wow interesting and comprehensive article. Thanks for the link! Nevertheless, I will need some more time to go through this extensive stuff.

GCChelp commented at 2016-06-10 13:47:¶

@jsmeix

It could become a lengthy step by step process
with some trial an error until you have all you need
in the recovery system so that rpc.statd can run there.

I already made some of those steps...
By explicitly adding some more entries I came a bit further:
COPY_AS_IS=( "${COPY_AS_IS[@]}" /var/lib/nfs/sm /var/lib/nfs/sm.bak /var/lib/nfs/state )

Now, rpc.statd still complains

rpc.statd: Failed to open /proc/sys/fs/nfs/nsm_local_state: No such file or directory

The man page of sm-notify states that /var/lib/nfs/state is the NSM state number and /proc/sys/fs/nfs/nsm_local_state is the kernel's copy of the NSM state number.
But there even isn't a /proc/sys/fs/nfs/ directory in the rescue system. As this is a virtual file system, it does not work to copy it over with COPY_AS_IS ...

Any ideas how we could get this into the rescue kernel?

GCChelp commented at 2016-06-10 14:10:¶

Any ideas how we could get this into the rescue kernel?

Seems to come with the lockd kernel module. It even is available in the rescue system and I can load it with modprobe . After loading it, /proc/sys/fs/nfs/nsm_local_state is present!

How can I configure ReaR such that the lockd kernel module gets loaded automatically?

gdha commented at 2016-06-10 14:27:¶

I think we need a proper NFS prep script to cover the dependencies (with NFSv4 there are more daemons and alike we need in the rescue image).
@GCChelp LOAD_MODULES=( ${LOAD_MODULES[@]} lockd ) would do the trick

GCChelp commented at 2016-06-13 09:57:¶

@gdha
Unfortunately not.
Neither
LOAD_MODULES=( ${LOAD_MODULES[@]} lockd )
nor
LOAD_MODULES=( "${LOAD_MODULES[@]}" lockd )
worked. The module doesn't get loaded automatically.
Whereas I can manually do this with modprobe lockd without any problem.

Is there anything I can check to find out what's (going) wrong?

jsmeix commented at 2016-06-14 07:30:¶

@gdha
nowhere in the rear sources I find "LOAD_MODULES".
I think this is a typo in https://github.com/rear/rear/issues/870#issuecomment-225196861

@GCChelp
in general see usr/share/rear/conf/default.conf
what variables you can set in /etc/rear/local.conf
in particular

# autoload these modules in the given order
MODULES_LOAD=()

so that

MODULES_LOAD=( "${MODULES_LOAD[@]}" lockd )

is the right syntax.

jsmeix commented at 2016-06-14 07:39:¶

@GCChelp
regarding how to run "/usr/sbin/rpc.statd --no-notify" in your
https://github.com/rear/rear/issues/870#issuecomment-225159151

In usr/share/rear/conf/default.conf there is

################ ---- custom scripts
#
# NOTE: The scripts can be defined as an array
# to better handly spaces in parameters.
# The scripts are called like this:
# eval "${PRE_RECOVERY_SCRIPT[@]}"
# Call this after Rela-and-Recover did everything
# in the recover workflow.
# Use $TARGET_FS_ROOT (by default '/mnt/local')
# to refer to the recovered system.
POST_RECOVERY_SCRIPT=
# call this before Relax-and-Recover starts
# to do anything in the recover workflow.
# You have the rescue system but nothing else
PRE_RECOVERY_SCRIPT=

I did not test it myself - perhaps you can use it
to run "/usr/sbin/rpc.statd --no-notify" ?

GCChelp commented at 2016-06-14 09:46:¶

@jsmeix
Thanks for the clarification!
Unfortunately, the MODULES_LOAD variant doesn't work either. :-(

The lockd module is present and can be loaded manually. But it does not get loaded automatically.

I created the rescue media with debugging output, but didn't find a problem in the log. Nor did I find anything related in dmesg output after booting the rescue media.

Any hints where I can find enlightenment or what specifically I should search for?

jsmeix commented at 2016-06-14 10:24:¶

@GCChelp
I like to get a summary of your current state:

Please post your complete /etc/rear/local.conf file.

Additionally post what exact comands you have to run
manually in the recovery system so that afterwards
"rear recover" works successfully for you.

Finally I like to know what kind of system your NFS server is.
Only a blind guess: Perhaps there is something special
with your NFS server that leads to those isssues?

Then I can try to reproduce it (but do not expect too much
from me here because I don't know much about NFS).

GCChelp commented at 2016-06-14 11:07:¶

@jsmeix

Please post your complete /etc/rear/local.conf file.

# grep -v ^# /etc/rear/site.conf

OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP=NETFS
BACKUP_URL=nfs://nfsserver/backups/rear
BACKUP_OPTIONS="nfsvers=3,nolock"
OUTPUT_OPTIONS="nodiratime"
EXCLUDE_MOUNTPOINTS=( /home /a /b /c )
AUTOEXCLUDE_PATH=( /media /mnt )
AUTOEXCLUDE_AUTOFS=..
AUTOEXCLUDE_DISKS=y
SSH_ROOT_PASSWORD=XXX
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" sm-notify )
COPY_AS_IS=( "${COPY_AS_IS[@]}" /var/lib/nfs/sm /var/lib/nfs/sm.bak /var/lib/nfs/state /proc/sys/fs/nfs/nsm_local_state )
MODULES_LOAD=( "${MODULES_LOAD[@]}" lockd )

Additionally post what exact comands you have to run
manually in the recovery system so that afterwards
"rear recover" works successfully for you.

I never claimed that "rear recover" was working... It still fails with
ERROR: Could not start rpc.statd !

:-(

Finally I like to know what kind of system your NFS server is.
Only a blind guess: Perhaps there is something special
with your NFS server that leads to those isssues?

It's a Hitachi HNAS 4080 high performance storage system. We mount the NFS file systems via plain NFS3 protocol and it's working like a charm together with dozens of openSUSE workstations and SLES servers (plus machines with SGI/Irix).

This is what one of the NFS mounts look like in the running system:

# mount | grep :/home

nfsserver:/home on /home type nfs (rw,nosuid,nodev,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=155.250.43.213,mountvers=3,mountport=4048,mountproto=tcp,local_lock=none,addr=155.250.43.213)

We use TCP for NFS. I tried to mount a filesystem with "proto=udp" instead: works fine as well.

jsmeix commented at 2016-06-14 13:13:¶

I did not yet try to reproduce it with your setup.

For me the current GitHub rear master
works on an openSUSE 13.1 system
(x86_64 virtual KVM/QEMU machine)

I have this /etc/rear/local.conf

OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://10.160.4.244/nfs
NETFS_KEEP_OLD_BACKUP_COPY=yes
SSH_ROOT_PASSWORD="rear"
USE_DHCLIENT="yes"

cf.
usr/share/rear/conf/examples/SLE11-ext3-example.conf

In the rear recovery system during "rear recover"
the NFS share is mounted as
("mount" output line shown wrapped here):

10.160.4.244:/nfs on /tmp/rear.SWrvJXtLXSSvr6n/outputfs
type nfs (ro,relatime,vers=3,rsize=1048576,wsize=1048576,
namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,
sec=sys,mountaddr=10.160.4.244,mountvers=3,
mountport=20048,mountproto=udp,local_lock=all,
addr=10.160.4.244)

In my "rear -d -D recover" log file the NFS mount comand is

+++ mount -v -t nfs -o nfsvers=3,nolock 10.160.4.244:/nfs /tmp/rear.SWrvJXtLXSSvr6n/outputfs
mount.nfs: trying 10.160.4.244 prog 100003 vers 3 prot TCP port 2049
mount.nfs: trying 10.160.4.244 prog 100005 vers 3 prot UDP port 20048
mount.nfs: timeout set for Tue Jun 14 12:44:07 2016
mount.nfs: trying text-based options 'nfsvers=3,nolock,addr=10.160.4.244'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: prog 100005, trying vers=3, prot=17
++ StopIfError ...

Interestingly for me rpcbind and rpc.statd are running
in the rear recovery system:

RESCUE e137:~ # ps auxw | grep rpc
root       632  0.0  0.0  34796   960 ?        Ss   12:42   0:00 rpcbind
root       637  0.0  0.1  17232  1308 ?        Ss   12:42   0:00 rpc.statd
root       669  0.0  0.0      0     0 ?        S<   12:42   0:00 [rpciod]
RESCUE e137:~ # journalctl | grep rpc
Jun 14 12:42:07 e137 rpc.statd[637]: Version 1.2.8 starting
Jun 14 12:42:07 e137 rpc.statd[637]: Failed to open directory sm: No such file or directory
Jun 14 12:42:07 e137 rpc.statd[637]: Failed to read /var/lib/nfs/state: No such file or directory
Jun 14 12:42:07 e137 rpc.statd[637]: Initializing NSM state
Jun 14 12:42:07 e137 rpc.statd[637]: Running as root.  chown /var/lib/nfs to choose different user

Regarding 'rpc' my "rear -d -D recover" log file contains
(excerpts):

++ PROGS=(${PROGS[@]:-} rpc.statd rpcbind ... rpcinfo ...
...
++ MODULES=(${MODULES[@]:-} ... sunrpc ...
...
++ COPY_AS_IS=(${COPY_AS_IS[@]:-} ... /etc/rpc ...
...
++ CLONE_USERS=("${CLONE_USERS[@]:-}" ... rpc ...
...
++ has_binary rpcbind
++ type rpcbind
++ rpcinfo -p localhost
++ rpcbind
++ StopIfError 'Could not start port mapper [rpcbind] !'
++ rpcinfo -p localhost
++ has_binary rpc.statd
++ type rpc.statd
++ rpcinfo -p localhost
++ rpc.statd
++ StopIfError 'Could not start rpc.statd !'

My NFS server is an openSUSE Leap 42.1 system
(which is also the virtualization host for
that openSUSE 13.1 virtual machine)
with this /etc/exports

/nfs    *(rw,no_root_squash,sync,no_subtree_check)

For my general testing setup
see "First steps with Relax-and-Recover" in
https://en.opensuse.org/SDB:Disaster_Recovery

I think this particular isue here is something special
otherwise there should be other user reports like this.

I feel there is very little what I can really do here
because I think a NFS expert and/or a RPC expert
is needed to further analyze it.

What I can do is basically only blind guesswork.

jsmeix commented at 2016-06-14 13:20:¶

Regarding 'sm-notify' in the initial comment:
I do not have it in my recovery system
and 'sm-notify' is not mentioned in my
"rear -d -D recover" log file.
I.e. for me it works without 'sm-notify'.

GCChelp commented at 2016-06-22 13:39:¶

@jsmeix

Interestingly for me rpcbind and rpc.statd are running
in the rear recovery system:

RESCUE e137:~ # ps auxw | grep rpc
root 632 0.0 0.0 34796 960 ? Ss 12:42 0:00 rpcbind
root 637 0.0 0.1 17232 1308 ? Ss 12:42 0:00 rpc.statd
root 669 0.0 0.0 0 0 ? S< 12:42 0:00 [rpciod]

RESCUE e137:~ # journalctl | grep rpc
Jun 14 12:42:07 e137 rpc.statd[637]: Version 1.2.8 starting
Jun 14 12:42:07 e137 rpc.statd[637]: Failed to open directory sm: No such file or directory
Jun 14 12:42:07 e137 rpc.statd[637]: Failed to read /var/lib/nfs/state: No such file or directory
Jun 14 12:42:07 e137 rpc.statd[637]: Initializing NSM state
Jun 14 12:42:07 e137 rpc.statd[637]: Running as root. chown /var/lib/nfs to choose different use

In my recovery system, rpcstat.d is not running:

ps auxw | grep rpc¶

root 619 0.0 0.0 0 0 ? S< Jun21 0:00 [rpciod]
root 896 0.0 0.0 48636 1056 ? Ss Jun21 0:00 rpcbind
root 1172 0.0 0.0 9008 684 pts/1 S+ 13:20 0:00 grep rpc

journalctl | grep rpc¶

Jun 21 10:46:08 testupd rpc.statd[901]: Version 1.2.8 starting
Jun 21 10:46:08 testupd rpc.statd[901]: Running as root. chown /var/lib/nfs to choose different user
Jun 21 10:46:08 testupd rpc.statd[901]: failed to create RPC listeners, exiting
Jun 21 10:46:08 testupd systemd[1]: Received SIGCHLD from PID 901 (rpc.statd).
Jun 21 10:46:08 testupd systemd[1]: Child 901 (rpc.statd) died (code=exited, status=1/FAILURE)
Jun 21 10:46:08 testupd rear[908]: ERROR: Could not start rpc.statd !

But nevertheless, I can manually mount the NFS file system, if I do it without locking:
# mount -o nolock nfsserver:/backups /mnt

And it looks very much like your mount output from above, only IP addresses and [rw]size differ!

jsmeix commented at 2016-06-22 14:15:¶

@GCChelp

in your initial comment https://github.com/rear/rear/issues/870#issue-159368065 you wrote

    Work-around, if any
  ln -s /bin/true /bin/rpc.statd

but in your later comment https://github.com/rear/rear/issues/870#issuecomment-225849729 you wrote

>   Additionally post what exact comands you have to run
>   manually in the recovery system so that afterwards
>   "rear recover" works successfully for you.
I never claimed that "rear recover" was working... It still fails with
ERROR: Could not start rpc.statd !

and now you wrote

I can manually mount the NFS file system

I am confused.

When you can manually mount the NFS file system
withut a running rpc.stad then it matches your
initial comment that "ln -s /bin/true /bin/rpc.statd"
should help to get "rear recover" working.

Or in other words:

When you can manually mount the NFS file system
withut a running rpc.stad then it should help
to simply disable the currently hardcoded
error exit if it "Could not start rpc.statd !".

The "Could not start rpc.statd !" is only in
usr/share/rear/verify/NETFS/default/05_start_required_daemons.sh

        rpc.statd
        StopIfError "Could not start rpc.statd !"

and if you change that to something like

        rpc.statd
        LogPrintIfError "Could not start rpc.statd !"

it would only log the error message to the rear log file
and additionally print the error message on the terminal
but
it would no longer abort at that error.

Does then "rear recover" work for you?

jsmeix commented at 2016-06-23 14:34:¶

With https://github.com/rear/rear/pull/891
it is no longer fatal when RPC status rpc.statd is unavailable
so that it does no longer error out with
"Could not start rpc.statd !"

Instead it now only shows a message like
"RPC status rpc.statd unavailable
(you may have to mount NFS without locking 'nolock')."

@GCChelp
please test it it works better for you with
the current rear GitHub master code.

GCChelp commented at 2016-06-24 13:07:¶

@jsmeix
Trying to catch up some loose ends here...

For completeness sake: I repeated my experiments with an NFS server based on openSUSE 13.1.
Same results. Non of the problems seem to be related to the special NFS server.

I am confused.

Sorry to confuse you. ;-)
I meant it did not yet work for me the automated way. And not to success.
My manual workarounds only led me one step further.

But even with them I don't get a working system with "rear recover": It runs and finishes. But when I try to boot the recovered system, it fails because GRUB 2 is not configured correctly. That doesn't really surprise me, because we are still using GRUB 1 (legacy)... But I suspect, this would be substance for another (new) issue?

The "Could not start rpc.statd !" is only in
usr/share/rear/verify/NETFS/default/05_start_required_daemons.sh
   rpc.statd
  StopIfError "Could not start rpc.statd !"
and if you change that to something like
   rpc.statd
  LogPrintIfError "Could not start rpc.statd !"
it would only log the error message to the rear log file
and additionally print the error message on the terminal
but it would no longer abort at that error.

Does then "rear recover" work for you?

Yes, with this workaround "rear recover" runs successful!
(Finally leading to an unbootable machine, see above...)

And with this workaround I even no longer need any definition of REQUIRED_PROGS, COPY_AS_IS and MODULES_LOAD in my configuration!

When you can manually mount the NFS file system
withut a running rpc.stad [...]

When I can manually mount the NFS file system with "-o nolock", why can't rear do this when I define
BACKUP_OPTIONS="nfsvers=3,nolock"
in site.conf?

jsmeix commented at 2016-06-27 09:27:¶

@GCChelp

Regarding GRUB 2 versus GRUB legacy:
Please submit a new separated issue.
Again post your /etc/rear/local.conf file in the new issue.
Do "rear -d -D recover" and afterwards when you are
still within in the recovery system get the rear log file
out of the recovery system, see
"Debugging issues with Relax-and-Recover (rear)" at
https://en.opensuse.org/SDB:Disaster_Recovery
Then inspect the rear log file for messages which could be
of interest reagrding the incorrect bootloader setup and
post those log messages in your new separated issue.

Regarding

    rpc.statd
    LogPrintIfError "Could not start rpc.statd !"
with this workaround "rear recover" runs successful

This means that https://github.com/rear/rear/pull/891
avoids this particular issue here.
Accordingly I close this issue here as "fixed"
(actually it is "avoided").

Regarding

When I can manually mount the NFS file system with "-o nolock",
why can't rear do this when I define
BACKUP_OPTIONS="nfsvers=3,nolock"

"rear recover" did not reach the point where it would mount
the NFS file system because it aborted when "rpc.statd"
failed to run.
Since https://github.com/rear/rear/pull/891 it is no longer
fatal when "rpc.statd" fails to run so that "rear recover"
can proceed to mount the NFS file system and then
it does it with the BACKUP_OPTIONS (at least for me)
see my https://github.com/rear/rear/issues/870#issuecomment-225876504

BACKUP_OPTIONS="nfsvers=3,nolock"
...
In my "rear -d -D recover" log file the NFS mount comand is
+++ mount -v -t nfs -o nfsvers=3,nolock 10.160.4.244:/nfs /tmp/rear.SWrvJXtLXSSvr6n/outputfs

GCChelp commented at 2016-06-27 12:18:¶

@jsmeix

Regarding GRUB 2 versus GRUB legacy:
Please submit a new separated issue.

Will do this.

BACKUP_OPTIONS="nfsvers=3,nolock"
...
In my "rear -d -D recover" log file the NFS mount comand is

+++ mount -v -t nfs -o nfsvers=3,nolock 10.160.4.244:/nfs /tmp/rear.SWrvJXtLXSSvr6n/outputfs

Confirmed

[Export of Github issue for rear/rear.]

#870 Issue closed: Recover problems with rpc.statd¶

GCChelp opened issue at 2016-06-09 09:47:¶

jsmeix commented at 2016-06-09 10:31:¶

GCChelp commented at 2016-06-09 12:05:¶

jsmeix commented at 2016-06-09 14:07:¶

GCChelp commented at 2016-06-10 08:05:¶

GCChelp commented at 2016-06-10 09:22:¶

grep mount_url /var/log/rear/rear-testupd.log¶

jsmeix commented at 2016-06-10 09:25:¶

jsmeix commented at 2016-06-10 09:27:¶

GCChelp commented at 2016-06-10 09:37:¶

jsmeix commented at 2016-06-10 10:15:¶

GCChelp commented at 2016-06-10 11:31:¶

ps -elf | grep rpc.statd | grep -v grep¶

GCChelp commented at 2016-06-10 11:36:¶

GCChelp commented at 2016-06-10 13:47:¶

GCChelp commented at 2016-06-10 14:10:¶

gdha commented at 2016-06-10 14:27:¶

GCChelp commented at 2016-06-13 09:57:¶

jsmeix commented at 2016-06-14 07:30:¶

jsmeix commented at 2016-06-14 07:39:¶

GCChelp commented at 2016-06-14 09:46:¶

jsmeix commented at 2016-06-14 10:24:¶

GCChelp commented at 2016-06-14 11:07:¶

jsmeix commented at 2016-06-14 13:13:¶

jsmeix commented at 2016-06-14 13:20:¶

GCChelp commented at 2016-06-22 13:39:¶

ps auxw | grep rpc¶

journalctl | grep rpc¶

jsmeix commented at 2016-06-22 14:15:¶

jsmeix commented at 2016-06-23 14:34:¶

GCChelp commented at 2016-06-24 13:07:¶

jsmeix commented at 2016-06-27 09:27:¶

GCChelp commented at 2016-06-27 12:18:¶

#870 Issue `closed`: Recover problems with rpc.statd¶