#506 Issue closed: cannot recover, ERROR mount command AND INIT: cannot execute “/sbin/mingetty”

Labels: support / question

nelteren opened issue at 2014-11-11 20:47:

Daily I made a backup with rear of all our linux host. Works fine and I can recover all hosts, except one.

I made a lab configuration for testing the recovery, VMWare with a nfs server for storing the backups

I get the error:
I mount the ISO and start automatic recovery
ERROR mount command ‘ mount –v –t nfs –o rw, notime xxx.xxx.xxx.xxx/rear/dest failed
Check rear-s03.log for details

But on other host with rear recovery I can mount this path with no problems

Furthermore I cannot go to option 2. Go to Relax and Reccover shell. There I get errors,
INIT: cannot execute “/sbin/mingetty”
INIT: ID “t3” respawning too fast: disabled for 5 minutes

Local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL=nfs://xxx.xxx.xxx.xxx/rear/s03.net-products.nl/odd

(I have a destination for odd and even days)

os.conf
OS_VENDOR='Debian'
OS_VERSION='4.3.5-4'

Last part of rear.log
2014-11-09 08:13:55 Copying kernel
/boot/vmlinuz-2.6.32-5-amd64' ->/tmp/rear.0FzkvZFgDgkUdbB/tmp/isofs/isolinux/kernel'
/tmp/rear.0FzkvZFgDgkUdbB/tmp/initrd.cgz' ->/tmp/rear.0FzkvZFgDgkUdbB/tmp/isofs/isolinux/initrd.cgz'
2014-11-09 08:13:55 Including output/ISO/Linux-i386/81_prepare_multiple_iso.sh
2014-11-09 08:13:55 Including output/ISO/Linux-i386/82_create_iso_image.sh
2014-11-09 08:13:55 Starting '/usr/bin/genisoimage'
2014-11-09 08:13:55 Making ISO image
genisoimage 1.1.11 (Linux)
Scanning .
Scanning ./isolinux
29 360
30 1722 isolinux
Cache hit for 'isolinux/.'
Cache hit for 'isolinux/..'
29 29 ./.
29 29 (null)
30 30 ./isolinux
34 34 ./isolinux/TRANS.TBL
47 56 ./isolinux/chain.c32
57 191 ./isolinux/hdt.c32
192 13722 ./isolinux/initrd.cgz
35 46 ./isolinux/isolinux.bin
13723 13724 ./isolinux/isolinux.cfg
13725 14908 ./isolinux/kernel
14909 14938 ./isolinux/menu.c32
14939 14939 ./isolinux/message
14940 14940 ./isolinux/poweroff.com
14941 14941 ./isolinux/rear.help
14942 14942 ./isolinux/reboot.c32
Writing: Initial Padblock Start Block 0
Done with: Initial Padblock Block(s) 16
Writing: Primary Volume Descriptor Start Block 16
Done with: Primary Volume Descriptor Block(s) 1
Writing: Eltorito Volume Descriptor Start Block 17
Size of boot image is 4 sectors -> No emulation
Done with: Eltorito Volume Descriptor Block(s) 1
Writing: Joliet Volume Descriptor Start Block 18
Done with: Joliet Volume Descriptor Block(s) 1
Writing: End Volume Descriptor Start Block 19
Done with: End Volume Descriptor Block(s) 1
Writing: Version block Start Block 20
Done with: Version block Block(s) 1
Writing: Path table Start Block 21
Done with: Path table Block(s) 4
Writing: Joliet path table Start Block 25
Done with: Joliet path table Block(s) 4
Writing: Directory tree Start Block 29
Done with: Directory tree Block(s) 2
Writing: Joliet directory tree Start Block 31
Done with: Joliet directory tree Block(s) 2
Writing: Directory tree cleanup Start Block 33
Done with: Directory tree cleanup Block(s) 0
Writing: Extension record Start Block 33
Done with: Extension record Block(s) 1
Writing: The File(s) Start Block 34
Total extents scheduled to be written = 15093
33.18% done, estimate finish Sun Nov 9 08:13:55 2014
66.26% done, estimate finish Sun Nov 9 08:13:55 2014
Total translation table size: 2048
Total rockridge attributes bytes: 1357
Total directory bytes: 2408
Path table size(bytes): 26
Done with: The File(s) Block(s) 14909
Writing: Ending Padblock Start Block 14943
Done with: Ending Padblock Block(s) 150
Max brk space used 0
15093 extents written (29 MB)
2014-11-09 08:13:55 Wrote ISO image: /var/lib/rear/output/rear-s03.iso (30M)
2014-11-09 08:13:55 Including output/ISO/Linux-i386/85_check_for_errors.sh
2014-11-09 08:13:55 Including output/default/94_grub_rescue.sh
2014-11-09 08:13:55 Including output/default/95_copy_result_files.sh
2014-11-09 08:13:55 Copying resulting files to nfs location
2014-11-09 08:13:55 Copying files '/var/lib/rear/output/rear-s03.iso' to nfs location
/var/lib/rear/output/rear-s03.iso' ->/tmp/rear.0FzkvZFgDgkUdbB/outputfs/s03/rear-s03.iso'
/usr/share/rear/conf/templates/RESULT_usage_ISO.txt' ->/tmp/rear.0FzkvZFgDgkUdbB/outputfs/s03/README'

Tail of backup.log
block 271975574: /var/local/mysqlbackups/
block 271975575: /var/local/mysqlbackups/dumps.tar
block 272439576: /var/list/
block 272439577: /var/list/.qmail
block 272439579: /lib64
block 272439580: /var/log/rear/rear-s03.log
Total bytes written: 139489249280 (130GiB, 12MiB/s)
191785851+1 records in
191785851+1 records out
98194356154 bytes (98 GB) copied, 11508.5 s, 8.5 MB/s

I guess it is the ISO.
What can I do?

schlomo commented at 2014-11-11 21:22:

Which ReaR version and which distro?

If you have several systems and only one cannot be restore - how is it different from the others?

Was mingetty installed in the source system? Extract the initrd from the ISO and check if mingetty is actuallly in it. chroot into the extracted initrd to check if some libraries are missing.

Can you SSH into the recovery system (requires ssh keys in root account)? If yes, check recovery log there for details about failed mount.

Does the recovery system use same IP as source system or different? That might be a reason for NFS mount failure.

In any case you have several issues (mount failure and mingetty missing).

nelteren commented at 2014-11-13 14:40:

Hi
I use Relax-and-Recover 1.16 / Git
All our linux host are debian, the problem host:
Linux s03.net-products.nl 2.6.32-5-amd64 #1 SMP Sat May 5 01:12:59 UTC 2012 x86_64 GNU/Linux
Mingetty not installed….

Working host
Relax-and-Recover 1.15 / Git
Linux ontwikkel 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3 x86_64 GNU/Linux
Mingetty installed

I installed mingetty on the problem host en make an new iso, and now I can go to the shell.

But mounting is still a problem. Seems to be same problem as mentioned on posted issue : Mounting of NFS share fails at the time of recovering Ubuntu_12.04_x64 VM to physical system #463
So I changed /usr/share/rear/lib/global-functions.sh and added nolock and recreated the iso

< local defaultoptions="rw,noatime,nolock"

local defaultoptions="rw,noatime"

Now I am restoring. Thanks!!!

And Yes, I recover to the same ip, we made a lab, with same environment222

Met vriendelijke groet,

Nico van Elteren

Van: Schlomo Schapiro [mailto:notifications@github.com]
Verzonden: dinsdag 11 november 2014 22:23
Aan: rear/rear
CC: Nico v. Elteren
Onderwerp: Re: [rear] cannot recover, ERROR mount command AND INIT: cannot execute “/sbin/mingetty” (#506)

Which ReaR version and which distro?

If you have several systems and only one cannot be restore - how is it different from the others?

Was mingetty installed in the source system? Extract the initrd from the ISO and check if mingetty is actuallly in it. chroot into the extracted initrd to check if some libraries are missing.

Can you SSH into the recovery system (requires ssh keys in root account)? If yes, check recovery log there for details about failed mount.

Does the recovery system use same IP as source system or different? That might be a reason for NFS mount failure.

In any case you have several issues (mount failure and mingetty missing).


Reply to this email directly or view it on GitHubhttps://github.com/rear/rear/issues/506#issuecomment-62623252.

schlomo commented at 2014-11-13 16:40:

use BACKUP_OPTIONS="nolock" in the local.conf file.

NFS lock requires rpc.statd IIRC. We have code that should deal with it automatically:

~/src/rear$ grep -r rpc.statd .
./usr/share/rear/conf/GNU/Linux.conf:rpc.statd
./usr/share/rear/verify/NETFS/default/08_start_required_daemons.sh:     if has_binary rpc.statd; then
./usr/share/rear/verify/NETFS/default/08_start_required_daemons.sh:             rpc.statd
./usr/share/rear/verify/NETFS/default/08_start_required_daemons.sh:             StopIfError "Could not start rpc.statd !"

You might want to investigate why this did not work in your case.

nelteren commented at 2014-11-15 20:21:

mentioned problems are solved.
thanks!


[Export of Github issue for rear/rear.]