#1576 Issue `closed`: External NSR00 workflow for EMC Networker only works with ReaR 1.17¶

Labels: support / question, fixed / solved / done, external tool

rnarnians opened issue at 2017-11-16 03:29:¶

Relax-and-Recover (ReaR) Issue Template¶

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

rear version (/usr/sbin/rear -V): Relax-and-Recover rear-2.2-5.el6.x86_64
OS version
OS_VENDOR=RedHatEnterpriseServer
OS_VERSION=6
rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):

/etc/rear/local.conf -- Conf file content

NSR00_ROOT_DIR is relocatable - default location is /opt/networker 
NSR00_ROOT_DIR=/nsr COPY_AS_IS_NSR00=( $NSR00_ROOT_DIR /opt/networker /opt/nsr /usr/lib/nsr /usr/lib64/gconv ) 
COPY_AS_IS_EXCLUDE_NSR00=( "$NSR00_ROOT_DIR/logs/*" "$NSR00_ROOT_DIR/debug/*" "$NSR00_ROOT_DIR/index/*" "$NSR00_ROOT_DIR/lockbox/*" "$NSR00_ROOT_DIR/mm/*" "$NSR00_ROOT_DIR/repository/*" "$NSR00_ROOT_DIR/scripts/*" "$NSR00_ROOT_DIR/utils/*" ) PROGS_NSR00=( nsrexec nsrexecd mminfo save savefs savepnpc nsrfsra nsrinfo nsrretrieve nsrwatch nsrports uasm ) 
# NSRSERVER is normally found automatically, but for the rare cases it is not found pls define it in local.conf 
NSRSERVER= 
#####################
OUTPUT=ISO 
BACKUP=NSR00

#Static IP (no DHCP!) 
USE_DHCLIENT= USE_STATIC_NETWORKING="y"

#Root PW (optional) 
SSH_ROOT_PASSWORD='$1$jLBgM/pI$FGs7v6Uo2JtU7vWK6PThU.'

#NTP 
TIMESYNC=NTP

BIOS with HP ILO
Brief description of the issue:
Hello All ,

I am new to this REAR concept , and I am trying to recover with Networker Backup . The configuration seems fine , but when it tries to recover it fails with /mnt/local or some directories seems missing . Could you please help in identifying the issue and fix the bug .

Logs from the Recovery

Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 22 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
+++ tune2fs -m 5 -c 34 -i 180d /dev/mapper/VolGroup00-LogVol00
tune2fs 1.41.12 (17-May-2010)
Setting maximal mount count to 34
Setting interval between checks to 15552000 seconds
Setting reserved blocks percentage to 5% (3502336 blocks)
+++ LogPrint 'Mounting filesystem /'
+++ Log 'Mounting filesystem /'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2017-11-15 15:14:58.953423870 '
+++ test 1 -gt 0
+++ echo '2017-11-15 15:14:58.953423870 Mounting filesystem /'
2017-11-15 15:14:58.953423870 Mounting filesystem /
+++ Print 'Mounting filesystem /'
+++ test 1
+++ echo -e 'Mounting filesystem /'
+++ mkdir -p /mnt/local/
+++ mount -o rw /dev/mapper/VolGroup00-LogVol00 /mnt/local/
+++ component_created fs:/ fs
+++ local device=fs:/
+++ local type=fs
+++ local touchfile=fs-fs:-
+++ touch /tmp/rear.06mKjincA28lkZn/tmp/touch/fs-fs:-
+++ create_component fs:/boot fs
+++ local device=fs:/boot
+++ local type=fs
+++ local touchfile=fs-fs:-boot
+++ '[' -e /tmp/rear.06mKjincA28lkZn/tmp/touch/fs-fs:-boot ']'
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ LogPrint 'Creating filesystem of type ext4 with mount point /boot on /dev/sda1.'
+++ Log 'Creating filesystem of type ext4 with mount point /boot on /dev/sda1.'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2017-11-15 15:14:59.265543157 '
+++ test 1 -gt 0
+++ echo '2017-11-15 15:14:59.265543157 Creating filesystem of type ext4 with mount point /boot on /dev/sda1.'
2017-11-15 15:14:59.265543157 Creating filesystem of type ext4 with mount point /boot on /dev/sda1.
+++ Print 'Creating filesystem of type ext4 with mount point /boot on /dev/sda1.'
+++ test 1
+++ echo -e 'Creating filesystem of type ext4 with mount point /boot on /dev/sda1.'
+++ wipefs --all --force /dev/sda1
wipefs: unrecognized option '--force'
Usage: wipefs [options] <device>

Options:
 -a, --all           wipe all magic strings (BE CAREFUL!)
 -h, --help          this help
 -n, --no-act        everything to be done except for the write() call
 -o, --offset <num>  offset to erase, in bytes
 -p, --parsable      print out in parsable instead of printable format

For more information see wipefs(8).
+++ wipefs --all /dev/sda1
+++ mkfs -t ext4 -b 4096 -i 16355 -U 6257d579-42e4-467e-9d23-e7c66699632e -F /dev/sda1
mke2fs 1.41.12 (17-May-2010)
/dev/sda1 alignment is offset by 229888 bytes.
This may result in very poor performance, (re)-partitioning suggested.
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=64 blocks, Stripe width=64 blocks
66528 inodes, 265080 blocks
13254 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=272629760
9 block groups
32768 blocks per group, 32768 fragments per group
7392 inodes per group
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376

Writing inode tables: 0/91/92/93/94/95/96/97/98/9done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
+++ tune2fs -m 5 -c 27 -i 180d /dev/sda1
tune2fs 1.41.12 (17-May-2010)
Setting maximal mount count to 27
Setting interval between checks to 15552000 seconds
Setting reserved blocks percentage to 5% (13254 blocks)
+++ LogPrint 'Mounting filesystem /boot'
+++ Log 'Mounting filesystem /boot'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2017-11-15 15:14:59.535895118 '
+++ test 1 -gt 0
+++ echo '2017-11-15 15:14:59.535895118 Mounting filesystem /boot'
2017-11-15 15:14:59.535895118 Mounting filesystem /boot
+++ Print 'Mounting filesystem /boot'
+++ test 1
+++ echo -e 'Mounting filesystem /boot'
+++ mkdir -p /mnt/local/boot
+++ mount -o rw /dev/sda1 /mnt/local/boot
+++ component_created fs:/boot fs
+++ local device=fs:/boot
+++ local type=fs
+++ local touchfile=fs-fs:-boot
+++ touch /tmp/rear.06mKjincA28lkZn/tmp/touch/fs-fs:-boot
+++ create_component swap:/dev/mapper/VolGroup00-LogVol01 swap
+++ local device=swap:/dev/mapper/VolGroup00-LogVol01
+++ local type=swap
+++ local touchfile=swap-swap:-dev-mapper-VolGroup00-LogVol01
+++ '[' -e /tmp/rear.06mKjincA28lkZn/tmp/touch/swap-swap:-dev-mapper-VolGroup00-LogVol01 ']'
+++ return 0
+++ LogPrint 'Creating swap on /dev/mapper/VolGroup00-LogVol01'
+++ Log 'Creating swap on /dev/mapper/VolGroup00-LogVol01'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2017-11-15 15:14:59.739807112 '
+++ test 1 -gt 0
+++ echo '2017-11-15 15:14:59.739807112 Creating swap on /dev/mapper/VolGroup00-LogVol01'
2017-11-15 15:14:59.739807112 Creating swap on /dev/mapper/VolGroup00-LogVol01
+++ Print 'Creating swap on /dev/mapper/VolGroup00-LogVol01'
+++ test 1
+++ echo -e 'Creating swap on /dev/mapper/VolGroup00-LogVol01'
+++ mkswap -U 50324905-ca30-42a5-bbad-1836e722b56b /dev/mapper/VolGroup00-LogVol01
mkswap: /dev/mapper/VolGroup00-LogVol01: warning: don't erase bootbits sectors
        on whole disk. Use -f to force.
Setting up swapspace version 1, size = 11673596 KiB
no label, UUID=50324905-ca30-42a5-bbad-1836e722b56b
+++ component_created swap:/dev/mapper/VolGroup00-LogVol01 swap
+++ local device=swap:/dev/mapper/VolGroup00-LogVol01
+++ local type=swap
+++ local touchfile=swap-swap:-dev-mapper-VolGroup00-LogVol01
+++ touch /tmp/rear.06mKjincA28lkZn/tmp/touch/swap-swap:-dev-mapper-VolGroup00-LogVol01
+++ set +x
2017-11-15 15:14:59.745570898 Disk layout created.
2017-11-15 15:14:59.748714068 Including layout/recreate/default/250_verify_mount.sh
2017-11-15 15:14:59.753433516 Finished running 'layout/recreate' stage in 22 seconds
2017-11-15 15:14:59.755368178 ======================
2017-11-15 15:14:59.757032164 Running 'restore' stage
2017-11-15 15:14:59.758677478 ======================
2017-11-15 15:14:59.765296303 Including restore/NSR00/default/20_prompt_user_to_start_restore.sh
2017-11-15 15:14:59.767367336 Please start the restore process on your backup host.
Make sure that you restore the data into '/mnt/local' instead of '/' because the
hard disks of the recovered system are mounted there.

2017-11-15 15:14:59.770928481 Please restore your backup in the provided shell and, when finished, type exit
in the shell to continue recovery.
2017-11-15 15:16:08.174385872 Including restore/NSR00/default/40_restore_with_nsr.sh
2017-11-15 15:16:08.176458766 Starting nsrwatch on console 8
/usr/share/rear/restore/NSR00/default/40_restore_with_nsr.sh: line 4: nsrwatch: command not found
2017-11-15 15:16:08.181655862 Restore filesystem  with recover
2017-11-15 15:16:08.191927590 Including restore/Fedora/050_copy_dev_files.sh
2017-11-15 15:16:08.225392897 Including restore/default/050_remount_async.sh
2017-11-15 15:16:08.227612369 Including restore/NSR00/default/400_restore_with_nsr.sh
2017-11-15 15:16:08.229696277 Starting nsrwatch on console 8
/usr/share/rear/restore/NSR00/default/400_restore_with_nsr.sh: line 4: nsrwatch: command not found
2017-11-15 15:16:08.234989175 Restore filesystem  with recover
2017-11-15 15:16:08.245536036 Including restore/default/500_selinux_autorelabel.sh
2017-11-15 15:16:08.249192523 Created /.autorelabel file : after reboot SELinux will relabel all files
2017-11-15 15:16:08.251595380 Including restore/default/900_create_missing_directories.sh
2017-11-15 15:16:08.253912332 Restore the Mountpoints (with permissions) from /var/lib/rear/recovery/mountpoint_permissions
chmod: cannot access `tmp': No such file or directory
2017-11-15 15:16:08.281256690 Including restore/default/990_move_away_restored_files.sh
2017-11-15 15:16:08.284146897 Including restore/default/995_remount_sync.sh
2017-11-15 15:16:08.286269440 Finished running 'restore' stage in 69 seconds
2017-11-15 15:16:08.288212388 ======================
2017-11-15 15:16:08.290031150 Running 'finalize' stage
2017-11-15 15:16:08.291728791 ======================
2017-11-15 15:16:08.298499989 Including finalize/default/010_prepare_checks.sh
2017-11-15 15:16:08.302977414 Including finalize/default/100_populate_dev.sh
2017-11-15 15:16:08.346461027 Including finalize/GNU/Linux/250_migrate_disk_devices_layout.sh
2017-11-15 15:16:08.348597626 Including finalize/GNU/Linux/250_migrate_lun_wwid.sh
2017-11-15 15:16:08.350676265 Including finalize/GNU/Linux/260_rename_diskbyid.sh
2017-11-15 15:16:08.379247068 Including finalize/GNU/Linux/280_migrate_uuid_tags.sh
2017-11-15 15:16:08.381449981 Including finalize/GNU/Linux/300_create_mac_mapping.sh
2017-11-15 15:16:08.383627871 Including finalize/GNU/Linux/310_migrate_udev_rules.sh
diff: /mnt/local//etc/udev/rules.d/70-persistent-net.rules: No such file or directory
2017-11-15 15:16:08.390770653 Updating udev configuration (70-persistent-net.rules)
`/etc/udev/rules.d/70-persistent-net.rules' -> `/mnt/local//etc/udev/rules.d/70-persistent-net.rules'
cp: cannot create regular file `/mnt/local//etc/udev/rules.d/70-persistent-net.rules': No such file or directory
2017-11-15 15:16:08.395010840 ERROR: Could not copy '/etc/udev/rules.d/70-persistent-net.rules' -> '/mnt/local//etc/udev/rules.d/70-persistent-net.rules'
==== Stack trace ====
Trace 0: /usr/bin/rear:538 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:21 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:95 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:49 Source
Trace 4: /usr/share/rear/finalize/GNU/Linux/310_migrate_udev_rules.sh:36 source
Trace 5: /usr/share/rear/lib/_input-output-functions.sh:242 StopIfError
Message: Could not copy '/etc/udev/rules.d/70-persistent-net.rules' -> '/mnt/local//etc/udev/rules.d/70-persistent-net.rules'
== End stack trace ==
2017-11-15 15:16:08.402438214 Running exit tasks.
2017-11-15 15:16:08.405619071 Finished in 92 seconds
2017-11-15 15:16:08.407743453 Removing build area /tmp/rear.06mKjincA28lkZn
removed directory: `/tmp/rear.06mKjincA28lkZn'
2017-11-15 15:16:08.415896674 End of program reached

error
logs

hpannenb commented at 2017-11-16 09:14:¶

@rnarnians : First of all the NSR00 workflow is no official workflow of the current ReaR versions. Second: Does the recovery image create all required filesystems in /mnt/local and did You recover the system's files via the EMC networker client into /mnt/local? Third: Please try this NSR00 workflow with the older release 1.17 of ReaR for which I wrote this workflow for. Unfortunately I had not yet the time to publish it to the community - though still on my to-do list.

jsmeix commented at 2017-11-16 10:51:¶

It fails in
finalize/GNU/Linux/310_migrate_udev_rules.sh
with

cp: cannot create regular file `/mnt/local//etc/udev/rules.d/70-persistent-net.rules': No such file or directory

which indicates that the /etc/udev/rules.d/ directory is
missing in the restored system (i.e. in /mnt/local/).
If this is the actual reason ( @rnarnians you need to check that)
there is fortunately (actually intentionally) the script
restore/default/900_create_missing_directories.sh
which is there exactly for such reasons.
You can configure what directories should be
recreated by this script via the
DIRECTORY_ENTRIES_TO_RECOVER array
see usr/share/rear/conf/default.conf how to
set DIRECTORY_ENTRIES_TO_RECOVER.

rnarnians commented at 2017-11-28 05:24:¶

Hello All ,

Thanks for the response !
As suggested by the above , I have now downgraded the REAR to 1.17 and have set the workflow to NSR00 ! It worked successfully !

Thank you so much :)

jsmeix commented at 2017-11-28 10:36:¶

@hpannenb
many thanks for your help here!
Now I see that I had no idea at all what the issue actually is about.
Somehow I had overlooked the 'NSR00_' variables in
https://github.com/rear/rear/issues/1576#issue-274382976
that do not exist in our official ReaR code.
We look forward to a pull request from you that implements
your NSR00 workflow in the current ReaR.

hpannenb commented at 2017-11-28 11:36:¶

@jsmeix I found the time and it was merged with pull request #1584.

jsmeix commented at 2017-11-28 12:53:¶

@hpannenb
hopefully I understand it now correctly
that your "NSR00 workflow" for ReaR 1.17
is now implemented as the NSR_CLIENT_MODE.

Many thanks for it!

[Export of Github issue for rear/rear.]

#1576 Issue closed: External NSR00 workflow for EMC Networker only works with ReaR 1.17¶