#1479 Issue `closed`: An error occurred during layout recreation - SLES11SP4 on Power¶

Labels: support / question, fixed / solved / done

sebaobi opened issue at 2017-09-08 11:42:¶

rear version (/usr/sbin/rear -V): Relax-and-Recover 2.2-git.0.2658745.unknown.changed / 2017-08-03
OS version (cat /etc/rear/os.conf or lsb_release -a): SUSE Linux Enterprise Server 11 SP4 (ppc64)
rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):

BACKUP=NSR
OUTPUT=ISO
OUTPUT_URL=nfs://192.168.12.10/data/col1/rear
NSRSERVER=nsr_server
POOLNAME=ddos
RETENTION_TIME=Month
USE_CFG2HTML=y
AUTOEXCLUDE_MULTIPATH=n
BOOT_OVER_SAN=y
REQUIRED_PROGS=(
"${REQUIRED_PROGS[@]}"
parted
sfdisk
)

PROGS=(
"${PROGS[@]}"
partprobe
fdisk
cfdisk
bootlist
pseries_platform
nvram
ofpathname
bc
agetty
)

#Exclude firmware only when running in Virtualized mode (meaning not PowerNV mode).
if [[ $(awk '/platform/ {print $NF}' < /proc/cpuinfo) != PowerNV ]] ; then
    COPY_AS_IS_EXCLUDE=( ${COPY_AS_IS_EXCLUDE[@]} /lib*/firmware )
fi

KERNEL_CMDLINE="LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us console=hvc0"

Are you using legacy BIOS or UEFI boot? BIOS
Brief description of the issue:
We are trying to restore a SLES 11 SP4 PPC running on Power. Source and destination server have the same disk layout but we encountered an issue during layout recreation.
An extract of the log:

+++ LogPrint 'Creating LVM PV /dev/mapper/relax01_rvg_part2'
+++ Log 'Creating LVM PV /dev/mapper/relax01_rvg_part2'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2017-09-08 13:10:01.097854488 '
+++ test 1 -gt 0
+++ echo '2017-09-08 13:10:01.097854488 Creating LVM PV /dev/mapper/relax01_rvg_part2'
2017-09-08 13:10:01.097854488 Creating LVM PV /dev/mapper/relax01_rvg_part2
+++ Print 'Creating LVM PV /dev/mapper/relax01_rvg_part2'
+++ test 1
+++ echo -e 'Creating LVM PV /dev/mapper/relax01_rvg_part2'
+++ lvm vgchange -a n OS
  Volume group "OS" not found
+++ true
+++ lvm pvcreate -ff --yes -v --uuid 7DGexv-5mGb-83xw-STdI-aKDH-zahP-eHZ87u --norestorefile /dev/mapper/relax01_rvg_part2
    connect() failed on local socket: No such file or directory
    Internal cluster locking initialisation failed.
    WARNING: Falling back to local file-based locking.
    Volume Groups with the clustered attribute will be inaccessible.
  Device /dev/mapper/relax01_rvg_part2 not found (or ignored by filtering).
2017-09-08 13:10:03.515589517 An error occurred during layout recreation.
2017-09-08 13:11:38.224627824 User selected: 6) Abort Relax-and-Recover
2017-09-08 13:11:38.226158664 Error detected during restore.
2017-09-08 13:11:38.227441044 Restoring backup of /var/lib/rear/layout/disklayout.conf
2017-09-08 13:11:38.229934269 ERROR: There was an error restoring the system layout. See /var/log/rear/relax01.log for details.
==== Stack trace ====
Trace 0: /bin/rear:538 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:22 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:95 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:49 Source
Trace 4: /usr/share/rear/layout/recreate/default/200_run_script.sh:65 source
Message: There was an error restoring the system layout. See /var/log/rear/relax01.log for details.
== End stack trace ==
2017-09-08 13:11:38.234191271 Running exit tasks.
2017-09-08 13:11:38.236102995 Finished in 152 seconds
2017-09-08 13:11:38.237305155 Removing build area /tmp/rear.MnFW1oKS6GzX9B4
removed directory: `/tmp/rear.MnFW1oKS6GzX9B4'
2017-09-08 13:11:38.242741139 End of program reached

jsmeix commented at 2017-09-08 11:53:¶

@sebaobi
can you attach your complete
var/lib/rear/layout/disklayout.conf
file?

schabrolles commented at 2017-09-08 11:56:¶

@sebaobi Could you please add also var/lib/rear/layout/diskrestore.sh please

sebaobi commented at 2017-09-08 15:36:¶

RESCUE relax01:/ # less /var/lib/rear/layout/disklayout.conf

lvmdev /dev/OS /dev/mapper/relax01_rvg_part2 7DGexv-5mGb-83xw-STdI-aKDH-zahP-eHZ87u 41525248
lvmgrp /dev/OS 4096 5068 20758528
lvmvol /dev/OS root 1536 12582912
lvmvol /dev/OS swap 1024 8388608
lvmvol /dev/OS tmp 1024 8388608
lvmvol /dev/OS var 1024 8388608
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/mapper/OS-root / ext3 uuid=5fabe5fc-22ee-4e51-8311-8917169d316d label= blocksize=4096 reserved_blocks=4% max_mounts=23 check_interval=180d bytes_per_inode=16384 options=rw,acl,user_xattr
fs /dev/mapper/OS-tmp /tmp ext3 uuid=f9f8eed5-f1a7-496e-af2c-cbeb451a0a42 label= blocksize=4096 reserved_blocks=4% max_mounts=27 check_interval=180d bytes_per_inode=16384 options=rw,acl,user_xattr
fs /dev/mapper/OS-var /var ext3 uuid=22af31d8-7b3c-475d-910b-800df685ef68 label= blocksize=4096 reserved_blocks=4% max_mounts=20 check_interval=180d bytes_per_inode=16384 options=rw,acl,user_xattr
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/mapper/OS-swap uuid=0b4fb267-464d-4e33-a671-83892c305437 label=
/dev/sdd
part /dev/mapper/relax01_rvg 212860928 1048576 primary boot,prep /dev/mapper/relax01_rvg_part1
part /dev/mapper/relax01_rvg 21260926976 213909504 primary lvm /dev/mapper/relax01_rvg_part2

RESCUE relax01:~ # less /var/lib/rear/layout/diskrestore.sh

#!/bin/bash

LogPrint "Start system layout restoration."

mkdir -p /mnt/local
if create_component "vgchange" "rear" ; then
    lvm vgchange -a n >/dev/null
    component_created "vgchange" "rear"
fi

set -e
set -x

if create_component "/dev/mapper/mpathb" "multipath" ; then
# Create /dev/mapper/mpathb (multipath)
LogPrint "Creating partitions for disk /dev/mapper/mpathb (msdos)"
my_udevsettle
parted -s /dev/mapper/mpathb mklabel msdos >&2
my_udevsettle
my_udevsettle
parted -s /dev/mapper/mpathb mkpart '"primary"' 2097152B 214958079B >&2
my_udevsettle
my_udevsettle
parted -s /dev/mapper/mpathb set 1 boot on >&2
my_udevsettle
my_udevsettle
parted -s /dev/mapper/mpathb set 1 prep on >&2
my_udevsettle
my_udevsettle
parted -s /dev/mapper/mpathb mkpart '"primary"' 214962176B 21474836479B >&2
my_udevsettle
my_udevsettle
parted -s /dev/mapper/mpathb set 2 lvm on >&2
my_udevsettle
sleep 1
if ! partprobe -s /dev/mapper/mpathb >&2 ; then
    LogPrint 'retrying partprobe /dev/mapper/mpathb after 10 seconds'
    sleep 10
    if ! partprobe -s /dev/mapper/mpathb >&2 ; then
        LogPrint 'retrying partprobe /dev/mapper/mpathb after 1 minute'
        sleep 60
        if ! partprobe -s /dev/mapper/mpathb >&2 ; then
            LogPrint 'partprobe /dev/mapper/mpathb failed, proceeding bona fide'
        fi
    fi
fi
component_created "/dev/mapper/mpathb" "multipath"
else
    LogPrint "Skipping /dev/mapper/mpathb (multipath) as it has already been created."
fi

if create_component "/dev/mapper/relax01_rvg_part1" "part" ; then
# Create /dev/mapper/relax01_rvg_part1 (part)
component_created "/dev/mapper/relax01_rvg_part1" "part"
else
    LogPrint "Skipping /dev/mapper/relax01_rvg_part1 (part) as it has already been created."
fi

if create_component "/dev/mapper/relax01_rvg_part2" "part" ; then
# Create /dev/mapper/relax01_rvg_part2 (part)
component_created "/dev/mapper/relax01_rvg_part2" "part"
else
    LogPrint "Skipping /dev/mapper/relax01_rvg_part2 (part) as it has already been created."
fi

if create_component "pv:/dev/mapper/relax01_rvg_part2" "lvmdev" ; then
# Create pv:/dev/mapper/relax01_rvg_part2 (lvmdev)
LogPrint "Creating LVM PV /dev/mapper/relax01_rvg_part2"
lvm vgchange -a n OS || true
lvm pvcreate -ff --yes -v --uuid "7DGexv-5mGb-83xw-STdI-aKDH-zahP-eHZ87u" --norestorefile /dev/mapper/relax01_rvg_part2 >&2
component_created "pv:/dev/mapper/relax01_rvg_part2" "lvmdev"
else
    LogPrint "Skipping pv:/dev/mapper/relax01_rvg_part2 (lvmdev) as it has already been created."
fi

if create_component "/dev/OS" "lvmgrp" ; then
# Create /dev/OS (lvmgrp)
LogPrint "Creating LVM VG OS"
if [ -e "/dev/OS" ] ; then
    rm -rf "/dev/OS"
fi
lvm vgcreate --physicalextentsize 4096k OS /dev/mapper/relax01_rvg_part2 >&2
lvm vgchange --available y OS >&2
component_created "/dev/OS" "lvmgrp"
else
    LogPrint "Skipping /dev/OS (lvmgrp) as it has already been created."
fi

if create_component "/dev/mapper/OS-root" "lvmvol" ; then
# Create /dev/mapper/OS-root (lvmvol)
LogPrint "Creating LVM volume OS/root"
lvm lvcreate -l 1536 -n root OS <<<y
component_created "/dev/mapper/OS-root" "lvmvol"
else
    LogPrint "Skipping /dev/mapper/OS-root (lvmvol) as it has already been created."
fi

if create_component "/dev/mapper/OS-swap" "lvmvol" ; then
# Create /dev/mapper/OS-swap (lvmvol)
LogPrint "Creating LVM volume OS/swap"
lvm lvcreate -l 1024 -n swap OS <<<y
component_created "/dev/mapper/OS-swap" "lvmvol"
else
    LogPrint "Skipping /dev/mapper/OS-swap (lvmvol) as it has already been created."
fi

if create_component "/dev/mapper/OS-tmp" "lvmvol" ; then
# Create /dev/mapper/OS-tmp (lvmvol)
LogPrint "Creating LVM volume OS/tmp"
lvm lvcreate -l 1024 -n tmp OS <<<y
component_created "/dev/mapper/OS-tmp" "lvmvol"
else
    LogPrint "Skipping /dev/mapper/OS-tmp (lvmvol) as it has already been created."
fi

if create_component "/dev/mapper/OS-var" "lvmvol" ; then
# Create /dev/mapper/OS-var (lvmvol)
LogPrint "Creating LVM volume OS/var"
lvm lvcreate -l 1024 -n var OS <<<y
component_created "/dev/mapper/OS-var" "lvmvol"
else
    LogPrint "Skipping /dev/mapper/OS-var (lvmvol) as it has already been created."
fi

if create_component "fs:/" "fs" ; then
# Create fs:/ (fs)
# Wait until udev had created '/dev/mapper/OS-root' before creating a filesystem there:
my_udevsettle
LogPrint 'Creating filesystem of type 'ext3' with mount point '/' on '/dev/mapper/OS-root'.'
# Using wipefs to cleanup '/dev/mapper/OS-root' before creating filesystem.
wipefs --all --force /dev/mapper/OS-root || wipefs --all /dev/mapper/OS-root || dd if=/dev/zero of=/dev/mapper/OS-root bs=512 count=1 || true
# Try 'mkfs -U' to create the filesystem with initially correct UUID
# but if that fails assume it failed because of missing support for '-U'
# (e.g. in RHEL 5 it fails, see https://github.com/rear/rear/issues/890)
# then fall back to using mkfs without '-U' plus 'tune2fs/tune4fs -U'
if ! mkfs -t ext3 -b 4096 -i 16384 -U 5fabe5fc-22ee-4e51-8311-8917169d316d -F /dev/mapper/OS-root >&2 ; then
    mkfs -t ext3 -b 4096 -i 16384 -F /dev/mapper/OS-root >&2
    tune2fs -U 5fabe5fc-22ee-4e51-8311-8917169d316d /dev/mapper/OS-root >&2
fi
tune2fs  -m 4 -c 23 -i 180d /dev/mapper/OS-root >&2
LogPrint "Mounting filesystem /"
mkdir -p /mnt/local/
mount -o rw,acl,user_xattr /dev/mapper/OS-root /mnt/local/
component_created "fs:/" "fs"
else
    LogPrint "Skipping fs:/ (fs) as it has already been created."
fi

if create_component "fs:/tmp" "fs" ; then
# Create fs:/tmp (fs)
# Wait until udev had created '/dev/mapper/OS-tmp' before creating a filesystem there:
my_udevsettle
LogPrint 'Creating filesystem of type 'ext3' with mount point '/tmp' on '/dev/mapper/OS-tmp'.'
# Using wipefs to cleanup '/dev/mapper/OS-tmp' before creating filesystem.
wipefs --all --force /dev/mapper/OS-tmp || wipefs --all /dev/mapper/OS-tmp || dd if=/dev/zero of=/dev/mapper/OS-tmp bs=512 count=1 || true
# Try 'mkfs -U' to create the filesystem with initially correct UUID
# but if that fails assume it failed because of missing support for '-U'
# (e.g. in RHEL 5 it fails, see https://github.com/rear/rear/issues/890)
# then fall back to using mkfs without '-U' plus 'tune2fs/tune4fs -U'
if ! mkfs -t ext3 -b 4096 -i 16384 -U f9f8eed5-f1a7-496e-af2c-cbeb451a0a42 -F /dev/mapper/OS-tmp >&2 ; then
    mkfs -t ext3 -b 4096 -i 16384 -F /dev/mapper/OS-tmp >&2
    tune2fs -U f9f8eed5-f1a7-496e-af2c-cbeb451a0a42 /dev/mapper/OS-tmp >&2
fi
tune2fs  -m 4 -c 27 -i 180d /dev/mapper/OS-tmp >&2
LogPrint "Mounting filesystem /tmp"
mkdir -p /mnt/local/tmp
mount -o rw,acl,user_xattr /dev/mapper/OS-tmp /mnt/local/tmp
component_created "fs:/tmp" "fs"
else
    LogPrint "Skipping fs:/tmp (fs) as it has already been created."
fi

if create_component "fs:/var" "fs" ; then
# Create fs:/var (fs)
# Wait until udev had created '/dev/mapper/OS-var' before creating a filesystem there:
my_udevsettle
LogPrint 'Creating filesystem of type 'ext3' with mount point '/var' on '/dev/mapper/OS-var'.'
# Using wipefs to cleanup '/dev/mapper/OS-var' before creating filesystem.
wipefs --all --force /dev/mapper/OS-var || wipefs --all /dev/mapper/OS-var || dd if=/dev/zero of=/dev/mapper/OS-var bs=512 count=1 || true
# Try 'mkfs -U' to create the filesystem with initially correct UUID
# but if that fails assume it failed because of missing support for '-U'
# (e.g. in RHEL 5 it fails, see https://github.com/rear/rear/issues/890)
# then fall back to using mkfs without '-U' plus 'tune2fs/tune4fs -U'
if ! mkfs -t ext3 -b 4096 -i 16384 -U 22af31d8-7b3c-475d-910b-800df685ef68 -F /dev/mapper/OS-var >&2 ; then
    mkfs -t ext3 -b 4096 -i 16384 -F /dev/mapper/OS-var >&2
    tune2fs -U 22af31d8-7b3c-475d-910b-800df685ef68 /dev/mapper/OS-var >&2
fi
tune2fs  -m 4 -c 20 -i 180d /dev/mapper/OS-var >&2
LogPrint "Mounting filesystem /var"
mkdir -p /mnt/local/var
mount -o rw,acl,user_xattr /dev/mapper/OS-var /mnt/local/var
component_created "fs:/var" "fs"
else
    LogPrint "Skipping fs:/var (fs) as it has already been created."
fi

if create_component "swap:/dev/mapper/OS-swap" "swap" ; then
# Create swap:/dev/mapper/OS-swap (swap)
LogPrint "Creating swap on /dev/mapper/OS-swap"
mkswap -U 0b4fb267-464d-4e33-a671-83892c305437 /dev/mapper/OS-swap >&2
mkswap -U 0b4fb267-464d-4e33-a671-83892c305437 /dev/mapper/OS-swap >&2
component_created "swap:/dev/mapper/OS-swap" "swap"
else
    LogPrint "Skipping swap:/dev/mapper/OS-swap (swap) as it has already been created."
fi


set +x
set +e

LogPrint "Disk layout created."

schabrolles commented at 2017-09-10 09:29:¶

@sebaobi There is definitely something strange here.

disklayout file doesn't have any disk or multipath type device. (only part)
I assume you use multipath on the source system, so you should have multipath devices define in you disklayout.
diskrestore.sh script starts with creating partions on /dev/mapper/mpathb and then try to use /dev/mapper/relax01_rvg as device name to create LVM.
How many disk/LUN do you have on your system ? (could you send multipath -ll output ?)
when you ran rear recover, did you have a menu with some "Choice" to make ? (press 1 to confirm mappings, press 5 to continue etc ....)
If it is the case, could you please try to recover again and attach a copy of what you have on screen (menu + choice you make) and the full log file generated in /var/log/rear.

sebaobi commented at 2017-09-13 10:13:¶

Hello,
I have one LUN, here you are multipath output:

`RESCUE iterelax01:~ # multipath -ll
mpathb (3600507680c8182613800000000000417) dm-0 IBM,2145
size=20G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 1:0:2:0 sda 8:0  active ready running
| `- 2:0:0:0 sdc 8:32 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 1:0:3:0 sdb 8:16 active ready running
  `- 2:0:1:0 sdd 8:48 active ready running`

This is output from my recover session:

`RESCUE relax01:~ # rear -v recover
Relax-and-Recover 2.2-git.0.2658745.unknown.changed / 2017-08-03
Using log file: /var/log/rear/rear-relax01.log
Running workflow recover within the ReaR rescue/recovery system
Will do driver migration (recreating initramfs/initrd)
We will recover the following file systems from EMC NetWorker: / /tmp /var
Setting up multipathing
Activating multipath
multipath activated
mpathb  (253, 0)
mpathb (3600507680c8182613800000000000417) dm-0 IBM,2145
size=20G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=0 status=active
| |- 1:0:2:0 sda 8:0  active undef running
| `- 2:0:0:0 sdc 8:32 active undef running
`-+- policy='round-robin 0' prio=0 status=enabled
  |- 1:0:3:0 sdb 8:16 active undef running
  `- 2:0:1:0 sdd 8:48 active undef running
Comparing disks.
Device mapper!relax01_rvg does not exist.
Switching to manual disk layout configuration.
Using /dev/mapper/mpathb (same size) for recreating /dev/mapper/relax01_rvg
Current disk mapping table (source -> target):
    /dev/mapper/relax01_rvg /dev/mapper/mpathb
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default 1 timeout 300 seconds)

Continuing 'rear recover' by default
Please confirm that '/var/lib/rear/layout/disklayout.conf' is as you expect.

1) View disk layout (disklayout.conf)  4) Go to Relax-and-Recover shell
2) Edit disk layout (disklayout.conf)  5) Continue recovery
3) View original disk space usage      6) Abort Relax-and-Recover



#? 1
lvmdev /dev/OS /dev/mapper/relax01_rvg_part2 7DGexv-5mGb-83xw-STdI-aKDH-zahP-eHZ87u 41525248
lvmgrp /dev/OS 4096 5068 20758528
lvmvol /dev/OS root 1536 12582912
lvmvol /dev/OS swap 1024 8388608
lvmvol /dev/OS tmp 1024 8388608
lvmvol /dev/OS var 1024 8388608
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/mapper/OS-root / ext3 uuid=5fabe5fc-22ee-4e51-8311-8917169d316d label= blocksize=4096 reserved_blocks=4% max_mounts=23 check_interval=180d bytes_per_inode=16384 options=rw,acl,user_xattr
fs /dev/mapper/OS-tmp /tmp ext3 uuid=f9f8eed5-f1a7-496e-af2c-cbeb451a0a42 label= blocksize=4096 reserved_blocks=4% max_mounts=27 check_interval=180d bytes_per_inode=16384 options=rw,acl,user_xattr
fs /dev/mapper/OS-var /var ext3 uuid=22af31d8-7b3c-475d-910b-800df685ef68 label= blocksize=4096 reserved_blocks=4% max_mounts=20 check_interval=180d bytes_per_inode=16384 options=rw,acl,user_xattr
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/mapper/OS-swap uuid=0b4fb267-464d-4e33-a671-83892c305437 label=
multipath /dev/mapper/mpathb 21474836480 /dev/sda,/dev/sdb,/dev/sdc,/dev/sdd
part /dev/mapper/mpathb 212860928 1048576 primary boot,prep /dev/mapper/relax01_rvg_part1
part /dev/mapper/mpathb 21260926976 213909504 primary lvm /dev/mapper/relax01_rvg_part2



#? 5
Partition primary on /dev/mapper/mpathb: size reduced to fit on disk.
Please confirm that '/var/lib/rear/layout/diskrestore.sh' is as you expect.

1) View restore script (diskrestore.sh)
2) Edit restore script (diskrestore.sh)
3) View original disk space usage
4) Go to Relax-and-Recover shell
5) Continue recovery
6) Abort Relax-and-Recover

1) View restore script (diskrestore.sh)
2) Edit restore script (diskrestore.sh)
3) View original disk space usage
4) Go to Relax-and-Recover shell
5) Continue recovery
6) Abort Relax-and-Recover
#? 5
Start system layout restoration.
Creating partitions for disk /dev/mapper/mpathb (msdos)
Creating LVM PV /dev/mapper/relax01_rvg_part2
An error occurred during layout recreation.`

And logs from recover session:

2017-09-13 11:31:19.495479661 Relax-and-Recover 2.2-git.0.2658745.unknown.changed / 2017-08-03
2017-09-13 11:31:19.496542861 Command line options: /bin/rear -v recover
2017-09-13 11:31:19.497558613 Using log file: /var/log/rear/rear-relax01.log
2017-09-13 11:31:19.498905378 Including /etc/rear/os.conf
2017-09-13 11:31:19.500439675 Including conf/Linux-ppc64.conf
2017-09-13 11:31:19.501945824 Including conf/GNU/Linux.conf
2017-09-13 11:31:19.506324162 Including conf/SUSE_LINUX.conf
2017-09-13 11:31:19.507889027 Including /etc/rear/site.conf
2017-09-13 11:31:19.511782357 Including /etc/rear/local.conf
2017-09-13 11:31:19.513038052 Including /etc/rear/rescue.conf
2017-09-13 11:31:19.515631292 ======================
2017-09-13 11:31:19.516622561 Running 'init' stage
2017-09-13 11:31:19.517527049 ======================
2017-09-13 11:31:19.521405821 Including init/default/010_set_drlm_env.sh
2017-09-13 11:31:19.522589073 Including init/default/030_update_recovery_system.sh
2017-09-13 11:31:19.523734920 Including init/default/050_check_rear_recover_mode.sh
2017-09-13 11:31:19.524896500 Running workflow recover within the ReaR rescue/recovery system
2017-09-13 11:31:19.526126495 Finished running 'init' stage in 0 seconds
2017-09-13 11:31:19.532631154 Using build area '/tmp/rear.6nRwpcczyG6Mh6F'
mkdir: created directory `/tmp/rear.6nRwpcczyG6Mh6F/rootfs'
mkdir: created directory `/tmp/rear.6nRwpcczyG6Mh6F/tmp'
2017-09-13 11:31:19.535209271 Running recover workflow
2017-09-13 11:31:19.536251265 ======================
2017-09-13 11:31:19.537150144 Running 'setup' stage
2017-09-13 11:31:19.538026163 ======================
2017-09-13 11:31:19.541887326 Including setup/default/010_pre_recovery_script.sh
2017-09-13 11:31:19.542939945 Finished running 'setup' stage in 0 seconds
2017-09-13 11:31:19.543946426 ======================
2017-09-13 11:31:19.544881676 Running 'verify' stage
2017-09-13 11:31:19.545815785 ======================
2017-09-13 11:31:19.549688661 Including verify/default/020_cciss_scsi_engage.sh
2017-09-13 11:31:19.551666874 Including verify/default/020_translate_url.sh
2017-09-13 11:31:19.553169429 Including verify/default/030_translate_tape.sh
2017-09-13 11:31:19.554453012 Including verify/default/040_validate_variables.sh
2017-09-13 11:31:19.556796213 Including verify/default/050_create_mappings_dir.sh
2017-09-13 11:31:19.558006790 Including verify/GNU/Linux/050_sane_recovery_check.sh
2017-09-13 11:31:19.559257184 Including verify/GNU/Linux/230_storage_and_network_modules.sh
2017-09-13 11:31:19.560484444 Including storage drivers
2017-09-13 11:31:19.565035456 Including network drivers
2017-09-13 11:31:19.569818084 Including crypto drivers
2017-09-13 11:31:19.573269603 Including virtualization drivers
2017-09-13 11:31:19.576256660 Including additional drivers
2017-09-13 11:31:19.579115033 Including verify/GNU/Linux/260_recovery_storage_drivers.sh
2017-09-13 11:31:19.626720548 Will do driver migration (recreating initramfs/initrd)
2017-09-13 11:31:19.628058705 Including verify/NSR/default/400_verify_nsr.sh
2017-09-13 11:31:19.629205669 NSRSERVER (c301bcksrv) was defined in /etc/rear/local.conf
2017-09-13 11:31:19.630634474 Skipping ping test
2017-09-13 11:31:19.631945247 EMC NetWorker server NSRSERVER=c301bcksrv
2017-09-13 11:31:19.633220485 Including verify/NSR/default/410_verify_nsr_paths.sh
2017-09-13 11:31:19.635456897 We will recover the following file systems from EMC NetWorker: / /tmp /var
2017-09-13 11:31:19.636667587 Finished running 'verify' stage in 0 seconds
2017-09-13 11:31:19.637650770 ======================
2017-09-13 11:31:19.638564973 Running 'layout/prepare' stage
2017-09-13 11:31:19.639523268 ======================
2017-09-13 11:31:19.643709899 Including layout/prepare/default/010_prepare_files.sh
2017-09-13 11:31:19.647157063 Including layout/prepare/GNU/Linux/100_include_partition_code.sh
2017-09-13 11:31:19.654270898 Including layout/prepare/GNU/Linux/110_include_lvm_code.sh
2017-09-13 11:31:19.660011949 Including layout/prepare/GNU/Linux/120_include_raid_code.sh
/usr/share/rear/lib/_input-output-functions.sh: line 199: type: mdadm: not found
2017-09-13 11:31:19.661249357 Including layout/prepare/GNU/Linux/130_include_filesystem_code.sh
2017-09-13 11:31:19.664067010 Including layout/prepare/GNU/Linux/130_include_mount_filesystem_code.sh
2017-09-13 11:31:19.665895961 Including layout/prepare/GNU/Linux/130_include_mount_subvolumes_code.sh
2017-09-13 11:31:19.670475845 Including layout/prepare/GNU/Linux/140_include_swap_code.sh
2017-09-13 11:31:19.675200575 Including layout/prepare/GNU/Linux/150_include_drbd_code.sh
2017-09-13 11:31:19.676610080 Including layout/prepare/GNU/Linux/160_include_luks_code.sh
2017-09-13 11:31:19.678075024 Including layout/prepare/GNU/Linux/170_include_hpraid_code.sh
/usr/share/rear/lib/_input-output-functions.sh: line 199: type: hpacucli: not found
/usr/share/rear/lib/_input-output-functions.sh: line 199: type: hpssacli: not found
2017-09-13 11:31:19.679716178 Including layout/prepare/default/200_recreate_hpraid.sh
2017-09-13 11:31:19.681788953 Including layout/prepare/GNU/Linux/210_load_multipath.sh
2017-09-13 11:31:19.683849604 Setting up multipathing
2017-09-13 11:31:19.685083793 Activating multipath
2017-09-13 11:31:19.697866040 multipath activated
2017-09-13 11:31:19.705361773 Including layout/prepare/default/250_compare_disks.sh
2017-09-13 11:31:19.706524343 Comparing disks.
2017-09-13 11:31:19.711304693 Looking for mapper!relax01_rvg...
2017-09-13 11:31:19.712561017 Device mapper!relax01_rvg does not exist.
2017-09-13 11:31:19.713901280 Switching to manual disk layout configuration.
2017-09-13 11:31:19.715231567 Including layout/prepare/default/270_overrule_migration_mode.sh
2017-09-13 11:31:19.717605641 Including layout/prepare/default/300_map_disks.sh
2017-09-13 11:31:19.767502479 Using /dev/mapper/mpathb (same size) for recreating /dev/mapper/relax01_rvg
2017-09-13 11:31:19.772174292 Current disk mapping table (source -> target):
2017-09-13 11:31:19.774464661     /dev/mapper/relax01_rvg /dev/mapper/mpathb
2017-09-13 11:31:19.776208320 UserInput -p Confirm or edit the disk mapping -D Confirm disk mapping and continue 'rear recover' Confirm disk mapping and continue 'rear recover' Edit disk mapping (/var/lib/rear/layout/disk_mappings) Use Relax-and-Recover shell and return back to here Abort 'rear recover'
2017-09-13 11:31:19.786938203 Confirm or edit the disk mapping
2017-09-13 11:31:19.788240011 1) Confirm disk mapping and continue 'rear recover'
2017-09-13 11:31:19.789446695 2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
2017-09-13 11:31:19.790657059 3) Use Relax-and-Recover shell and return back to here
2017-09-13 11:31:19.791967849 4) Abort 'rear recover'
2017-09-13 11:31:19.793162407 (default 1 timeout 300 seconds)
2017-09-13 11:32:11.988017099 UserInput: 'read' got as user input ''
2017-09-13 11:32:11.989946001 Continuing 'rear recover' by default
2017-09-13 11:32:11.993875067 Including layout/prepare/default/310_remove_exclusions.sh
2017-09-13 11:32:11.997593260 Including layout/prepare/default/320_apply_mappings.sh
/usr/share/rear/layout/prepare/default/320_apply_mappings.sh: line 56: ((: < 12 : syntax error: operand expected (error token is "< 12 ")
/usr/share/rear/layout/prepare/default/320_apply_mappings.sh: line 56: ((: < 12 : syntax error: operand expected (error token is "< 12 ")
/usr/share/rear/layout/prepare/default/320_apply_mappings.sh: line 107: ((: < 12 : syntax error: operand expected (error token is "< 12 ")
2017-09-13 11:32:12.008256875 Including layout/prepare/default/400_autoresize_disks.sh
2017-09-13 11:32:12.013139087 Including layout/prepare/default/500_confirm_layout.sh
2017-09-13 11:32:12.014361435 Please confirm that '/var/lib/rear/layout/disklayout.conf' is as you expect.
+++ echo '2017-09-13 11:34:07.914033284 Creating partitions for disk /dev/mapper/mpathb (msdos)'
2017-09-13 11:34:07.914033284 Creating partitions for disk /dev/mapper/mpathb (msdos)
+++ Print 'Creating partitions for disk /dev/mapper/mpathb (msdos)'
+++ test 1
+++ echo -e 'Creating partitions for disk /dev/mapper/mpathb (msdos)'
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/mapper/mpathb mklabel msdos
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/mapper/mpathb mkpart '"primary"' 2097152B 214958079B
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ parted -s /dev/mapper/mpathb set 1 boot on
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ return 0
+++ local device=/dev/mapper/mpathb
+++ local type=multipath
+++ local touchfile=multipath--dev-mapper-mpathb
+++ touch /tmp/rear.6nRwpcczyG6Mh6F/tmp/touch/multipath--dev-mapper-mpathb
+++ create_component /dev/mapper/relax01_rvg_part1 part
+++ local device=/dev/mapper/relax01_rvg_part1
+++ local type=part
+++ local touchfile=part--dev-mapper-relax01_rvg_part1
+++ '[' -e /tmp/rear.6nRwpcczyG6Mh6F/tmp/touch/part--dev-mapper-relax01_rvg_part1 ']'
+++ return 0
+++ component_created /dev/mapper/relax01_rvg_part1 part
+++ local device=/dev/mapper/relax01_rvg_part1
+++ local type=part
+++ local touchfile=part--dev-mapper-relax01_rvg_part1
+++ touch /tmp/rear.6nRwpcczyG6Mh6F/tmp/touch/part--dev-mapper-relax01_rvg_part1
+++ create_component /dev/mapper/relax01_rvg_part2 part
+++ local device=/dev/mapper/relax01_rvg_part2
+++ local type=part
+++ local touchfile=part--dev-mapper-relax01_rvg_part2
+++ '[' -e /tmp/rear.6nRwpcczyG6Mh6F/tmp/touch/part--dev-mapper-relax01_rvg_part2 ']'
+++ return 0
+++ component_created /dev/mapper/relax01_rvg_part2 part
+++ local device=/dev/mapper/relax01_rvg_part2
+++ local type=part
+++ local touchfile=part--dev-mapper-relax01_rvg_part2
+++ touch /tmp/rear.6nRwpcczyG6Mh6F/tmp/touch/part--dev-mapper-relax01_rvg_part2
+++ create_component pv:/dev/mapper/relax01_rvg_part2 lvmdev
+++ local device=pv:/dev/mapper/relax01_rvg_part2
+++ local type=lvmdev
+++ local touchfile=lvmdev-pv:-dev-mapper-relax01_rvg_part2
+++ '[' -e /tmp/rear.6nRwpcczyG6Mh6F/tmp/touch/lvmdev-pv:-dev-mapper-relax01_rvg_part2 ']'
+++ return 0
+++ LogPrint 'Creating LVM PV /dev/mapper/relax01_rvg_part2'
+++ Log 'Creating LVM PV /dev/mapper/relax01_rvg_part2'
++++ date '+%Y-%m-%d %H:%M:%S.%N '
+++ local 'timestamp=2017-09-13 11:34:38.364223420 '
+++ test 1 -gt 0
+++ echo '2017-09-13 11:34:38.364223420 Creating LVM PV /dev/mapper/ITE_iterelax01_rvg_part2'
2017-09-13 11:34:38.364223420 Creating LVM PV /dev/mapper/ITE_iterelax01_rvg_part2
+++ Print 'Creating LVM PV /dev/mapper/ITE_iterelax01_rvg_part2'
+++ test 1
+++ echo -e 'Creating LVM PV /dev/mapper/ITE_iterelax01_rvg_part2'
+++ lvm vgchange -a n OS
  Volume group "OS" not found
+++ true
+++ lvm pvcreate -ff --yes -v --uuid 7DGexv-5mGb-83xw-STdI-aKDH-zahP-eHZ87u --norestorefile /dev/mapper/ITE_iterelax01_rvg_part2
    connect() failed on local socket: No such file or directory
    Internal cluster locking initialisation failed.
    WARNING: Falling back to local file-based locking.
    Volume Groups with the clustered attribute will be inaccessible.
  Device /dev/mapper/ITE_iterelax01_rvg_part2 not found (or ignored by filtering).
2017-09-13 11:34:40.895634689 An error occurred during layout recreation.
2017-09-13 11:36:49.811943195 User selected: 6) Abort Relax-and-Recover
2017-09-13 11:36:49.813350154 Error detected during restore.
2017-09-13 11:36:49.814571490 Restoring backup of /var/lib/rear/layout/disklayout.conf
2017-09-13 11:36:49.817068652 ERROR: There was an error restoring the system layout. See /var/log/rear/rear-iterelax01.log for details.
==== Stack trace ====
Trace 0: /bin/rear:538 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:22 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:95 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:49 Source
Trace 4: /usr/share/rear/layout/recreate/default/200_run_script.sh:65 source
Message: There was an error restoring the system layout. See /var/log/rear/rear-iterelax01.log for details.
== End stack trace ==
2017-09-13 11:36:49.821225283 Running exit tasks.
2017-09-13 11:36:49.823175789 Finished in 330 seconds
2017-09-13 11:36:49.824358941 Removing build area /tmp/rear.6nRwpcczyG6Mh6F
removed directory: `/tmp/rear.6nRwpcczyG6Mh6F'
2017-09-13 11:36:49.829686577 End of program reached

schabrolles commented at 2017-09-13 10:25:¶

2017-09-13 11:32:11.997593260 Including layout/prepare/default/320_apply_mappings.sh
/usr/share/rear/layout/prepare/default/320_apply_mappings.sh: line 56: ((: < 12 : syntax error: operand expected (error token is "< 12 ")
/usr/share/rear/layout/prepare/default/320_apply_mappings.sh: line 56: ((: < 12 : syntax error: operand expected (error token is "< 12 ")
/usr/share/rear/layout/prepare/default/320_apply_mappings.sh: line 107: ((: < 12 : syntax error: operand expected (error token is "< 12 ")

@sebaobi could you pleas show me the content of cat /etc/rear/os.conf
it is like $OS_VERSION variable is not set on your system.

sebaobi commented at 2017-09-13 10:30:¶

RESCUE relax01:~ # cat /etc/rear/os.conf
OS_VENDOR=SUSE_LINUX
OS_VERSION=
# The following information was added automatically by the mkrescue workflow:
ARCH='Linux-ppc64'
OS='GNU/Linux'
OS_VERSION=''
OS_VENDOR='SUSE_LINUX'
OS_VENDOR_VERSION='SUSE_LINUX/'
OS_VENDOR_ARCH='SUSE_LINUX/ppc64'
# End of what was added automatically by the mkrescue workflow.

schabrolles commented at 2017-09-13 10:34:¶

Change OS_VERSION='' to OS_VERSION='11' and try to recover again. If it works we will have a look why this variable was not set correctly during rear mkrescue.

sebaobi commented at 2017-09-13 10:47:¶

Hehe thnx for this hint :), it seems almost perfect but now i have issue with udev:

RESCUE relax01:~ # rear -v recover
Relax-and-Recover 2.2-git.0.2658745.unknown.changed / 2017-08-03
Using log file: /var/log/rear/rear-relax01.log
Running workflow recover within the ReaR rescue/recovery system
Will do driver migration (recreating initramfs/initrd)
We will recover the following file systems from EMC NetWorker: / /tmp /var
Setting up multipathing
Activating multipath
multipath activated
mpathb  (253, 0)
mpathb (3600507680c8182613800000000000417) dm-0 IBM,2145
size=20G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=0 status=active
| |- 1:0:2:0 sda 8:0  active undef running
| `- 2:0:0:0 sdc 8:32 active undef running
`-+- policy='round-robin 0' prio=0 status=enabled
  |- 1:0:3:0 sdb 8:16 active undef running
  `- 2:0:1:0 sdd 8:48 active undef running
Comparing disks.
Device mapper!relax01_rvg does not exist.
Switching to manual disk layout configuration.
Using /dev/mapper/mpathb (same size) for recreating /dev/mapper/relax01_rvg
Current disk mapping table (source -> target):
    /dev/mapper/relax01_rvg /dev/mapper/mpathb
Confirm or edit the disk mapping
1) Confirm disk mapping and continue 'rear recover'
2) Edit disk mapping (/var/lib/rear/layout/disk_mappings)
3) Use Relax-and-Recover shell and return back to here
4) Abort 'rear recover'
(default 1 timeout 300 seconds)
1
User confirmed disk mapping
Please confirm that '/var/lib/rear/layout/disklayout.conf' is as you expect.

1) View disk layout (disklayout.conf)  4) Go to Relax-and-Recover shell
2) Edit disk layout (disklayout.conf)  5) Continue recovery
3) View original disk space usage      6) Abort Relax-and-Recover
#? 5
Partition primary on /dev/mapper/mpathb: size reduced to fit on disk.
Please confirm that '/var/lib/rear/layout/diskrestore.sh' is as you expect.

1) View restore script (diskrestore.sh)
2) Edit restore script (diskrestore.sh)
3) View original disk space usage
4) Go to Relax-and-Recover shell
5) Continue recovery
6) Abort Relax-and-Recover
#? 5
Start system layout restoration.
Creating partitions for disk /dev/mapper/mpathb (msdos)
Creating LVM PV /dev/mapper/mpathb_part2
Creating LVM VG OS
Creating LVM volume OS/root
  Logical volume "root" created
Creating LVM volume OS/swap
  Logical volume "swap" created
Creating LVM volume OS/tmp
  Logical volume "tmp" created
Creating LVM volume OS/var
  Logical volume "var" created
Creating filesystem of type ext3 with mount point / on /dev/mapper/OS-root.
Mounting filesystem /
Creating filesystem of type ext3 with mount point /tmp on /dev/mapper/OS-tmp.
Mounting filesystem /tmp
Creating filesystem of type ext3 with mount point /var on /dev/mapper/OS-var.
Mounting filesystem /var
Creating swap on /dev/mapper/OS-swap
Disk layout created.
Starting nsrwatch on console 8
Restore filesystem / /tmp /var  with recover
Function 'iconv_open' failed for UTF-8 to ANSI conversion. The function may be unable to determine the current locale. Verify appropriate values in environment variables LC_MESSAGES, LC_ALL and LANG.
Restore the Mountpoints (with permissions) from /var/lib/rear/recovery/mountpoint_permissions
Updating udev configuration (70-persistent-net.rules)
ERROR: Could not copy '/etc/udev/rules.d/70-persistent-net.rules' -> '/mnt/local//etc/udev/rules.d/70-persistent-net.rules'
Aborting due to an error, check /var/log/rear/rear-relax01.log for details
Terminated

Log:

2017-09-13 12:32:09.219198132 Using build area '/tmp/rear.lVf2CxV7YcPa1Wk'
...skipping...
2017-09-13 12:33:03.905347824 Updating udev configuration (70-persistent-net.rules)
`/etc/udev/rules.d/70-persistent-net.rules' -> `/mnt/local//etc/udev/rules.d/70-persistent-net.rules'
cp: cannot create regular file `/mnt/local//etc/udev/rules.d/70-persistent-net.rules': No such file or directory
2017-09-13 12:33:03.907776501 ERROR: Could not copy '/etc/udev/rules.d/70-persistent-net.rules' -> '/mnt/local//etc/udev/rules.d/70-persistent-net.rules'
==== Stack trace ====
Trace 0: /bin/rear:538 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:26 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:95 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:49 Source
Trace 4: /usr/share/rear/finalize/GNU/Linux/310_migrate_udev_rules.sh:36 source
Trace 5: /usr/share/rear/lib/_input-output-functions.sh:242 StopIfError
Message: Could not copy '/etc/udev/rules.d/70-persistent-net.rules' -> '/mnt/local//etc/udev/rules.d/70-persistent-net.rules'
== End stack trace ==
2017-09-13 12:33:03.911742036 Running exit tasks.
2017-09-13 12:33:03.913542675 Finished in 54 seconds
2017-09-13 12:33:03.914707090 Removing build area /tmp/rear.lVf2CxV7YcPa1Wk
removed directory: `/tmp/rear.lVf2CxV7YcPa1Wk'
2017-09-13 12:33:03.919761121 End of program reached

schabrolles commented at 2017-09-13 10:58:¶

I don't know BACKUP=NSR, but are you sure that the restoration works ?
Tell me if you can enter in /mnt/local/etc/udev/rules.d

sebaobi commented at 2017-09-13 11:03:¶

Sure but something goes wrong because i can't see etc folder in /mnt/local/ 👍

RESCUE relax01:/mnt/local # df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/OS-root  6.0G  141M  5.6G   3% /mnt/local
/dev/mapper/OS-tmp   4.0G  137M  3.7G   4% /mnt/local/tmp
/dev/mapper/OS-var   4.0G  137M  3.7G   4% /mnt/local/var


RESCUE relax01:/mnt/local # cd /mnt/local/ ; pwd ; ls -al
/mnt/local
total 44
drwxr-xr-x  9 root root  4096 Sep 13 12:51 .
drwxr-xr-x  4 root root     0 Sep  8 13:01 ..
-rw-r--r--  1 root root     0 Sep 13 12:51 .autorelabel
drwxr-xr-x 15 root root  4096 Sep 13 12:51 dev
drwx------  2 root root 16384 Sep 13 12:51 lost+found
drwxr-xr-x  4 root root  4096 Sep 13 12:51 media
dr-xr-xr-x  2 root root  4096 Sep 13 12:51 proc
drwxr-xr-x  2 root root  4096 Sep 13 12:51 sys
drwxrwxrwt  3 root root  4096 Sep 13 12:51 tmp
drwxr-xr-x  3 root root  4096 Sep 13 12:51 var

schabrolles commented at 2017-09-13 11:04:¶

I think there is some problem in your NSR backup configuration.
@jsmeix any suggestion / comments ?

jsmeix commented at 2017-09-13 11:20:¶

@schabrolles
I am already looking... ;-)

jsmeix commented at 2017-09-13 11:38:¶

@sebaobi
I don't know about NSR but it looks
as if the backup restore was incomplete.

After backup restore you must have at least all the usual files
and directories back that are needed by the basic system.

When e.g. the etc/ directory is missing it is a fatal error
in the backup restore and then subsequent ReaR scripts
that run after the backup restore just fail because
those scripts expect the usual files and directories
below /mnt/local - see also
https://github.com/rear/rear/pull/1457#issuecomment-327492288

Accordingly you should inspect your logs
what exactly happpened during your particular
NSR backup restore.

In your
https://github.com/rear/rear/issues/1479#issuecomment-329130755
there is only

Starting nsrwatch on console 8
Restore filesystem / /tmp /var  with recover

which are from
usr/share/rear/restore/NSR/default/400_restore_with_nsr.sh
plus a suspicious

Function 'iconv_open' failed for UTF-8 to ANSI conversion. The function may be unable to determine the current locale. Verify appropriate values in environment variables LC_MESSAGES, LC_ALL and LANG.

message where I cannot know wherefrom that comes.
Perhapos it comes from the NSR 'recover' command in
usr/share/rear/restore/NSR/default/400_restore_with_nsr.sh
You need to inspect your log to find that out.
If it is from the NSR 'recover' command it could indicate
that the NSR backup restore failed (or failed at a certain point).

Regarding UTF-8 and ReaR see
"Character encoding" in
https://github.com/rear/rear/wiki/Coding-Style

Simply put:
UTF-8 is not supported in ReaR.
Only plain 7-bit ASCII (more precisely the POSIX/C locale)
works in ReaR.

jsmeix commented at 2017-09-13 11:49:¶

@schabrolles regarding OS_VERSION:
I think with an empty predefined OS_VERSION
in /etc/rear/os.conf the automated setting
by the SetOSVendorAndVersion function
in usr/share/rear/lib/config-functions.sh
is skipped.

schabrolles commented at 2017-09-13 12:07:¶

@jsmeix chat about adding a test in config-functions.sh

if (test "$OS_VERSION" = none || test -z "$OS_VERSION")  ; then
    OS_VERSION="$( lsb_release -r -s | tr -s '[:blank:]' '_' )"
    test "$OS_VERSION" || Error "Failed to detect OS_VERSION. You may manually specify it in $CONFIG_DIR/os.conf"
fi

jsmeix commented at 2017-09-13 12:59:¶

@schabrolles
I think the comment

# If OS_VERSION has the default value, detect the actual value:

means that any predefined value should be accepted at this place
which is o.k. for me because in general I prefer to
"give the user the final power".

I think testing only for empty OS_VERSION is insufficient
because the user could have set something invalid.

I think in the particular code that depends on
particular OS_VERSION values, there could be an
assertion with an Error exit if something invalid is used.

As far as I see in
https://github.com/rear/rear/issues/1479#issuecomment-329125807
I assume that code is in the get_part_device_name_format
function therein via OS_MASTER_VERSION which is set
from the OS_VERSION value by SetOSVendorAndVersion.
The current code in the get_part_device_name_format
uses "if OS_MASTER_VERSION ... else ... fi" which means
the else clause is also used when OS_MASTER_VERSION
has a totally unexpected value.

I don't know what the right way is:
A generic test at a central place that
certain variables have right values,
cf. https://github.com/rear/rear/pull/1440
or
testing as needed at each place in the code?

The former requires continuous maintenance to keep
that central test up-to-date but all is at one place
while the latter has that functionality spread over
the whole code but the latter may work better
in real life pactice?

Personally I prefer assertions as needed
at each place in the code.

jsmeix commented at 2017-09-14 08:38:¶

@schabrolles FYI
in general regarding code that depends on OS_VENDOR
or OS_VERSION or similar vendor or version strings see
https://github.com/rear/rear/issues/1368#issuecomment-302410707

That reads in particular (excerpts):

we cannot rely on what version strings
the system tells about itself.
We must inspect "the real thing",
i.e. the really existing ... stuff.
When we inspect the really existing ... stuff
it means ... ReaR works generically.
...
I think Linux distribution specific scripts or
even special Linux distribution version specific scripts
will not really help because in the end any user on any
recent Linux system can manually create any[thing]
so that in the end only generically working ... scripts
could properly implement [anything] in ReaR.

Accordingly the OS_MASTER_VENDOR/OS_MASTER_VERSION
specific behaviour of the get_part_device_name_format() function
should ideally be replaced by inspecting "the real thing"
on the current system and then do "the right thing"
according to what there really is on the current system.
But I don't know if it is possible in this case to autodetect
what "the real thing" is on the current system.

If inspecting "the real thing" does not work,
using vendor or version strings is the only feasible way out.

When using vendor or version strings is the only way,
the code of get_part_device_name_format() works
sufficiently well because in particular the (SUSE) case
tests the old way (SLES11 - which will eventually die out)
and uses the current way (since SLES12) by default and
as fallback.

When a user has OS_VERSION explicitly set to an
invalid value, it will still work for the current way
and only in case of the old way it will fail.
From my point of view this is totally acceptable.
We do not need to overengineer the ReaR code
to be fail-safe for any possible misconfiguration.

schabrolles commented at 2017-09-14 16:07:¶

@jsmeix I agree .... But in the case of get_part_device_name_format(), I don't really know how we can do that ....
get_part_device_name_format() is called by apply_layout_mappings() during disk layout migration step.
It will generate diskrestore.sh script and migrate sda2 => mpatha-part2 or the other way mpatha-part2 => vda2

Before the generation of diskrestore.sh script, there is no way to check how the current system will name multipath devices partitions (p1, _part1, -part1 etc....) because no partition has been created yet. That's why I decided to use the OS_VERSION (or OS_MASTER_VENDOR). I know it is not the best way ...

The only alternative is to dig into the udev scripts to try to find and understand how the mpath partition will be named (not easy and not really reliable as script are different for each version/distro, they can change...).

If you have a better idea, please tell me (I will also try to have a second look on it)

Regarding this issue, I think there where some old data with empty OS_VERSION when the rescue script ran. that's why you got the message # The following information was added automatically by the mkrescue workflow: at the third line (usually it is on the first one).
As you said, because it was set OS_VERSION=, ReaR doesn't try to get it via lsb-release when rescue workflow occured.

jsmeix commented at 2017-09-15 08:53:¶

@schabrolles
many thanks for your detailed explanation why in this case
using vendor or version strings is the only reasonable way
(digging into udev will make you lose your reason ;-)

Regarding
"ReaR doesn't try to get it via lsb-release when rescue workflow":
There is no lsb-release program in the ReaR recovery system.
On my SLES12 system the /usr/bin/lsb_release script
seems to work in the ReaR recovery system:

RESCUE e205:~ # scp root@e205:/usr/bin/lsb_release /usr/bin/lsb_release

RESCUE e205:~ # type -a lsb_release
lsb_release is /bin/lsb_release

RESCUE e205:~ # lsb_release -i -s
SUSE

RESCUE e205:~ # lsb_release -r -s
12.2

RESCUE e205:~ # cat /etc/os-release 
NAME="SLES"
VERSION="12-SP2"
VERSION_ID="12.2"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP2"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp2"

But even lsb_release in the ReaR recovery system
won't help in this case here because the user had
specified an invalid value for OS_VERSION
in his config file and ReaR must obey what the
user has explicity specified.

I think the user must have the final power to even
specify invalid values.
I think a computer program must never ever make a final decision
against something its master (i.e. the user) has commanded.

A good example is nowadays "rm -rf /":

# rm -rf /
rm: it is dangerous to operate recursively on '/'
rm: use --no-preserve-root to override this failsafe

# rm -rf --no-preserve-root /
[what the user commanded happens]

What is problematic with the OS_VENDOR and OS_VERSION
variables in ReaR is that they do not behave in compliance with
many (but not all!) other ReaR config variables because
for the OS_VENDOR and OS_VERSION variables the
lsb_release automatism is used in SetOSVendorAndVersion only

if test "$OS_VENDOR" = generic -o "$OS_VERSION" = none

in contrast to other ReaR config variables because:
Often when a ReaR config variable is empty it tells ReaR to
"do something automatic", cf. default.conf

# Most variables can be set to an empty
# value VAR= which means that this
# setting is off or set to some automatic mode.

but note the 'most' (i.e. not all) and
the 'off or automatic' therein and note also in default.conf

# In case of doubt inspect the scripts
# how exactly a particular variable works.

and the OS_VENDOR and OS_VERSION variables
are described (shortly) in default conf and nothing there
indicates that empy values "do it right automatically".
I think the current OS_VENDOR and OS_VERSION behaviour
is probably not ideal but that "possibly unexpected behaviour"
also could happen is sufficiently documented.
Furthermore:
Setting a OS_VENDOR or OS_VERSION variable
explicitly to an empty value should raise the question
what then happens - i.e. how then things behave that
depend on OS_VENDOR or OS_VERSION (if nothing
depends on them there would be no such variables).

schabrolles commented at 2017-09-15 09:11:¶

@jsmeix When I say "ReaR doesn't try to get it via lsb-release when rescue workflow":
I mean when rear mkrescue command was process .... this mean on the source system, during backup/mkrescue process; not when the system is in rescue mode. I may be not use the good workflow name (my mistake)

jsmeix commented at 2017-09-15 09:24:¶

@schabrolles
ah! - thanks for clarification.

Via
https://github.com/rear/rear/commit/efd5537ae18ffb9eb82594c5891fb22aa17b5920
I added more explanatory descriptions for the
OS_VENDOR and OS_VERSION variables
in default.conf.

jsmeix commented at 2017-10-06 14:00:¶

Because "no news is good news" I assume
the questions are sufficiently answered
so that I can close this issue.
If the same issue persists this issue report can be reopened but
new and different issues need new and separated issue reports.

[Export of Github issue for rear/rear.]

#1479 Issue closed: An error occurred during layout recreation - SLES11SP4 on Power¶