#2685 Issue closed: BACKUP=CDM ldd test fails with "needs additional libraries"

Labels: support / question, fixed / solved / done, external tool

bbabaria opened issue at 2021-09-29 15:31:

Relax-and-Recover (ReaR) Issue Template

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

  • ReaR version ("/usr/sbin/rear -V"): Relax-and-Recover 2.6 / Git

  • OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"): Red Hat Enterprise Linux Server release 7.6

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

[root@njidlpuppet01 ~]# cat /etc/redhat*
cat: /etc/redhat-access-insights: Is a directory
Red Hat Enterprise Linux Server release 7.6 (Maipo)
[root@njidlpuppet01 ~]# cat /etc/rear/site.conf
cat: /etc/rear/site.conf: No such file or directory
[root@njidlpuppet01 ~]# cat /etc/rear/local.conf
# This file etc/rear/local.conf is intended for the user's
# manual configuration of Relax-and-Recover (ReaR).
# For configuration through packages and other automated means
# we recommend a separated file named site.conf next to this file
# and leave local.conf as is (ReaR upstream will never ship a site.conf).
# The default OUTPUT=ISO creates the ReaR rescue medium as ISO image.
# You need to specify your particular backup and restore method for your data
# as the default BACKUP=REQUESTRESTORE does not really do that (see "man rear").
# Configuration variables are documented in /usr/share/rear/conf/default.conf
# and the examples in /usr/share/rear/conf/examples/ can be used as templates.
# ReaR reads the configuration files via the bash builtin command 'source'
# so bash syntax like VARIABLE="value" (no spaces at '=') is mandatory.
# Because 'source' executes the content as bash scripts you can run commands
# within your configuration files, in particular commands to set different
# configuration values depending on certain conditions as you need like
# CONDITION_COMMAND && VARIABLE="special_value" || VARIABLE="usual_value"
# but that means CONDITION_COMMAND gets always executed when 'rear' is run
# so ensure nothing can go wrong if you run commands in configuration files.


# Sets output to an be an ISO file
OUTPUT=ISO
# Specifies CDM as the backup and recovery application
BACKUP=CDM
[root@njidlpuppet01 ~]#
  • Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR): VM

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device): x86_64

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):

BIOS Information
        Vendor: Phoenix Technologies LTD
        Version: 6.00
        Release Date: 12/12/2018
        Address: 0xEA500
        Runtime Size: 88832 bytes
        ROM Size: 64 kB
        Characteristics:
  • Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe): Vmware virtual disk (SAN)

  • Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT"):

NAME                        KNAME     PKNAME    TRAN   TYPE FSTYPE       SIZE MOUNTPOINT
/dev/fd0                    /dev/fd0                   disk                4K
/dev/sda                    /dev/sda                   disk              100G
|-/dev/sda1                 /dev/sda1 /dev/sda         part xfs          500M /boot
|-/dev/sda2                 /dev/sda2 /dev/sda         part vfat         500M /boot/efi
`-/dev/sda3                 /dev/sda3 /dev/sda         part LVM2_member   85G
  |-/dev/mapper/rootVG-root /dev/dm-0 /dev/sda3        lvm  xfs           20G /
  |-/dev/mapper/rootVG-swap /dev/dm-1 /dev/sda3        lvm  swap          20G [SWAP]
  |-/dev/mapper/rootVG-usr  /dev/dm-2 /dev/sda3        lvm  xfs           10G /usr
  |-/dev/mapper/rootVG-tmp  /dev/dm-3 /dev/sda3        lvm  xfs           10G /tmp
  |-/dev/mapper/rootVG-home /dev/dm-4 /dev/sda3        lvm  xfs            5G /home
  |-/dev/mapper/rootVG-opt  /dev/dm-5 /dev/sda3        lvm  xfs           10G /opt
  `-/dev/mapper/rootVG-var  /dev/dm-6 /dev/sda3        lvm  xfs           10G /var
/dev/sr0                    /dev/sr0            sata   rom              1024M
  • Description of the issue (ideally so that others can reproduce it): we are getting below error while running rear -v mkrescue

  • and it generated the ISO file ..but using that ISO file we tried to restore the system and /var file systems get full and no rubrik agent is exist on restore system.

  • Workaround, if any:

  • Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):

[njidlpuppet01_rear-D mkrescue.output.txt](https://github.com/rear/rear/files/7253167/njidlpuppet01_rear-D.mkrescue.output.txt)
[root@njidlpuppet01 grub2]# rear -D mkrescue
Relax-and-Recover 2.6 / Git
Running rear mkrescue (PID 20809 date 2021-09-29 11:17:56)
Command line options: /usr/sbin/rear -D mkrescue
Using log file: /var/log/rear/rear-njidlpuppet01.log
Using build area: /var/tmp/rear.4GdGPQ4Gs3byR6x
Running workflow mkrescue on the normal/original system
Using '/bin/mkisofs' to create ISO filesystem images
Using autodetected kernel '/boot/vmlinuz-3.10.0-957.1.3.el7.x86_64' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
Disabling excluded components in /var/lib/rear/layout/disklayout.conf
Using guessed bootloader 'GRUB' for 'rear recover' (found in first bytes on /dev/sda)
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /var/lib/rear/layout/disklayout.conf)
Creating recovery system root filesystem skeleton layout
Handling network interface 'ens192'
ens192 is a physical device
Handled network interface 'ens192'
Skipping 'virbr0': not bound to any physical interface.
Included current keyboard mapping (via 'dumpkeys -f')
Included default US keyboard mapping /lib/kbd/keymaps/legacy/i386/qwerty/defkeymap.map.gz
Included other keyboard mappings in /lib/kbd/keymaps
Copying logfile /var/log/rear/rear-njidlpuppet01.log into initramfs as '/tmp/rear-njidlpuppet01-partial-2021-09-29T11:18:03-0400.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/3.10.0-957.1.3.el7.x86_64 (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
Skip copying broken symlink '/etc/mtab' target '/proc/10993/mounts' on /proc/ /sys/ /dev/ or /run/
Broken symlink '/usr/lib/modules/3.10.0-957.1.3.el7.x86_64/build' in recovery system because 'readlink' cannot determine its link target
Broken symlink '/usr/lib/modules/3.10.0-957.1.3.el7.x86_64/source' in recovery system because 'readlink' cannot determine its link target
Symlink '/usr/lib64/firefox/dictionaries' -> '/usr/share/myspell' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/share/myspell' via the 'COPY_AS_IS' configuration variable.
Testing that the recovery system in /var/tmp/rear.4GdGPQ4Gs3byR6x/rootfs contains a usable system
Testing each binary with 'ldd' and look for 'not found' libraries within the recovery system
/usr/lib64/eog/plugins/libfullscreen.so requires additional libraries
        libeog.so => not found
/usr/lib64/eog/plugins/libreload.so requires additional libraries
        libeog.so => not found
/usr/lib64/eog/plugins/libstatusbar-date.so requires additional libraries
        libeog.so => not found
/usr/lib64/python2.7/site-packages/dmidecodemod.so requires additional libraries
        libxml2mod.so => not found
/usr/lib64/samba/libaddns-samba4.so requires additional libraries
        libsamba-debug-samba4.so => not found
        libgenrand-samba4.so => not found
/usr/lib64/samba/libsamba-modules-samba4.so requires additional libraries
        libsamba-debug-samba4.so => not found
/usr/lib64/samba/pdb/smbpasswd.so requires additional libraries
        libsamba-debug-samba4.so => not found
        libreplace-samba4.so => not found
        libsecrets3-samba4.so => not found
        libsamba-security-samba4.so => not found
        libsamba3-util-samba4.so => not found
/usr/lib64/webkit2gtk-4.0/libicui18n.so.57.1 requires additional libraries
        libicuuc.so.57 => not found
/usr/lib64/webkit2gtk-4.0/libicuuc.so.57.1 requires additional libraries
        libicudata.so.57 => not found
/usr/lib64/firefox/libmozavcodec.so requires additional libraries
        libmozavutil.so => not found
/usr/lib64/firefox/libxul.so requires additional libraries
        libmozsandbox.so => not found
        liblgpllibs.so => not found
        libmozsqlite3.so => not found
        libmozgtk.so => not found
/usr/lib64/firefox/plugin-container requires additional libraries
        libxul.so => not found
/usr/lib64/libsmbldap.so.2 requires additional libraries
        libsamba-debug-samba4.so => not found
        libsmbd-shim-samba4.so => not found
        libsamba-security-samba4.so => not found
/usr/lib64/clang-private/libclang.so.6.0 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangIndex.so.6 => not found
        libclangLex.so.6 => not found
        libclangSema.so.6 => not found
        libclangTooling.so.6 => not found
        libclangARCMigrate.so.6 => not found
/usr/lib64/clang-private/libclangRewrite.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
/usr/lib64/clang-private/libclangARCMigrate.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangAnalysis.so.6 => not found
        libclangBasic.so.6 => not found
        libclangEdit.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangLex.so.6 => not found
        libclangRewrite.so.6 => not found
        libclangSema.so.6 => not found
        libclangSerialization.so.6 => not found
        libclangStaticAnalyzerCheckers.so.6 => not found
/usr/lib64/clang-private/libclangToolingCore.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
        libclangRewrite.so.6 => not found
/usr/lib64/clang-private/libclangAST.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
/usr/lib64/clang-private/libclangRewriteFrontend.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangEdit.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangLex.so.6 => not found
        libclangRewrite.so.6 => not found
        libclangSerialization.so.6 => not found
/usr/lib64/clang-private/libclangASTMatchers.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
/usr/lib64/clang-private/libclangAnalysis.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
/usr/lib64/clang-private/libclangSema.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangAnalysis.so.6 => not found
        libclangBasic.so.6 => not found
        libclangEdit.so.6 => not found
        libclangLex.so.6 => not found
/usr/lib64/clang-private/libclangCodeGen.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangAnalysis.so.6 => not found
        libclangBasic.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangLex.so.6 => not found
/usr/lib64/clang-private/libclangSerialization.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
        libclangSema.so.6 => not found
/usr/lib64/clang-private/libclangCrossTU.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangIndex.so.6 => not found
/usr/lib64/clang-private/libclangDriver.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
/usr/lib64/clang-private/libclangHandleCXX.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
        libclangCodeGen.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangLex.so.6 => not found
        libclangTooling.so.6 => not found
/usr/lib64/clang-private/libclangDynamicASTMatchers.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangASTMatchers.so.6 => not found
        libclangBasic.so.6 => not found
/usr/lib64/clang-private/libclangEdit.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
/usr/lib64/clang-private/libclangFormat.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
        libclangToolingCore.so.6 => not found
/usr/lib64/clang-private/libclangStaticAnalyzerCore.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangASTMatchers.so.6 => not found
        libclangAnalysis.so.6 => not found
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
        libclangRewrite.so.6 => not found
/usr/lib64/clang-private/libclangFrontend.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangDriver.so.6 => not found
        libclangEdit.so.6 => not found
        libclangLex.so.6 => not found
        libclangParse.so.6 => not found
        libclangSema.so.6 => not found
        libclangSerialization.so.6 => not found
/usr/lib64/clang-private/libclangFrontendTool.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
        libclangCodeGen.so.6 => not found
        libclangDriver.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangRewriteFrontend.so.6 => not found
        libclangARCMigrate.so.6 => not found
        libclangStaticAnalyzerFrontend.so.6 => not found
/usr/lib64/clang-private/libclangIndex.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangFormat.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangRewrite.so.6 => not found
        libclangSerialization.so.6 => not found
        libclangToolingCore.so.6 => not found
/usr/lib64/clang-private/libclangTooling.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangASTMatchers.so.6 => not found
        libclangBasic.so.6 => not found
        libclangDriver.so.6 => not found
        libclangFormat.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangLex.so.6 => not found
        libclangRewrite.so.6 => not found
        libclangToolingCore.so.6 => not found
/usr/lib64/clang-private/libclangLex.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
/usr/lib64/clang-private/libclangParse.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
        libclangSema.so.6 => not found
/usr/lib64/clang-private/libclangStaticAnalyzerCheckers.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangASTMatchers.so.6 => not found
        libclangAnalysis.so.6 => not found
        libclangBasic.so.6 => not found
        libclangLex.so.6 => not found
        libclangStaticAnalyzerCore.so.6 => not found
/usr/lib64/clang-private/libclangStaticAnalyzerFrontend.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangAnalysis.so.6 => not found
        libclangBasic.so.6 => not found
        libclangFrontend.so.6 => not found
        libclangLex.so.6 => not found
        libclangStaticAnalyzerCheckers.so.6 => not found
        libclangStaticAnalyzerCore.so.6 => not found
/usr/lib64/clang-private/libclangToolingASTDiff.so.6.0.1 requires additional libraries
        libclangBasic.so.6 => not found
        libclangAST.so.6 => not found
        libclangLex.so.6 => not found
/usr/lib64/clang-private/libclangToolingRefactor.so.6.0.1 requires additional libraries
        libclangAST.so.6 => not found
        libclangASTMatchers.so.6 => not found
        libclangBasic.so.6 => not found
        libclangFormat.so.6 => not found
        libclangIndex.so.6 => not found
        libclangLex.so.6 => not found
        libclangRewrite.so.6 => not found
        libclangToolingCore.so.6 => not found
ReaR recovery system in '/var/tmp/rear.4GdGPQ4Gs3byR6x/rootfs' needs additional libraries, check /var/log/rear/rear-njidlpuppet01.log for details
Testing that the existing programs in the PROGS array can be found as executable command within the recovery system
Testing that each program in the REQUIRED_PROGS array can be found as executable command within the recovery system
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (758455261 bytes) in 109 seconds
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-njidlpuppet01.iso (733M)
Exiting rear mkrescue (PID 20809) and its descendant processes ...
Running exit tasks
To remove the build area use (with caution): rm -Rf --one-file-system /var/tmp/rear.4GdGPQ4Gs3byR6x

gdha commented at 2021-09-29 15:40:

@bbabaria See also similar issue #2674

pcahyna commented at 2021-10-05 18:00:

@DamaniN seems to be some issue with Rubrik CDM support, could you please have a look?
By the way, in case of BACKUP=CDM, the ISO seems to contain too much. Do you really need the files under /usr/lib64/samba, /usr/lib64/firefox, /usr/lib64/eog etc. ?

DamaniN commented at 2021-10-05 23:52:

@bbabaria, I have a few questions:

  • What version of CDM are you running?

  • If you include the statement:

    export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/bind9-export:/usr/lib64/eog:/usr/lib64/python3.6/site-packages:/usr/lib64/samba:/usr/lib64/firefox:/usr/lib64/python3.6/site-packages/hawk ey:/usr/lib64/mutter-4"

    in your /etc/rear/local.conf file, do the errors go away?

  • Can you verify that the Rubrik client is installed on the source system?

    • Running rpm -qa | grep -i rubrik before running rear -v mkrescue will confirm this.
  • If you check /var/tmp/rear.<random_name>/rootfs/bin/rubrik/ after rear -v mkrescue is run, do you see any files there?

    • If not, the script is failing to copy the Rubrik client over.
  • Can you check that the Rubrik client was copied to the ISO by running the following commands:

    • If you are running out of space in /tmp, you can use a different file system from /tmp/rear.initrd
    • Verify that you see the Rubrik binaries after the ls command.
    # mount -o loop rear-localhost.iso /mnt
    
    # file /mnt/isolinux/initrd.cgz
    /mnt/isolinux/initrd.cgz: gzip compressed data
    
    # mkdir /tmp/rear.initrd
    
    # cd /tmp/rear.initrd
    
    # zcat /mnt/isolinux/initrd.cgz | cpio -idmv
    ...
    
    # umount /mnt
    
    # ls rootfs/bin/rubrik/
    
  • How much memory is in the recovery VM (the one you are booting the ISO on)?

    • I've found that there are boot problems with VMs less than 4 GB of memory.
  • When you state that and it generated the ISO file ..but using that ISO file we tried to restore the system and /var file systems get full and no rubrik agent is exist on restore system. is that /var on the recovery target?

  • Also on the recovery target, do you see /bin/rubrik or /usr/bin/rubrik?

  • When you run rear recover on the recovery target, do you get a prompt for CDM like this one:

image

DamaniN commented at 2021-10-06 00:08:

@pcahyna,

I've used:

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/bind9-export:/usr/lib64/eog:/usr/lib64/python3.6/site-packages:/usr/lib64/samba:/usr/lib64/firefox:/usr/lib64/python3.6/site-packages/hawk ey:/usr/lib64/mutter-4"

to work around the check of ldd not finding libraries in the past. It's likely that this can be skipped, however, the errors during rear mkrescue will persist. Is there a way to suppress the ldd errors another way?

jsmeix commented at 2021-10-06 09:34:

In general FYI regarding LD_LIBRARY_PATH see
https://github.com/rear/rear/issues/2674#issuecomment-935821409
and follow the links therein.

Regarding "Is there a way to suppress the ldd errors another way?"
see NON_FATAL_BINARIES_WITH_MISSING_LIBRARY
in usr/share/rear/conf/default.conf currently at
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L1732

When CDM binaries need a special LD_LIBRARY_PATH
then the right solution for CDM is using a CDM_LD_LIBRARY_PATH
config variable in the same way as the current *_LD_LIBRARY_PATH
variables (e.g. TSM_LD_LIBRARY_PATH and others) are used
in particular in usr/share/rear/build/default/990_verify_rootfs.sh

jsmeix commented at 2021-10-06 12:17:

@bbabaria
you wrote that you use "Relax-and-Recover 2.6 / Git"
but I don't know what exact Git commit you use.

If you use an older Git commit before Nov 26 2020
you should update to our latest GitHub master code because
the GitHub master code is the only place where we fix things
so it helps when what you use matches exactly where
we would fix things.

On Nov 26 2020 I completely overhauled ldd test in 990_verify_rootfs.sh
via https://github.com/rear/rear/pull/2523
so ensure you have at least its matching commit
https://github.com/rear/rear/commit/6b804a5cfcc45c1e2dc33110e33113d4b2db613f

See "Testing current ReaR upstream GitHub master code" at
https://en.opensuse.org/SDB:Disaster_Recovery

bbabaria commented at 2021-10-06 14:14:

Hi Damani,

Thank you so much for your help.

· What version of CDM are you running? – not sure but I used Git 2.6

· How much memory is in the recovery VM (the one you are booting the ISO on)? I've found that there are boot problems with VMs less than 4 GB of memory. – 8GB

· Can you verify that the Rubrik client is installed on the system? Running rpm -qa | grep -i rubrik before running rear -v mkrescue will confirm this. - rubrik-agent-5.3.2.p1.a9f0b4c279665c858bdceb1746d15967-1.0.x86_64

· If you check /var/tmp/rear.<random_name>/rootfs/bin/rubrik/ after rear -v mkrescue is run, do you see any files there? If not, the script is failing to copy the Rubrik client over. – I see only outputfs and not rootfs

· Can you check that the Rubrik client was copied to the ISO by running the following commands:

 *   If you are running out of space in /tmp, you can use a different file system from /tmp/rear.initrd

· When you state that and it generated the ISO file ..but using that ISO file we tried to restore the system and /var file systems get full and no rubrik agent is exist on restore system. is that /var on the recovery target? - Yes

· Also on the recovery target to you see /bin/rubrik or /usr/bin/rubrik? - we are missing rubrik client after restore done

· When you run rear recover on the recovery target do you get a prompt for CDM like this one: Yes

Thank you,
Babulal

bbabaria commented at 2021-10-06 14:21:

Hi Johannes,

Thank you so much for your help and support.

I used this link to download rear and installed on my redhat 7.6 server

git clone https://github.com/rear/rear.git

and I do see 990_verify_rootfs.sh file in under /usr/share/rear/build/default/ with matching commit

Thank you,
Babulal

pcahyna commented at 2021-10-06 14:34:

@bbabaria @DamaniN

If you check /var/tmp/rear.<random_name>/rootfs/bin/rubrik/ after rear -v mkrescue is run, do you see any files there? If not, the script is failing to copy the Rubrik client over. – I see only outputfs and not rootfs

you need to run rear in debug mode (or debugscripts mode): rear -d mkrescue, otherwise it deletes the temporary directory at the end. Or, you can set KEEP_BUILD_DIR=y. (debug mode sets this automatically, verbose mode does not.)

pcahyna commented at 2021-10-06 14:36:

@bbabaria by the way, it is strange that you see outputfs there without debug mode. Can you please check what is inside? (ls -a /var/tmp/rear.<random_name>/outputfs.) I suspect there is some bug in temporary directory removal.

pcahyna commented at 2021-10-06 14:40:

@bbabaria or was it with -D ?

bbabaria commented at 2021-10-06 14:47:

Hi Pcahyna,

I see empty folder with server name only

***@***.*** njidlpuppet01]# pwd
/var/tmp/rear.XjxKxdanJDxXR1J/outputfs/njidlpuppet01
***@***.*** njidlpuppet01]# ls -ltr
total 0
***@***.*** njidlpuppet01]#

Thank you,
Babulal

pcahyna commented at 2021-10-06 15:09:

@bbabaria thanks, but please add -a to ls options - there could be some dot-file. And was it created using rear -v or rear -D?

bbabaria commented at 2021-10-06 15:14:

Hi Pcahyna,

There are only one empty files generated and I had used rear –v mkrescue

***@***.*** njidlpuppet01]# ls -al
total 0
drwxr-x---. 2 root root 23 Oct 6 00:09 .
drwx------. 3 root root 27 Oct 6 00:09 ..
-rw-------. 1 root root 0 Oct 6 00:09 .lockfile
***@***.*** njidlpuppet01]# cat .lockfile
***@***.*** njidlpuppet01]#

Thank you,
Babulal

pcahyna commented at 2021-10-06 15:18:

@bbabaria thanks for the information, so .lockfile was not properly removed. Now, back to the main issue - please use -D to keep the directory that @DamaniN asked for, and please attach the complete log (/var/log/rear/rear-njidlpuppet01.log) here, not just the terminal output.

bbabaria commented at 2021-10-06 15:51:

Hi Pcahyna,

Thank you so much for your continuously help and support.

Looks like we are exceeding file transfer size over email ..i got mail delivery failed message

Here is the output in small portion.

2021-10-06 11:26:42.738255333 Relax-and-Recover 2.6 / Git
2021-10-06 11:26:42.745436303 Running rear mkrescue (PID 11879 date 2021-10-06 11:26:42)
2021-10-06 11:26:42.751345559 Command line options: /usr/sbin/rear -D mkrescue
2021-10-06 11:26:42.756082469 Using log file: /var/log/rear/rear-njidlpuppet01.log
2021-10-06 11:26:42.759966833 Using build area: /var/tmp/rear.4vMz5UR273VOvkW
2021-10-06 11:26:42.766251806 Current set of flags is 'hB'
2021-10-06 11:26:42.773125951 The debugscripts flags are 'x'
2021-10-06 11:26:42.777976821 Combining configuration files
2021-10-06 11:26:42.783029788 Including /etc/rear/os.conf
2021-10-06 11:26:42.786900620 Entering debugscript mode via 'set -x'.
+ source /etc/rear/os.conf
++ OS_VENDOR=RedHatEnterpriseServer
++ OS_VERSION=7.6
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.794895514 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.794895514 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:42.801808456 Including conf/Linux-i386.conf
2021-10-06 11:26:42.805636354 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/conf/Linux-i386.conf
++ REQUIRED_PROGS+=(sfdisk)
++ PROGS+=(grub lilo)
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.814232002 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.814232002 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:42.818856361 Including conf/GNU/Linux.conf
2021-10-06 11:26:42.822823379 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/conf/GNU/Linux.conf
++ REQUIRED_PROGS+=(chroot ip less parted readlink)
++ PROGS+=(partprobe fdisk cfdisk sfdisk)
++ PROGS+=(rpc.statd rpcbind mknod blkid vol_id udev_volume_id portmap rpcinfo tac reboot halt shutdown killall5 killall tee ifconfig nslookup route ifenslave ifrename nameif klogd syslog-ng syslogd rsyslogd date rmdir init telinit ethtool expand insmod modprobe lsmod true false mingetty agetty getty rmmod hostname uname sleep logger pstree ln dirname basename tty ping netstat free traceroute xxd vi pico nano rmmod df dmesg du gzip netcat curl top iptraf joe getent id ldd strace rsync tail head find md5sum mkfs mkfs.ext2 mkfs.ext3 mkfs.ext4 mkfs.ext4dev mkfs.jfs mkfs.xfs mkfs.reiserfs mkfs.vfat mkfs.btrfs mkreiserfs fsck fsck.ext2 fsck.ext3 fsck.ext4 fsck.ext4dev fsck.xfs fsck.reiserfs reiserfsck fsck.btrfs findmnt btrfsck tune2fs tune4fs xfs_admin xfs_db xfs_repair xfs_info xfs_growfs btrfs jfs_tune reiserfstune egrep fgrep chmod chown stat mkswap swapon swapoff mknod touch scsi_id lsscsi logd initctl lspci usleep mktemp /bin/true strace which mkfifo seq openvt poweroff chacl getfacl setfacl attr getfattr setfattr mpath_wait xargs sg_inq env w dosfslabel sysctl blockdev lsblk clear)
++ LIBS+=(/lib*/libnss_dns* /lib*/libnss_files* /lib/*/libnss_dns* /lib/*/libnss_files* /lib*/libgcc_s* /lib*/libresolv* /usr/lib*/rsyslog/*so /lib*/rsyslog/*so /usr/lib*/syslog-ng/* /lib*/libnsspem.so* /usr/lib*/libnsspem.so* /lib*/libfreebl*.so* /usr/lib*/libfreebl*.so* /lib*/libnss3.so* /usr/lib*/libnss3.so* /lib*/libnssutil3.so* /usr/lib*/libnssutil3.so* /lib*/libsoftokn3.so* /usr/lib*/libsoftokn3.so* /lib*/libsqlite3.so* /usr/lib*/libsqlite3.so* /lib*/libfreeblpriv3.so* /usr/lib*/libfreeblpriv3.so* /lib*/libssl.so* /usr/lib*/libssl.so* /lib*/libnssdbm3.so* /usr/lib*/libnssdbm3.so*)
++ COPY_AS_IS+=(/dev /etc/inputr[c] /etc/protocols /etc/services /etc/rpc /etc/termcap /etc/terminfo /lib*/terminfo /usr/share/terminfo /etc/netconfig /etc/mke2fs.conf /etc/*-release /etc/localtime /etc/magic /usr/share/misc/magic /etc/dracut.conf /etc/dracut.conf.d /usr/lib/dracut /sbin/modprobe.ksplice-orig /etc/sysctl.conf /etc/sysctl.d /etc/e2fsck.conf)
++ COPY_AS_IS+=('/etc/ssl/certs/*' '/etc/pki/*' '/usr/lib/ssl/*' '/usr/share/ca-certificates/*' '/etc/ca-certificates/*')
++ COPY_AS_IS_EXCLUDE+=(dev/shm/\*)
++ COPY_AS_IS_EXCLUDE+=('/etc/pki/tls/private' '/etc/pki/CA/private' '/etc/pki/nssdb/key*.db' '/usr/lib/ssl/private')
++ KERNEL_CMDLINE+=' selinux=0'
++ CLONE_USERS+=(daemon rpc usbmuxd usbmux vcsa nobody dbus)
++ CLONE_GROUPS+=(tty usbmuxd usbmux fuse kvm oinstall dbus)
++ type -p lspci
++ lspci
++ grep --quiet ' VGA .*AMD'
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.877586565 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.877586565 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:42.885368001 Including /etc/rear/local.conf
2021-10-06 11:26:42.889254570 Entering debugscript mode via 'set -x'.
+ source /etc/rear/local.conf
++ OUTPUT=ISO
++ BACKUP=CDM
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.897865617 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.897865617 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:42.904361621 ======================
2021-10-06 11:26:42.908057882 Running 'init' stage
2021-10-06 11:26:42.911746384 ======================
2021-10-06 11:26:42.924248468 Including init/default/005_verify_os_conf.sh
2021-10-06 11:26:42.927953745 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/init/default/005_verify_os_conf.sh
++ test -f /etc/rear/os.conf
++ return
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.935945708 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.935945708 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:42.943851797 Including init/default/010_EFISTUB_check.sh
2021-10-06 11:26:42.948000212 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/init/default/010_EFISTUB_check.sh
++ is_false no
++ case "$1" in
++ return 0
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.956700347 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.956700347 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:42.964563264 Including init/default/010_set_drlm_env.sh
2021-10-06 11:26:42.968378357 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/init/default/010_set_drlm_env.sh
++ is_true n
++ case "$1" in
++ return 1
++ return 0
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.976745123 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.976745123 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:42.984854385 Including init/default/030_update_recovery_system.sh
2021-10-06 11:26:42.988931802 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/init/default/030_update_recovery_system.sh
++ test ''
++ return 0
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:42.997504863 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:42.997504863 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:43.005599974 Including init/default/050_check_rear_recover_mode.sh
2021-10-06 11:26:43.009643170 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/init/default/050_check_rear_recover_mode.sh
++ test -f /etc/rear-release
++ case "$WORKFLOW" in
++ LogPrint 'Running workflow mkrescue on the normal/original system'
++ Log 'Running workflow mkrescue on the normal/original system'
++ test -w /var/log/rear/rear-njidlpuppet01.log
++ echo '2021-10-06 11:26:43.021093862 Running workflow mkrescue on the normal/original system'
2021-10-06 11:26:43.021093862 Running workflow mkrescue on the normal/original system
++ Print 'Running workflow mkrescue on the normal/original system'
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log
+ echo '2021-10-06 11:26:43.027983299 Leaving debugscript mode (back to previous bash flags and options settings).'
2021-10-06 11:26:43.027983299 Leaving debugscript mode (back to previous bash flags and options settings).
2021-10-06 11:26:43.036540244 Including init/default/950_check_missing_programs.sh
2021-10-06 11:26:43.040633953 Entering debugscript mode via 'set -x'.
+ source /usr/share/rear/init/default/950_check_missing_programs.sh
++ local prog
++ missing_progs=()
++ local missing_progs
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary /usr/sbin/rear
++ for bin in '$@'
++ type /usr/sbin/rear
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary awk
++ for bin in '$@'
++ type awk
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary bash
++ for bin in '$@'
++ type bash
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary bc
++ for bin in '$@'
++ type bc
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary cat
++ for bin in '$@'
++ type cat
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary cmp
++ for bin in '$@'
++ type cmp
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary cp
++ for bin in '$@'
++ type cp
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary cpio
++ for bin in '$@'
++ type cpio
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary cut
++ for bin in '$@'
++ type cut
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary dd
++ for bin in '$@'
++ type dd
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary diff
++ for bin in '$@'
++ type diff
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary df
++ for bin in '$@'
++ type df
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary dumpkeys
++ for bin in '$@'
++ type dumpkeys
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary echo
++ for bin in '$@'
++ type echo
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary expr
++ for bin in '$@'
++ type expr
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary file
++ for bin in '$@'
++ type file
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary getopt
++ for bin in '$@'
++ type getopt
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary grep
++ for bin in '$@'
++ type grep
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary join
++ for bin in '$@'
++ type join
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary kbd_mode
++ for bin in '$@'
++ type kbd_mode
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary loadkeys
++ for bin in '$@'
++ type loadkeys
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary ls
++ for bin in '$@'
++ type ls
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary mkdir
++ for bin in '$@'
++ type mkdir
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary mount
++ for bin in '$@'
++ type mount
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary mountpoint
++ for bin in '$@'
++ type mountpoint
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary mv
++ for bin in '$@'
++ type mv
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary pidof
++ for bin in '$@'
++ type pidof
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary ps
++ for bin in '$@'
++ type ps
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary pwd
++ for bin in '$@'
++ type pwd
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary rm
++ for bin in '$@'
++ type rm
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary sed
++ for bin in '$@'
++ type sed
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary seq
++ for bin in '$@'
++ type seq
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary sort
++ for bin in '$@'
++ type sort
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary strings
++ for bin in '$@'
++ type strings
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary sync
++ for bin in '$@'
++ type sync
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary tar
++ for bin in '$@'
++ type tar
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary test
++ for bin in '$@'
++ type test
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary tr
++ for bin in '$@'
++ type tr
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary umount
++ for bin in '$@'
++ type umount
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary uniq
++ for bin in '$@'
++ type uniq
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary wc
++ for bin in '$@'
++ type wc
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary sfdisk
++ for bin in '$@'
++ type sfdisk
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary chroot
++ for bin in '$@'
++ type chroot
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary ip
++ for bin in '$@'
++ type ip
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary less
++ for bin in '$@'
++ type less
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary parted
++ for bin in '$@'
++ type parted
++ return 0
++ for prog in '"${REQUIRED_PROGS[@]}"'
++ has_binary readlink
++ for bin in '$@'
++ type readlink
++ return 0
++ contains_visible_char ''
+++ tr -d -c '[:graph:]'
++ test ''
++ return 0
+ source_return_code=0
+ test 0 -eq 0
+ cd /rear-cdm/rear
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-njidlpuppet01.log

I ran rear –D mkrescue and now I see rootfs directory under /var/tmp/rear.4vMz5UR273VOvkW

***@***.*** rear.4vMz5UR273VOvkW]# ls -al
total 4
drwx------.  5 root root   47 Oct  6 11:31 .
drwxrwxrwt. 18 root root 4096 Oct  6 11:32 ..
drwx------.  3 root root   27 Oct  6 11:31 outputfs
drwxr-xr-x. 15 root root  223 Oct  6 11:30 rootfs
drwxr-xr-x.  6 root root  247 Oct  6 11:31 tmp

ls –ltr rootfs
total 1448
drwxr-xr-x. 15 root root     223 Oct  6 11:30 .
drwx------.  5 root root      47 Oct  6 11:31 ..
drwxr-xr-x.  3 root root    8192 Oct  6 11:28 bin
drwxr-xr-x.  3 root root      19 Oct  6 11:26 boot
drwxr-xr-x. 18 root root    4096 Oct  6 10:15 dev
drwxr-xr-x. 26 root root    4096 Oct  6 11:28 etc
lrwxrwxrwx.  1 root root       8 Oct  6 11:26 init -> bin/init
lrwxrwxrwx.  1 root root       7 Oct  6 11:26 lib -> usr/lib
lrwxrwxrwx.  1 root root       9 Oct  6 11:26 lib64 -> usr/lib64
-rw-r--r--.  1 root root 1454428 Oct  6 11:30 md5sums.txt
drwxr-xr-x.  4 root root      32 Aug 18 10:29 mnt
drwxr-xr-x.  2 root root       6 Aug 18 12:07 proc
drwx------.  3 root root      39 Oct  6 11:28 root
drwxr-xr-x.  7 root root      77 Aug 18 10:29 run
lrwxrwxrwx.  1 root root       3 Oct  6 11:26 sbin -> bin
drwxr-xr-x.  2 root root       6 Aug 18 12:07 selinux
drwxr-xr-x.  2 root root       6 Aug 18 12:07 sys
drwxr-xr-x.  2 root root      69 Oct  6 11:26 tmp
drwxr-xr-x.  7 root root      96 Oct  6 11:26 usr
drwxr-xr-x.  8 root root      75 Aug 18 10:29 var

Thank you,
Babulal

pcahyna commented at 2021-10-06 15:58:

@bbabaria please don't use e-mail for that then, use the web interface. And please answer @DamaniN 's question: "If you check /var/tmp/rear.<random_name>/rootfs/bin/rubrik/ after rear -v mkrescue is run, do you see any files there? If not, the script is failing to copy the Rubrik client over." [rear -v mkrescue should read rear -D mkrescue.]

bbabaria commented at 2021-10-06 16:11:

Hi Pcahyna,

After ran rear –D mkrescue I see files under /var/tmp/rear.4vMz5UR273VOvkW/rootfs/bin/rubrik

***@***.*** rubrik]# pwd
/var/tmp/rear.4vMz5UR273VOvkW/rootfs/bin/rubrik
***@***.*** rubrik]# ls -al
total 129652
drwxr-xr-x. 3 root root 100 Sep 21 20:12 .
drwxr-xr-x. 3 root root 8192 Oct 6 11:28 ..
-rwxr--r--. 1 root root 18977992 Sep 21 20:12 backup_agent_main
-rwxr-xr-x. 1 root root 22233904 Jun 4 10:33 bootstrap_agent_main
-rwxr-xr-x. 1 root root 91531528 Sep 21 20:12 oracle_agent_main
drwxr-xr-x. 2 root root 64 Sep 21 20:12 sap_hana
***@***.*** rubrik]#

Thank you,
Babulal

bbabaria commented at 2021-10-06 16:31:

Hi Pcahyna,

And if I run rear –v mkresuce I don’t see rootfs directory at all. So do you think I need to always use –D option ?

Thank you,
Babulal

pcahyna commented at 2021-10-06 16:36:

if I run rear –v mkresuce I don’t see rootfs directory at all. So do you think I need to always use –D option ?

You need to use -d or -D only if you need to debug the content of rootfs. For normal operation, -v or no option is sufficient, because it removes rootfs and other temporary files at the end of rear run to prevent wasting space in /var.

DamaniN commented at 2021-10-06 22:10:

@bbabaria,

It seems that the Rubrik files are making it into the build directory. Can you check the ISO image next using these instructions:

  • Can you check that the Rubrik client was copied to the ISO by running the following commands:

    • If you are running out of space in /tmp, you can use a different file system from /tmp/rear.initrd
    • Verify that you see the Rubrik binaries after the ls command.
    # mount -o loop rear-localhost.iso /mnt
    
    # file /mnt/isolinux/initrd.cgz
    /mnt/isolinux/initrd.cgz: gzip compressed data
    
    # mkdir /tmp/rear.initrd
    
    # cd /tmp/rear.initrd
    
    # zcat /mnt/isolinux/initrd.cgz | cpio -idmv
    ...
    
    # umount /mnt
    
    # ls rootfs/bin/rubrik/
    

bbabaria commented at 2021-10-07 13:05:

Hi Damani,

Good Morning !!

It seems like rubrik files are not there either.

boot/grub2/grubenv
boot/grub2/grub.cfg
boot/grub2/grub.cfg.1547621242.rpmsave
md5sums.txt
3516924 blocks

***@***.*** rear.initrd]# umount /mnt
***@***.*** rear.initrd]# ...
bash: ...: command not found...
Similar command is: '..'
***@***.*** rear.initrd]# ls rootfs/bin/rubrik/
ls: cannot access rootfs/bin/rubrik/: No such file or directory

***@***.*** rear.initrd]# pwd
/tmp/rear.initrd
***@***.*** rear.initrd]# ls -ltr
total 1448
drwxr-xr-x. 2 root root 6 Aug 18 12:07 sys
drwxr-xr-x. 2 root root 6 Aug 18 12:07 selinux
drwxr-xr-x. 2 root root 6 Aug 18 12:07 proc
-rw-r--r--. 1 root root 1454465 Oct 7 00:08 md5sums.txt
drwxr-xr-x. 26 root root 4096 Oct 7 08:58 etc
drwxr-xr-x. 3 root root 8192 Oct 7 08:58 bin
drwx------. 3 root root 39 Oct 7 08:58 root
drwxr-xr-x. 4 root root 32 Oct 7 08:58 mnt
drwxr-xr-x. 18 root root 4096 Oct 7 08:58 dev
drwxr-xr-x. 2 root root 69 Oct 7 08:58 tmp
drwxr-xr-x. 7 root root 77 Oct 7 08:58 run
drwxr-xr-x. 7 root root 96 Oct 7 08:58 usr
drwxr-xr-x. 8 root root 75 Oct 7 08:58 var
lrwxrwxrwx. 1 root root 3 Oct 7 08:58 sbin -> bin
lrwxrwxrwx. 1 root root 9 Oct 7 08:58 lib64 -> usr/lib64
lrwxrwxrwx. 1 root root 7 Oct 7 08:58 lib -> usr/lib
lrwxrwxrwx. 1 root root 8 Oct 7 08:58 init -> bin/init
drwxr-xr-x. 3 root root 19 Oct 7 08:58 boot

Thank you,
Babulal

pcahyna commented at 2021-10-07 13:26:

@bbabaria @DamaniN

***@***.*** rear.initrd]# ls rootfs/bin/rubrik/ 
ls: cannot access rootfs/bin/rubrik/: No such file or directory

I suspect at this point one should use only bin/rubrik, not rootfs/bin/rubrik.

bbabaria commented at 2021-10-07 13:37:

Hi Pcahyna,

Good Morning!!

Looks like rubrik files are there in bin/rubrik. So what is the next step to resolve this issues while restoring from this ISO to get rubrik agent on target.

***@***.*** rear.initrd]# ls bin/rubrik
backup_agent_main bootstrap_agent_main oracle_agent_main sap_hana
***@***.*** rear.initrd]#

Thank you,
Babulal

pcahyna commented at 2021-10-07 13:45:

Hi Babulal, I am not familiar with Rubrik CDM at all, so I must defer this to @DamaniN .

bbabaria commented at 2021-10-07 15:13:

Thank you Pcahyna !!

Damani , can you please suggest on this one.

Thank you,
Babulal

github-actions commented at 2021-12-08 02:17:

Stale issue message

jsmeix commented at 2022-01-26 13:00:

This issue is likely avoided via https://github.com/rear/rear/pull/2747
but the actual root cause might be related to
https://github.com/rear/rear/issues/2743#issuecomment-1022133392
on the other hand when the whole /usr/lib64 is included
there should be nothing missing in /usr/lib64


[Export of Github issue for rear/rear.]