#2993 Issue open: Rubrik CDM agent missing after ReaR recovery

Labels: enhancement, external tool

drakkainenn opened issue at 2023-05-25 09:00:

  • ReaR version ("/usr/sbin/rear -V"):
    Relax-and-Recover 2.7 / 2023-04-25

  • OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
    RHEL 8.4

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
    BACKUP=CDM
    OUTPUT=ISO
    TIMESYNC=NTP
    ONLY_INCLUDE_VG=("vg_system")
    GRUB_RESCUE=n
    COPY_AS_IS=( "${COPY_AS_IS[@]}" "/etc/rubrik" "/usr/bin/rubrik" "/var/log/rubrik" "/etc/pki" )
    UEFI_BOOTLOADER=/boot/efi/EFI/redhat/grubx64.efi
    SECURE_BOOT_BOOTLOADER=/boot/efi/EFI/redhat/shimx64.efi
    DISKS_TO_BE_WIPED='/dev/sdaa'

  • Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
    HP

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
    x86

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
    N/A

  • Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
    N/A

  • Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):
    N/A

  • Description of the issue (ideally so that others can reproduce it):
    Rubrik agent is missing after recovery:

[root@server rear]# ls -la /etc/rubrik
ls: cannot access '/etc/rubrik': No such file or directory

Original issue: https://github.com/rear/rear/issues/2483
Issue still exist
Rubrik version: 8.0.1

  • Workaround, if any:
    After recovery but before rebooting copy /etc/rubrik to /mnt/local/rubrik. Also copy /usr/bin/rubrik to /mnt/local/usr/bin/rubrik. Both copies should also copy the file permissions.

schlomo commented at 2023-05-29 10:17:

(Sorry, new to the topic) So ReaR should put the CDM agent back into the recovered system, right? Will you submit a PR for this or do you need assistance from us?

I can offer some additional thoughts about the CDM:

  • Currently ReaR waits for recovery and starts a rear_shell to wait for the user to confirm. We could extend this to wait for the appearance of a file, similar to how NFS4SERVER is implemented and thereby faciliate automated recovery without user input.
  • Can you be sure that the CDM agent is always installed under the hard coded paths? If not, then maybe dynamically determine the paths and use that instead, similar to how we handle this now for GALAXY11

pcahyna commented at 2023-05-29 10:20:

@DamaniN can you please have a look ?

schlomo commented at 2023-05-29 12:05:

BTW, would it be possible to also update the test matrix in 16-Rubrik-CDM.adoc to reflect the version tested and possibly also add more recent distros?

drakkainenn commented at 2023-06-14 07:19:

(Sorry, new to the topic) So ReaR should put the CDM agent back into the recovered system, right? Will you submit a PR for this or do you need assistance from us?

I can offer some additional thoughts about the CDM:

  • Currently ReaR waits for recovery and starts a rear_shell to wait for the user to confirm. We could extend this to wait for the appearance of a file, similar to how NFS4SERVER is implemented and thereby faciliate automated recovery without user input.
  • Can you be sure that the CDM agent is always installed under the hard coded paths? If not, then maybe dynamically determine the paths and use that instead, similar to how we handle this now for GALAXY11

Could you help with PR ?

List of files:

[root@server~]# rpm -ql rubrik-agent
/etc/cron.d/rubrik
/etc/init.d/rubrikagents
/etc/logrotate.d/rubrik
/etc/rubrik
/etc/rubrik/check_and_restart_agents.sh
/etc/rubrik/conf
/etc/rubrik/conf/agent_flags.conf
/etc/rubrik/conf/agent_version
/etc/rubrik/conf/bootstrap_flags.conf
/etc/rubrik/conf/certificate_info
/etc/rubrik/db2
/etc/rubrik/gen_tls_cert.sh
/etc/rubrik/jobs
/etc/rubrik/keys
/etc/rubrik/keys/rubrik.crt
/etc/rubrik/rba-keygen.sh
/etc/rubrik/start_stop.sh
/etc/rubrik/start_stop_agent.sh
/etc/rubrik/start_stop_bootstrap.sh
/etc/rubrik/stop_all.sh
/etc/systemd/system/rubrikagents-backup.service
/etc/systemd/system/rubrikagents-bootstrap.service
/usr/bin/rubrik
/usr/bin/rubrik/backup_agent_main
/usr/bin/rubrik/bootstrap_agent_main
/usr/bin/rubrik/db2
/usr/bin/rubrik/db2/db2_fetch_log_main
/usr/bin/rubrik/db2/rubrik_db2_tool.pex
/usr/bin/rubrik/db2/vendorlib
/usr/bin/rubrik/oracle_agent_main
/usr/bin/rubrik/sap_hana
/usr/bin/rubrik/sap_hana/sap_hana_agent_main
/usr/bin/rubrik/sap_hana/sap_hana_bootstrap_main
/usr/bin/rubrik/sap_hana_v2
/usr/bin/rubrik/sap_hana_v2/sap_hana_backint_main
/usr/bin/rubrik/sap_hana_v2/sap_hana_restore.pex
/usr/bin/rubrik/timeout
/usr/lib/.build-id
/usr/lib/.build-id/0b
/usr/lib/.build-id/0b/d24bea2752053a80479109d0d860512b34efe7
/usr/lib/.build-id/1d
/usr/lib/.build-id/1d/7facb752179b86c88adefc74c897c8c3466ad3
/usr/lib/.build-id/28
/usr/lib/.build-id/28/ba79c778f7402713aec6af319ee0fbaf3a8014
/usr/lib/.build-id/2a
/usr/lib/.build-id/2a/4ae7d03b8e5290a81a03858523cc0a0059ba36
/usr/lib/.build-id/3c
/usr/lib/.build-id/3c/0fd30d9df62825bebe3caf0c5c391a947bbe10
/usr/lib/.build-id/52
/usr/lib/.build-id/52/ecaa413ac4b8c84de29115a9f58c51877654b1
/usr/lib/.build-id/64
/usr/lib/.build-id/64/dffb10c0b7de102c6c2f1208f0f779cfd0ebe3
/usr/lib/.build-id/7b
/usr/lib/.build-id/7b/9caade8d6aea0bf99095101245bbf8b2bc1745
/usr/lib/.build-id/81
/usr/lib/.build-id/81/e118d25e2289dae7772b23b96c079d4d103995
/usr/lib/.build-id/d2
/usr/lib/.build-id/d2/542992691af1071a152d85156cc7e7db0a480f
/usr/lib64/libinject.so
/var/log/rubrik
/var/log/rubrik/db2

so we need directories:
/etc/rubrik
/usr/bin/rubrik/
/var/log/rubrik

and files:
/etc/cron.d/rubrik
/etc/init.d/rubrikagents
/etc/logrotate.d/rubrik
/etc/systemd/system/rubrikagents-backup.service
/etc/systemd/system/rubrikagents-bootstrap.service

[Export of Github issue for rear/rear.]