#2337 Issue closed: "rear recover" with BACKUP=CDM fails in 630_install_grub.sh because /mnt/local/boot/grub does not exist

Labels: support / question, fixed / solved / done, minor bug

moonSymph opened issue at 2020-03-03 08:20:

Relax-and-Recover (ReaR) Issue Template

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

  • ReaR version ("/usr/sbin/rear -V"):
    Relax-and-Recover 2.5 / 2019-05-10

  • OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):
    OS_VENDOR=RedHatEnterpriseServer
    OS_VERSION=6.10

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
    cat /etc/rear/local.conf
    #2249
    OUTPUT=ISO
    BACKUP=CDM
    OUTPUT_URL=file:///root/backup/

  • Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
    VMWare VM

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
    x86

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
    BIOS, Bootloader: GRUB

  • Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
    Local Disk

  • Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT" or "lsblk" as makeshift):

[root@localhost ~]# lsblk
NAME                        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                           8:0    0   40G  0 disk
├─sda1                        8:1    0  500M  0 part /boot
└─sda2                        8:2    0 39.5G  0 part
  ├─VolGroup-lv_root (dm-0) 253:0    0 35.7G  0 lvm  /
  └─VolGroup-lv_swap (dm-1) 253:1    0  3.9G  0 lvm  [SWAP]
sr0                          11:0    1  3.6G  0 rom
  • Description of the issue (ideally so that others can reproduce it):
    Booted with the ISO, when i run "rear recover" error occurred, could not find directory /boot/grub.
    But i was able to go into /boot/grub..

  • Workaround, if any:
    NA

  • Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
    image

jsmeix commented at 2020-03-03 11:35:

@moonSymph
the message Could not find directory /boot/grub. was from
https://github.com/rear/rear/blob/master/usr/share/rear/finalize/Linux-i386/630_install_grub.sh#L61

test -d "$TARGET_FS_ROOT/boot/grub" || Error "Could not find directory /boot/grub."

that is meanwhile fixed, see below
https://github.com/rear/rear/issues/2337#issuecomment-593913279

I.e. it actually means Could not find directory /mnt/local/boot/grub
which is about the boot/grub directory within the restored target system
while your cd /boot/grub is within the currently running ReaR recovery system.

So it seems during your backup restore boot/grub was not restored
which is perhaps because boot/grub was not included in your backup?

jsmeix commented at 2020-03-03 11:53:

I fixed the misleading Error message via
https://github.com/rear/rear/commit/8e9a301850986b7dc4111ca5febd2ea4eb0472bb

moonSymph commented at 2020-03-04 01:48:

Hello Jsmeix,

Thanks for answering.. do i need to add any configuration to include /boot/grub on the newly mount point during the creation of ReaR iso?

I've attached the log which was created during recovery.
rear-localhost.log

jsmeix commented at 2020-03-06 10:56:

@moonSymph
what is in the ReaR recovery system below /mnt/local
after "rear recover" are the restored files from your backup.
When the boot/grub files are not there you likely do not
have them in your backup so you need to get them included
in your backup.

I am not a BACKUP=CDM user so that I cannot tell you what to do
or how to set up things to get the boot/grub files included in
your backup.

I use BACKUP=NETFS with the default tar backup program
and I have the boot/grub files included in my backup
(in my case I get boot/grub2 because on [open]SUSE
GRUB2 files are installed in grub2 sub directories while
on Red Hat GRUB2 files are installed in grub sub directories).

@DamaniN
could you have a look here?
Of course only as time permits.

DamaniN commented at 2020-03-06 17:13:

@moonSymph, How did you install ReaR? Was it from the OS package installer or did you clone the current version of this repo and use make install. ReaR 2.6 has not been released yet which is where CDM will be supported by the package installers. Instead you must clone this repo and then run make install.

From the logs it appears that ReaR is going directly from the restore phase to the finalize phase. With CDM support enabled it will pause and allow you to recover files from Rubrik CDM to /mnt/local. I see no CDM messages in the logs which leads me to believe that the wrong version of ReaR in installed.

moonSymph commented at 2020-03-06 17:54:

hello guys, I manage to solve this by using the rear from github.. by the way is there a specific script to extract the Linux server disk mapping and layout without having to make the iso? the iso size is pretty huge and if there is a lot of Linux server it gets bigger if I were to keep iso for each and every Linux host.. I was thinking to just extract the layout and using a golden image but edit the disk layout with the extracted disk layout as a reference..

jsmeix commented at 2020-03-09 11:14:

@DamaniN
thank you so much for your help!

A nice proof of
"given enough eyeballs, all bugs are shallow"
https://en.wikipedia.org/wiki/Linus%27s_law

@moonSymph
thank you for your prompt confirmation that it works for you.

jsmeix commented at 2020-03-09 11:27:

I am wondering if there might be a minor bug in ReaR
that it does not sufficiently check and error out
when an unsupported backup method is specified
but I cannot reproduce it.

For me with BACKUP=QQQ in my etc/rear/local.conf
rear mkrescue did not show a message or errors out
but rear mkbackup did error out with

ERROR: The BACKUP method 'QQQ' is not known to ReaR.

as it should in
usr/share/rear/backup/default/005_valid_backup_methods.sh
https://github.com/rear/rear/blob/master/usr/share/rear/backup/default/005_valid_backup_methods.sh

@DamaniN
might it somehow happen that with BACKUP=CDM
backup/default/005_valid_backup_methods.sh is not run
or its code does not error out?

jsmeix commented at 2020-03-09 11:30:

Hmmm... I think I found a generic issue:

For most external backup methods rear mkbackup is useless,
cf. the section "BACKUP SOFTWARE INTEGRATION" in man rear
https://github.com/rear/rear/blob/master/doc/rear.8.adoc
so for most external backup methods only rear mkrescue is used
and that does not check if a valid backup method is specified.

jsmeix commented at 2020-03-09 13:48:

At least for me moving
usr/share/rear/backup/default/005_valid_backup_methods.sh
to
usr/share/rear/prep/default/035_valid_backup_methods.sh
lets both rear mrbackup and rear mkrescue error out
when an unsupported backup method is specified.

jsmeix commented at 2020-03-09 14:25:

I think the issue is fixed via
https://github.com/rear/rear/commit/8eb72c1d845d2b91fa76f18d47c74454ed5432ec

@moonSymph
if you like to test our current master code with that fix
see the section
"Testing current ReaR upstream GitHub master code" in
https://en.opensuse.org/SDB:Disaster_Recovery

You would have to specify a wrong backup method
e.g. a typo of the value like BACKUP=CDN
to test if "rear mkrescue" errors out

Unfortunately a typo of the keyword like BAKCUP=CDM
cannot be recognized because default.conf sets

BACKUP=REQUESTRESTORE

https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L299


[Export of Github issue for rear/rear.]