#2560 Issue closed: Missing support for /dev/emcpower* devices: "No code has been generated to restore device pv:/dev/emcpowera2 (lvmdev)"

Labels: enhancement, needs sponsorship, won't fix / can't fix / obsolete, special hardware or VM

annonymous99999 opened issue at 2021-01-28 23:54:

Need a fix urgently please. I am hoping someone has seen this and has a fix.
/boot is /dev/emcpowera1 and root LVM2 is /dev/emcpowera2
No code has been generated to restore device pv:/dev/emcpowera2 (lvmdev)
Please add code /var/lib/rear/layout/diskrestore.sh
Boot from SAN with EMC powerpath on HP blades.
rear23a on SLES 12

gdha commented at 2021-01-29 08:04:

@annonymous99999 If you have a support contract with SuSe then you can open a tciket with them as the package rear23a is not owned by the upstream maintainers.

jsmeix commented at 2021-01-29 09:12:

@annonymous99999

first and foremost:
There is no such thing as "an urgent fix" for Relax-and-Recover,
see the section "Notes on the meaning of 'Relax' in 'Relax-and-Recover'" at
https://en.opensuse.org/SDB:Disaster_Recovery

Second:
When you click on the [New issue] button at
https://github.com/rear/rear/issues
you see this
https://raw.githubusercontent.com/rear/rear/master/.github/ISSUE_TEMPLATE.md
which describes how to submit issues so that we have at least a chance
to understand what goes on on your particular system with your particular hardware
in your particular environment which is a precondition to be able to help you.

Third:
See the section "SUSE support for Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery

Fourth:
In current ReaR upstream master code
I cannot find anything about emcpower

rear.github.master # find . -type f | xargs grep -i 'emc' | egrep -vi 'systemctl|EMC NetWorker|EMC Avamar|EMC Legato|scsi_dh_emc'
[no output]

This indicates that currenly ReaR does not support
that kind of devices /dev/emcpower*
so this issue seems to be about a new feature
"Support /dev/emcpower* devices".

annonymous99999 commented at 2021-01-29 11:08:

I assure you there is urgency in context of the need. I do have a support contract with SUSE and have talked to them, but I suspect this forum may have more knowledge of the rear code and timeliness. At least I am hoping :). PowerPath(emcpower devices) is the Dell/EMC version of multipath devices, example /dev/mpatha. That is why I am asking how to properly include them since the restore looks to be saying it did not record them. This is FCOE bare metal boot from SAN and we do have networker but that does not appear to be what the error is currently stuck on. I am hoping there is a simple way to include the emcpower devices during the mkbackup so the restore does not error. I am very new to this particular recovery product.
Also on a side note is there an option/flag to not rebuild the initramfs with Dracut at the end if I am restoring to the same exact system? That way I do not have to worry about the Powerpath modules being including or other such issues. Thank you very much for your replies and inputs.

jsmeix commented at 2021-01-29 12:43:

@annonymous99999

I did not question that the issue is urgent for you.
But you cannot enforce urgency on free software developers at ReaR upstream.

When you have a sufficient SUSE support contract for
SUSE Linux Enterprise High Availability Extension (SLE-HA)
(which is the only SUSE product/extension where SUSE officially supports ReaR)
then you should also (in addition to this issue at ReaR upstream)
file a separated issue at SUSE via your SUSE contact.
Depending on your SUSE support contract you may enforce urgency at SUSE.

But when current ReaR upstream does not support /dev/emcpower devices
then there is little hope things can be fixed in urgency mode in particular
when no such hardware is available to the developers who work on ReaR.
E..g. on my x86_64 homeoffice laptop I cannot reproduce anything
that depends on special hardware.

jsmeix commented at 2021-01-29 12:55:

Regarding rebuilding the initramfs.

During "rear recover" the initramfs is rebuilt after the backup was restored.
In the ReaR recovery system the target system where the backup gets restored to
is mounted at /mnt/local.
In the ReaR recovery system the initramfs is rebuilt via

chroot /mnt/local ... /sbin/mkinitrd

cf. in current ReaR upstream master code for x86 compatible architecture
usr/share/rear/finalize/SUSE_LINUX/i386/550_rebuild_initramfs.sh
https://github.com/rear/rear/blob/master/usr/share/rear/finalize/SUSE_LINUX/i386/550_rebuild_initramfs.sh

So during "rear recover" the initramfs is rebuilt inside the restored system
with the programs and files (also config files) as restored from the backup
so that the initramfs should be rebuilt same as when /sbin/mkinitrd
was called in the normal running system.

jsmeix commented at 2021-01-29 13:14:

@annonymous99999

regarding if there is a simple way to include the emcpower devices during the mkbackup:

You may run a command like

# find usr/share/rear/ -type f | xargs grep 'nvme'

which shows "the usual suspects" of the code places
where disk block device (base)names are normally handled, mainly in
usr/share/rear/layout/save/GNU/Linux/200_partition_layout.sh
plus some special case handling in the get_part_device_name_format() function
to guess a block device partition name from the block device disk name
(some have a p between the parent disk device and its partition devices)
and another special case handling for nvme in
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh

Perhaps - only a blind guess I never had anything to do
with /dev/emcpower devices or its corresponding hardware -
it is sufficient to add emcpower* to those usual code places.

Probably you need some trial and error attempts with "rear mkrescue"
(you do not need to run "rear mkbackup" for those tests)
until you get in your var/lib/rear/layout/disklayout.conf the needed
entries of your /dev/emcpower* devices so that "rear recover"
(now you need a "rear mkbackup" before) works in your case.

If you like to do that I would very much recommend to work
with current ReaR upstream master code because that is the only code
where we at ReaR upstream work with and where we would accept pull requests.

See the section "Testing current ReaR upstream GitHub master code" in
https://en.opensuse.org/SDB:Disaster_Recovery
and all the subsequent sections therein up to the section
"How to contribute to Relax-and-Recover"
how you could work yourself on ReaR.

annonymous99999 commented at 2021-01-29 15:17:

So there is no standard syntax to add to the file the error mentions to include those emcpower device files manually as the message eludes or options?
No code has been generated to restore device pv:/dev/emcpowera2 (lvmdev)
Please add code /var/lib/rear/layout/diskrestore.sh

jsmeix commented at 2021-02-05 14:19:

@annonymous99999
/var/lib/rear/layout/diskrestore.sh is a script that is generated
during "rear recover" primarily from the contents of
/var/lib/rear/layout/disklayout.conf

In exceptional/emergency cases you could manually enhance
the diskrestore.sh script as needed, cf. the section
"Restore to different hardware" - "The Ad-Hoc Way" in
https://github.com/rear/rear/blob/master/doc/user-guide/06-layout-configuration.adoc

But that is probably not what you like to do in case of emergency and time pressure
during a real disaster recovery.

So what needs to be done is to get the missing things automatically into
/var/lib/rear/layout/disklayout.conf which is created during "rear mkrescue"
so that later in case of emergency and time pressure during a real disaster recovery
"rear recover" could recreate the system fully automatically without labourious
and possibly erroneous manual interventions.

There is no standard how to get missing things into disklayout.conf.
This is manual trial-and-error adapting and enhancing the ReaR scripts
as I described it a little bit in
https://github.com/rear/rear/issues/2560#issuecomment-769797622

If /dev/emcpowera* devices need special setup actions to be recreated
then things get even more complicated because then the scripts that
run during "rear recover" would have to be adapted and enhanced
to provide support for /dev/emcpowera* device setup, cf. at
https://en.opensuse.org/SDB:Disaster_Recovery
the
"Essentials about disaster recovery with Relax-and-Recover presentation PDF"
and the
"Fundamentals about Relax-and-Recover presentation PDF"
which is a bit more specific for SLE12/15 in particular regarding
SUSE's default btrfs structure that is rather special, see also
https://en.opensuse.org/SDB:Disaster_Recovery#btrfs
and the section "Limitations with Btrfs" in
https://documentation.suse.com/sle-ha/12-SP5/html/SLE-HA-all/cha-ha-rear.html

By the way:
As long as you do not provide us (i.e. the ReaR upstream developers)
the information that I asked for in my
https://github.com/rear/rear/issues/2560#issuecomment-769680240
this ReaR upstream issue cannot move forward.

To make it more clear what information is needed
to analyze and debug a "rear recover" failure
I added an itemization that is about

To analyze and debug a "rear recover" failure the following information is mandatory

to the "Debugging issues with Relax-and-Recover" section in
https://en.opensuse.org/SDB:Disaster_Recovery

Alternatively we could stop here and continue only via the SUSE internal issue
that we have now for this issue but that would have the drawback that all other
ReaR upstream developers and users are excluded so there cannot be any help
from them - perhaps there are other ReaR users who have hardware with
/dev/emcpowera* devices who could contribute something helpful here?

jsmeix commented at 2021-02-23 12:57:

No info so it can't move forward.


[Export of Github issue for rear/rear.]