#1868 Issue closed: OUTPUT=OBDR support missing for PPC64LE (seems to be supported for PPC64)

Labels: enhancement, bug, fixed / solved / done

jsmeix opened issue at 2018-07-17 09:38:

We (i.e. SUSE) got an internal issue report
from a SUSE partner who tested SLES12
that OBDR support is missing for PPC64LE
while it seems to be supported for PPC64.

  • ReaR version ("/usr/sbin/rear -V"):
    I simulated it with current GitHub master code

  • ReaR configuration files ("cat /etc/rear/site.conf" or "cat /etc/rear/local.conf"):
    This is what our SUSE SLES12 partner reported that he uses:

# /etc/rear/local.conf
BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
BOOT_OVER_SAN=Y
AUTOEXCLUDE_MULTIPATH=n
ISO_MKISOFS_BIN=/usr/bin/mkisofs
# /etc/rear/local.conf
BACKUP=NETFS
OUTPUT=OBDR
OUTPUT_URL=null
TAPE_DEVICE=/dev/nst0
BACKUP_URL=tape:///dev/nst0
BOOT_OVER_SAN=Y
AUTOEXCLUDE_MULTIPATH=n
  • System architecture (x86 compatible or POWER PPC64/PPC64LE or what excat ARM device):
    Our SUSE SLES12 partner reported that he uses PPC64LE

  • Brief description of the issue:
    I neither have any tape device nor can I use OBDR
    so that I cannot test or reproduce anything that is specific for OBDR tape
    but I can - to some degree - simulate something
    as long as no real OBDR tape device is needed.

On my x86_64 system I use this /etc/rear/local.conf

OUTPUT=OBDR
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://10.160.4.244/nfs

On my x86_64 system I get those achitecture specific scripts

# usr/sbin/rear -s mkrescue | grep Linux-i386
Source conf/Linux-i386.conf
Source prep/OBDR/Linux-i386/330_find_isolinux.sh
Source pack/Linux-i386/300_copy_kernel.sh
Source output/OBDR/Linux-i386/300_create_isolinux.sh
Source output/OBDR/Linux-i386/800_create_isofs.sh

For POWER achitecture and OBDR there are those
achitecture specific scripts

# find usr/share/rear/ | grep OBDR | grep ppc
usr/share/rear/output/OBDR/Linux-ppc64
usr/share/rear/output/OBDR/Linux-ppc64/800_create_isofs.sh
usr/share/rear/output/OBDR/Linux-ppc64/300_create_yaboot.sh

There are no ppc64le achitecture specific scripts for OBDR
i.e. scripts with a /Linux-ppc64le/ sub directory
so that OBDR support is missing for PPC64LE.

  • Work-around, if any:
    Copy the scripts under Linux-ppc64 to Linux-ppc64le.

I think the real fix is to link those scripts because it seems
the Linux-ppc64 scripts work without changes also in Linux-ppc64le
but I cannot test or reproduce anything here (see above).

schabrolles commented at 2018-07-17 14:59:

@jsmeix, I do not have tapes connected to my Power for testing...
I had a quick look to the scripts and think we should not copy usr/share/rear/output/OBDR/Linux-ppc64 to usr/share/rear/output/OBDR/Linux-ppc64le as ppc64 system use yaboot bootloader versus grub2 for ppc64le.

I suggest to reuse script from ISO (ppc64le) to create OBDR (ppc64le) scripts:

usr/share/rear/output/ISO/Linux-ppc64le/300_create_grub2.sh
usr/share/rear/output/ISO/Linux-ppc64le/800_create_isofs.sh

What do you think ?

jsmeix commented at 2018-07-18 09:16:

@schabrolles
thank you very much!
It helps so much when a POWER user (pun intended ;-) has a look.

I fear our SUSE partner may have only tested that "rear mkrescue/mkbackup"
runs to the end without an error when "just copying" those scripts
but that he may not have verified that recovery actually works
(I got no exact information what he actually did).

On the other hand I noticed in a SUSE internal feature request
that was about providing "rear on power (SLE12)"
a comment that states

there's no yaboot support needed - SLE12 SP2 is grub2 only.

which may indicate that "just copying" some scripts from PPC64
could make things work on PPC64LE provided PPC64 and PPC64LE
actually use both the same bootloader GRUB2.

But I am not a POWER user so that I do not know
what bootloader is actually used where.

gdha commented at 2018-07-18 09:56:

Wow is OBDR still used these days? I haven't used it anymore since 2008 (when I wrote the code for x86)...

jsmeix commented at 2018-07-18 10:47:

@gdha
I only know that OBDR is still used these days.
I assume it is used on bigger systems in business/enterprise environments.
Personally I don't know how OBDR is actually used nowadays.
I was wondering if real tape media are still used and asked someone
who told me that real tape devices are still used for backups because
tape media are still the cheapest way to keep backups for a long time.

I guess @schabrolles at IBM might have a better chance to get in contact with
someone at IBM who actually knows how OBDR is still used these days ;-)

jsmeix commented at 2018-07-18 13:02:

@schabrolles
regarding using
usr/share/rear/output/ISO/Linux-ppc64le/800_create_isofs.sh
for OBDR on PPC64LE:

Because
usr/share/rear/output/OBDR/Linux-ppc64/800_create_isofs.sh
is a symbolic link to
usr/share/rear/output/ISO/Linux-ppc64/800_create_isofs.sh
which is again a symbolic link to
usr/share/rear/output/ISO/Linux-ppc64le/800_create_isofs.sh
I assume
this part works when I create another new symbolic link
usr/share/rear/output/OBDR/Linux-ppc64le/800_create_isofs.sh
that also points to
usr/share/rear/output/ISO/Linux-ppc64le/800_create_isofs.sh

schabrolles commented at 2018-07-18 13:12:

Yes, that should work !

jsmeix commented at 2018-07-18 13:49:

Regardless that I neither have a tape device nor can I use OBDR
I did https://github.com/rear/rear/pull/1873
as a basically blind initial tentative attempt to support OBDR on ppc64le.

jsmeix commented at 2018-07-18 13:57:

@schabrolles
in usr/share/rear/output/ISO/Linux-ppc64le/300_create_grub2.sh I noticed

ISO_FILES=( "${ISO_FILES[@]}" boot=boot ppc=ppc )

I wonder what that boot=boot ppc=ppc stuff is in ISO_FILES
because default.conf reads

# Which files to include in the ISO image:
ISO_FILES=()

but boot=boot ppc=ppc are likely no files.

Do you perhaps know or could you perhaps imagine what that is?

According to
git log -p --follow usr/share/rear/output/ISO/Linux-ppc64le/300_create_grub2.sh
this code was created via
https://github.com/rear/rear/commit/5d4af5137155b052ec80f326f98e095d65059119
perhaps Masanori Mitsugi is still at IBM and you could ask him
how that code is meant to work?

schabrolles commented at 2018-07-18 14:36:

@jsmeix, I have no idea sorry ... I've sended a mail to Masanori... I hope to get an answer tomorrow.

schabrolles commented at 2018-07-19 09:54:

@jsmeix,

Masanori told me he just replicate the script from ppc64 (yaboot) https://github.com/rear/rear/blame/3df109e9eeea908c469ba673f1945203c1665623/usr/share/rear/output/ISO/Linux-ppc64/300_create_yaboot.sh#L67

git blame said that @gdha added this line 9 years ago ;-) ... we should have an explanation if he can remember...

gdha commented at 2018-07-19 11:27:

@schabrolles @jsmeix That particular code was received from @mmitsugi himself if I'm not mistaken (I never had a ppc64(le) system)

jsmeix commented at 2018-07-19 11:36:

With https://github.com/rear/rear/pull/1873 merged
we have now initial tentative support for OBDR on PPC64LE
so that now users who have real OBDR hardware on PPC64LE
can try out our ReaR GitHub master code and test
if now OBDR on PPC64LE perhaps already works.

Even if it does not yet work the current state should be
at least a starting point to adapt and enhance ReaR
get OBDR supported also on PPC64LE.

I set the state here to "waiting for info" because I like
to let our SUSE partner who tests OBDR on PPC64LE on SLES12
try out our ReaR GitHub master code on his real OBDR hardware.

jsmeix commented at 2018-07-20 12:57:

I learned that I will not get any info if OBDR on PPC64LE works
on real OBDR hardware with our ReaR GitHub master code.

As far as I understand it the reason behind is basically the same
as what I think is the ultimate root cause behind 90% of bugs in software:
RFC 1925 item (6a):

It is always possible to add another level of indirection.

In this case our SUSE partner who reported the issue to us
is not the one who actually has the issue - i.e. our SUSE partner
also does not have real OBDR hardware to test it - but the one who
actually has the issue is another entity at another level of indirection
that is somehow hidden behind or obscured by our SUSE partner :-(

For the no fun of it
how that "Chinese whispers" game actually happens:

(0.)
The one who actually has the issue reports it to our SUSE partner.

(1. level of indirection)
Our SUSE partner passes the word to our SUSE support people.

(2. level of indirection)
Our SUSE support people passes the passed word to me.

(3. level of indirection)
I pass the passed passed word to ReaR upstream (this issue).

At ReaR upstream we cobble together a tentative fix and that's it.
Accordingly I close this issue (I hope it is "fixed/solved/done").

Nevertheless:
I wish you all at ReaR upstream a nice weekend!

PS:
If by accident the one who actually has the issue is listening here,
please - for heaven's sake (and even more for your own sake) -
speak up here so that we can get this issue properly solved.
Many thanks in advance.

pcahyna commented at 2021-05-21 19:08:

It appears that OBDR on ppc64/ppc64le has been broken since ReaR 2.2 - PR #1383, commit 4ef0f30156f0afea4a02d12f40c2c9d18cbe5e43 and 3658b799e404047ec7c443b60018b4f098d3d8c4. (And on PC since 1.15 - commit 41efc97eb7141c30455df45a871b98cd08e09fa7. Maybe ia64 could work...)


[Export of Github issue for rear/rear.]