#2344 Issue closed: OUTPUT=USB on PPC64le PowerVM not bootable (with RHEL 7.7)¶
Labels: support / question, no-issue-activity
mmetts opened issue at 2020-03-18 19:37:¶
- ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.5-git.3693.da115e6.master / 2020-03-18 - OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat
/etc/os-release"):
RHEL 7.7 - ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
[root@crcossr2 crcossr2]# cat /etc/rear/local.conf
OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/mnt' '/gpfs' '/media' '/projects' '/var/tmp' '/var/crash')
- Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM
guest or PoverVM LPAR):
PowerVM - System architecture (x86 compatible or PPC64/PPC64LE or what exact
ARM device):
PPC64le - Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or
ELILO or Petitboot):
BIOS ( I think ) - Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or
multipath (DM or NVMe):
Local disk is source. Target is a USB stick. - Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT" or "lsblk" as makeshift):
[root@crcossr2 crcossr2]# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda disk 264.3G
|-/dev/sda1 /dev/sda1 /dev/sda part linux_raid_member 4M
| `-/dev/md127 /dev/md127 /dev/sda1 raid1 4M
|-/dev/sda2 /dev/sda2 /dev/sda part xfs 1G
`-/dev/sda3 /dev/sda3 /dev/sda part LVM2_member 263.3G
|-/dev/mapper/rhel-swap /dev/dm-3 /dev/sda3 lvm swap 4G
|-/dev/mapper/rhel-home /dev/dm-4 /dev/sda3 lvm xfs 209.3G
`-/dev/mapper/rhel-root /dev/dm-5 /dev/sda3 lvm xfs 50G
/dev/sdb /dev/sdb disk 264.3G
|-/dev/sdb1 /dev/sdb1 /dev/sdb part 4M
|-/dev/sdb2 /dev/sdb2 /dev/sdb part xfs 1G /boot
`-/dev/sdb3 /dev/sdb3 /dev/sdb part LVM2_member 263.3G
|-/dev/mapper/rhel00-root /dev/dm-0 /dev/sdb3 lvm xfs 50G /
|-/dev/mapper/rhel00-swap /dev/dm-1 /dev/sdb3 lvm swap 4G [SWAP]
`-/dev/mapper/rhel00-home /dev/dm-2 /dev/sdb3 lvm xfs 209.3G /home
/dev/sdc /dev/sdc usb disk 239G
`-/dev/sdc1 /dev/sdc1 /dev/sdc part ext3 239G /mnt/extusb
- Description of the issue (ideally so that others can reproduce
it):
I made a rescue + back up on a USB stick and then moved the stick to another machine to discover that the stick was not bootable. - Workaround, if any:
My objective is to create a clone of one of our systems so that I can reorganize the underlying hardware. The first step I've taken is to attempt to create a self-contained bootable copy of the system using REAR and then restore that into a newly configured LPAR on a different box but even though it says it's creating something bootable it isn't. What am I doing wrong? - Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
[root@crcossr2 /]# !987
TMPDIR=/mnt/rear/tmp rear -d mkbackup
Relax-and-Recover 2.5-git.3693.da115e6.master / 2020-03-18
Running rear mkbackup (PID 30537)
Using log file: /var/log/rear/rear-crcossr2.log
Using backup archive '/mnt/rear/tmp/rear.ubSlLS32FHSikLD/outputfs/rear/crcossr2/20200318.0948/backup.tar.gz'
Using autodetected kernel '/boot/vmlinuz-3.10.0-957.10.1.el7.ppc64le' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
Using guessed bootloader 'PPC' (found PPC PreP boot partition 'ID=0x41' on /dev/sdb)
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct ...
Creating recovery system root filesystem skeleton layout
Handling network interface 'bond0'
bond0 is a bond
bond0 has lower interface enP2p1s0
enP2p1s0 is a physical device
bond0 has lower interface enp1s0
enp1s0 is a physical device
Handled network interface 'bond0'
Copying logfile /var/log/rear/rear-crcossr2.log into initramfs as '/tmp/rear-crcossr2-partial-2020-03-18T09:48:32-0600.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/3.10.0-957.10.1.el7.ppc64le (MODULES contains 'all_modules')
Omit copying files in /lib*/firmware/ (FIRMWARE_FILES='no')
Skip copying broken symlink '/etc/mtab' target '/proc/43017/mounts' on /proc/ /sys/ /dev/ or /run/
Symlink '/usr/lib/modules/3.10.0-957.10.1.el7.ppc64le/build' -> '/usr/src/kernels/3.10.0-957.10.1.el7.ppc64le' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/3.10.0-957.10.1.el7.ppc64le' via the 'COPY_AS_IS' configuration variable.
Symlink '/usr/lib/modules/3.10.0-957.10.1.el7.ppc64le/source' -> '/usr/src/kernels/3.10.0-957.10.1.el7.ppc64le' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/3.10.0-957.10.1.el7.ppc64le' via the 'COPY_AS_IS' configuration variable.
Broken symlink '/usr/lib/modules/3.10.0-957.10.1.el7.ppc64le/weak-updates/tracedev.ko' in recovery system because 'readlink' cannot determine its link target
Testing that the recovery system in /mnt/rear/tmp/rear.ubSlLS32FHSikLD/rootfs contains a usable system
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (94964554 bytes) in 17 seconds
Making backup (using backup method NETFS)
Creating tar archive '/mnt/rear/tmp/rear.ubSlLS32FHSikLD/outputfs/rear/crcossr2/20200318.0948/backup.tar.gz'
Archived 12829 MiB [avg 7559 KiB/sec] OK
WARNING: tar ended with return code 1 and below output:
---snip---
tar: /var/ct/b5564e44-47f4-486f-a875-0a1e1c8a129d/soc/mc/RMIBM.MgmtDomainRM.0: socket ignored
tar: /var/ct/b5564e44-47f4-486f-a875-0a1e1c8a129d/soc/mc/RMIBM.HostRM.0: socket ignored
tar: /opt/IBM/zimon/data/1/data_000014.dat: file changed as we read it
----------
This means that files have been modified during the archiving
process. As a result the backup may not be completely consistent
or may not be a perfect copy of the system. Relax-and-Recover
will continue, however it is highly advisable to verify the
backup in order to be sure to safely recover this system.
Archived 12829 MiB in 1739 seconds [avg 7554 KiB/sec]
Exiting rear mkbackup (PID 30537) and its descendant processes ...
Running exit tasks
You should also rm -Rf /mnt/rear/tmp/rear.ubSlLS32FHSikLD
pcahyna commented at 2020-03-18 19:52:¶
@rmetrich is that supposed to work?
rmetrich commented at 2020-03-18 19:55:¶
I think the issue is with
tar: /opt/IBM/zimon/data/1/data_000014.dat: file changed as we read it
mmetts commented at 2020-03-18 19:56:¶
I thought so. I'm trying to make something that I can boot and then restore on different hardware w/o any network needed. It was my belief that what I put in /etc/rear/local.conf would give me a bootable USB but it definitely isn't. Even a bootable ISO that's not expecting NFS would be okay. I just need to be able to restore the system using something self-contained - i.e. - no network. Also, the hardware arrangement will be slightly different so it would good to be able to specify which disk the restore is going to.
mmetts commented at 2020-03-18 19:57:¶
How so @rmetrich ? that's just an incidental thing. Are you saying that that warning caused it to not create the bootable media? This is a live production system I'm working with. Is there a way to get it to ignore that and proceed?
mmetts commented at 2020-03-18 20:17:¶
For example, shouldn't there be a PreP partition here:
[root@crcossr2 extusb]# parted /dev/sdc u s p
Model: Samsung Flash Drive (scsi)
Disk /dev/sdc: 501253132s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 16384s 501252095s 501235712s primary ext3 boot
[root@crcossr2 extusb]#
rmetrich commented at 2020-03-18 20:54:¶
@schabrolles may help here, since he knows PPC64 well and also has access to hardware, something I don't.
mmetts commented at 2020-03-18 22:13:¶
thanks, @rmetrich hi @schabrolles I have a RHEL 7.6 PPCle server with a botched root volume group. the guys who built it were supposed to create a RAID1 disk in hardware or create an LVM RAID1 root vg. they did neither. so this is a production machine running on one drive with the other drive just sitting there. I've been urged not to attempt to turn the root vg into a LVM RAID1 vg as (I'm told) that's just not the Linux way and it's not reliable. the consequence is that I would like to take a backup of this server, wipe the drives, create a RAID1 volume in hardware and then restore the system. the stage I'm at is I'm just trying take a backup and then do a test restore into a test LPAR I have prepared on another Power8 box. we already tried NFS but that approach assumes you're restoring the real machine and I'd have to take the real machine off the network - and I don't want to do that just for a test restore. the root vg is too big to fit into a single 4 GB ISO so now I've tried the USB approach which suggests that it will create a bootable USB with the backup onboard but it won't boot and moreover the structure of the partitions on the device don't look (to me) like a PPC arrangement. please see above. would love to get your thoughts. I'm trying my best to test this fully on a test LPAR before actually committing to wiping the system. thanks.
pcahyna commented at 2020-03-18 23:43:¶
@mmetts if ReaR supported more than 4 GB ISO would it be an option for you to use it instead of USB?
mmetts commented at 2020-03-18 23:48:¶
yes. I'm actually trying that now with xorriso ... which as I understand it supports > 4 GB ISOs. this thing is about 13 GB so I'm not really sure what that'll mean but giving it a try. If SMS will boot from it and restore the system then cool. otherwise, the USB things seems like it's broken. the logs comments that sure make it sound like it's creating a bootable USB but it not a PPCle bootable in the end. maybe some steps are missing in USB mode because I does indeed create a PPCle bootable ISO. we've tried that with the NFS approach but we're prepared to let the test LPAR take over the network identity of the real system so we stopped there.
pcahyna commented at 2020-03-18 23:50:¶
@mmetts we added xorriso to RHEL 7.7 exactly for that. https://bugzilla.redhat.com/show_bug.cgi?id=1462189 . But it does not work on ppc64le: https://bugzilla.redhat.com/show_bug.cgi?id=1726043 . I suppose fixing this would be possible.
mmetts commented at 2020-03-18 23:52:¶
how? I really need this to work. btw, I'm running RHEL 7.6 PPCle and RearR 1.25 at the moment.
pcahyna commented at 2020-03-18 23:57:¶
I mentioned that in https://bugzilla.redhat.com/show_bug.cgi?id=1726043#c2 ... mkisofs invocation on ppc64le is different and I think it is missing some option.
pcahyna commented at 2020-03-19 00:03:¶
the relevant file is /usr/share/rear/output/ISO/Linux-ppc64le/820_create_iso_image.sh
mmetts commented at 2020-03-19 00:04:¶
ok. so I just need to add '-iso-level 3' to the invocation of mkisofs in that file?
pcahyna commented at 2020-03-19 00:06:¶
I suppose so, but I haven't tested it. Also, you need to have xorriso installed, IIRC ReaR uses it automatically, it prefers it over mkisofs/genisoimage.
mmetts commented at 2020-03-19 00:06:¶
yes. have installed.
mmetts commented at 2020-03-19 00:11:¶
okay. I have made this change and I'm giving it a try. I should know something in about 25 minutes or so.
pcahyna commented at 2020-03-19 00:14:¶
that bug manifested already when creating an image, but if that succeeds, try booting it and recovering as well in case there are other unrelated problems.
mmetts commented at 2020-03-19 00:17:¶
absolutely. my plan is to test this completely on before using it for my original purpose described above. Thanks for your help thus far.
pcahyna commented at 2020-03-19 00:29:¶
@mmetts yw, I won't help you more today, I am going to sleep
schabrolles commented at 2020-03-19 07:58:¶
@mmetts It will be difficult for me to help you as we are forced to work from home due to covid-19, so I can't have physical access to systems. On top of that I'm working on client engagement which use almost all my time...
Anyway, correct me if I'm wrong:
- you want to be able to test the backup of you system on a new created LPAR.
- you would like to have USB or BIG ISO to test it because you have to disconnect the new LPAR from the network to avoid IP conflict during recovery (that's why you avoid NFS)
In the different test I made, NFS is the best choice... but I understand your concerns here... did you try the rear IP migration process described in the faq (http://relax-and-recover.org/documentation/faq). This should allow to recover a system with a different IP address and then may be helpful in your case (if I well understood your problem)
I personally never try it recently. @jsmeix do you know if this is still working with the current rear version ?
jsmeix commented at 2020-03-19 11:40:¶
@mmetts
I did not yet read the details here so for now only some general infos:
Because I notice here things like "restore on different hardware"
and "changing IP address" you may have a look at
https://github.com/rear/rear/issues/2310#issuecomment-595149640
In general regarding using a (USB) disk for ReaR see
https://github.com/rear/rear/issues/1854
I know nothing about booting on POWER architecture
in particular I don't know if POWER systems can boot
"as usual" from USB, cf. the section
"Non PC compatible architectures" in
https://en.opensuse.org/SDB:Disaster_Recovery
@schabrolles
I am afraid I do not understand the "IP Migration" section.
But it does not mean anything when I do not understand networking
stuff
because I am not at all a networking expert.
In general regarding "changing IP address" and ReaR
there are two different things:
Specifying the IP address of the ReaR recovery system
versus
changing the IP address of the recreated target system
compared to what it was on the original system.
By default the IP address of the ReaR recovery system
is the same as the IP address of the original system.
This leads to conflicts when the ReaR recovery system
is booted (e.g. for a test) while the original system is still running.
Therefore I use always DHCP for my ReaR recovery systems
via USE_DHCLIENT=yes cf. the dhcp related entries in default.conf
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L2583
Alternatively one can specify the IP address of the ReaR recovery
system
via kernel command line parameters as described in the section
"RESCUE IMAGE KERNEL COMMAND LINE OPTIONS" in "man rear"
https://github.com/rear/rear/blob/master/doc/rear.8.adoc
Regarding changing the IP address of the recreated target system
compared to what it was on the original system we have currently
https://github.com/rear/rear/issues/2310
and
https://github.com/rear/rear/issues/2312
Finally:
I am currently also forced to work from home and here I have only
a rather limited computer where I cannot run virtual machines
in a reasonably useful way for ReaR so that currently I cannot
reproduce anything with ReaR from home.
jsmeix commented at 2020-03-19 11:57:¶
@mmetts
an offhanded general thought about "BIG ISO" and things like that:
I wonder why your things get so big.
I guess because you have all your data in one same backup
and you have that backup in the ISO or on the USB medium.
If yes, why do you need to have all your data in one same backup
when all what ReaR should actually do (and what it is meant to do)
is to recreate the basic operating system (in your case RHEL 7.7)
but not application data like huge databases and things like that, cf.
https://en.opensuse.org/SDB:Disaster_Recovery#Basics
and see also
https://github.com/rear/rear/blob/master/doc/user-guide/11-multiple-backups.adoc
At least for your initial tests I recommend to keep things small,
cf. the sections
"Relax-and-Recover versus backup and restore"
and
"First steps with Relax-and-Recover" in
https://en.opensuse.org/SDB:Disaster_Recovery
First try to get your basic operating system recreated.
Then you can as a second step think about how to
move the "big data" into the new recreated system.
pcahyna commented at 2020-03-19 12:31:¶
@jsmeix @mmetts good remark! IMO the warnings from tar about
tar: /var/ct/b5564e44-47f4-486f-a875-0a1e1c8a129d/soc/mc/RMIBM.MgmtDomainRM.0: socket ignored
tar: /var/ct/b5564e44-47f4-486f-a875-0a1e1c8a129d/soc/mc/RMIBM.HostRM.0: socket ignored
tar: /opt/IBM/zimon/data/1/data_000014.dat: file changed as we read it
indicate that there are files being included in the backup that are not
part of the base OS. Especially the last one means that the backup of
/opt/IBM/zimon/data/1/data_000014.dat will probably be not usable and
you need some other way to migrate those data. This means that you could
exclude those files from the backup completely and save some space.
jsmeix commented at 2020-03-19 12:53:¶
@mmetts
in general regarding "to specify which disk the restore is going to"
see
https://github.com/rear/rear/blob/master/doc/user-guide/06-layout-configuration.adoc
What "rear recover" does by default is to recreate the disk layout
so that on the replacement hardware lsblk should show
basically the same as it was on the original system - except
possibly changed kernel device names in particular for higher
level storage objects where their kernel block device nodes
are not stable, e.g. kernel block device nodes for LVM LVs, cf.
https://github.com/rear/rear/pull/2291#issuecomment-567933705
When you need to change the disk layout on the recreated system
compared to what it was on the original system you must manually
adapt your var/lib/rear/layout/disklayout.conf file which could be a
rather laborious process when you need to do major changes.
See in particular the MIGRATION_MODE section in default.conf
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L351
and the subsequent section about
"Resizing partitions in MIGRATION_MODE during 'rear recover'"
which contains in particular this small little sentence
"This does not resize volumes on top of the affected partitions"
which means that LVM things are not automatically resized.
So things are probably much easier when your replacement hardware
has same number of disks ideally with each one exact same size
or at most a bit bigger disk sizes,
see default.conf (cf. above) how the autoresize things behave.
mmetts commented at 2020-03-19 15:15:¶
I have successfully restored a "big ISO" into a test LPAR with no network connection and limited resources. the ISO is about 10 GB. in the next phase of testing, I will expand the resources dedicated to the test LPAR (memory and CPU) to a reasonable level, add 2 NICs, temporarily shutdown the source production LPAR (it's part of a redundant pair) and see if what I've created is sound/functional enough to act in it's place. it at least boots consistently. thanks to all of you for you help thus far. I'll keep going and report back.
pcahyna commented at 2020-03-19 17:01:¶
@mmetts wow great, thanks for the report, can you please send a patch so we can see the exact change you made?
mmetts commented at 2020-03-19 19:27:¶
I'm not sure how to write a patch. I changed line 20 of this file /usr/share/rear/output/ISO/Linux-ppc64le/820_create_iso_image.sh from this:
$ISO_MKISOFS_BIN $v
$ISO_MKISOFS_OPTS -o "$ISO_DIR/$ISO_PREFIX.iso" \
-U $chrp_boot_option -R -J -volid "$ISO_VOLID" -v -graft-points \
"${ISO_FILES[@]}" >&2
to this:
$ISO_MKISOFS_BIN $v
$ISO_MKISOFS_OPTS -o "$ISO_DIR/$ISO_PREFIX.iso" -iso-level 3\
-U $chrp_boot_option -R -J -volid "$ISO_VOLID" -v -graft-points \
"${ISO_FILES[@]}" >&2
I am running RHEL 7.6 PPCle. I clone the current version of ReaR, created an RPM and installed it. I then installed xorriso from EPEL. This is what my /etc/rear/local.conf looks like:
OUTPUT=ISO
OUTPUT_URL=null
BACKUP=NETFS
BACKUP_URL="iso:///backup"
ISO_DIR="/mnt/rear"
BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/mnt' '/gpfs' '/media' '/projects' '/var/tmp' '/var/crash')
#NETFS_KEEP_OLD_BACKUP_COPY=
This was my invocation:
# TMPDIR=/mnt/rear/tmp rear -d mkbackup
I messed with TMPDIR because the LPAR does not have enough free space in the root file system to hold the tar file.
And that's pretty much it.
pcahyna commented at 2020-03-20 10:12:¶
@mmetts thanks!
jsmeix commented at 2020-03-20 11:34:¶
@mmetts
as far as I see all what you changed in
usr/share/rear/output/ISO/Linux-ppc64le/820_create_iso_image.sh
was to add the option -iso-level 3 directly in the script.
But the code
https://github.com/rear/rear/blob/master/usr/share/rear/output/ISO/Linux-ppc64le/820_create_iso_image.sh#L20
contains the ISO_MKISOFS_OPTS config variable
so that it should have also worked when you specifiy its value
as you need it in your particular case in your loval.conf like
ISO_MKISOFS_OPTS="-iso-level 3"
see the OUTPUT=ISO section in default.conf
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L616
therein in particular the ISO_MKISOFS_BIN and ISO_MKISOFS_OPTS
variables.
A general side note regarding xorriso cf.
https://github.com/rear/rear/issues/1360
Another issue I found that might be related is
https://github.com/rear/rear/issues/2243
therein I noticed in particular
https://github.com/rear/rear/issues/2243#issuecomment-537354570
And regarding USB and ISO at least on x86 architecture we have
https://github.com/rear/rear/issues/2210
Regarding
"I clone the current version of ReaR, created an RPM and installed it"
see also the section
"Testing current ReaR upstream GitHub master code" in
https://en.opensuse.org/SDB:Disaster_Recovery
I think this way it is simpler at least during testing phase.
pcahyna commented at 2020-03-20 23:17:¶
as far as I see all what you changed in
usr/share/rear/output/ISO/Linux-ppc64le/820_create_iso_image.sh
was to add the option-iso-level 3directly in the script.
But the code
https://github.com/rear/rear/blob/master/usr/share/rear/output/ISO/Linux-ppc64le/820_create_iso_image.sh#L20
contains theISO_MKISOFS_OPTSconfig variable
so that it should have also worked when you specifiy its value
as you need it in your particular case in your loval.conf likeISO_MKISOFS_OPTS="-iso-level 3"see the
OUTPUT=ISOsection in default.conf
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L616
therein in particular the ISO_MKISOFS_BIN and ISO_MKISOFS_OPTS
variables.
@jsmeix thanks, I would consider this only a workaround. The i386 counterpart has the option: https://github.com/rear/rear/blob/9795b703a29be04bd575d411e758fc4d8b417b43/usr/share/rear/output/ISO/Linux-i386/820_create_iso_image.sh#L33
so the ppc64le script should have it as well, for symmetry.
Thanks for the reference to other issues, #2243 is helpful, I was not sure whether one can simply dd the image to a disk on PowerPC and boot from it.
mmetts commented at 2020-03-20 23:53:¶
I have successfully, taken an ISO image of another PPC64le machine (the ~identical twin of the one I started this effort talking about) but this time I just specified
ISO_MKISOFS_OPTS="-iso-level 3"
in /etc/rear/local.conf ... and ISO creation worked just fine. so no need for my earlier hack. BTW, at the risk of bringing up a different topic, when I did my test restore of this machine, I encountered a script failure due to the fact that it found physical volumes with identical UUIDs. this was the case because the volume being presented to the LPAR is on SAN and so the ReaR script is puking because it doesn't understand a multi-path device. essentially, it fails to create the RHEL volume group because when an attempt is made to add PVs to it, it throws an error saying there are duplicates. I got around this by hacking the script putting these 3 lines at the top:
echo 1 > /sys/block/sdb/device/delete
echo 1 > /sys/block/sdc/device/delete
echo 1 > /sys/block/sdd/device/delete
... and then it worked (I was installing on /dev/sda). I realize this might seem whacky but I think it would be handy if the ReaR restore functions could tolerate a multi-path device on the target machine even if they weren't in use on the source. otherwise, you end up with this conundrum which is really a non-problem. i.e. - there aren't multiple disks with the same UUID ... rather it's the same disk seen through 4 paths. anyway, I'm a very happy ReaR customer so please don't take these as complaints. ReaR is on the verge of allowing us to correct a totally egregious error committed by an erstwhile vendor of ours. cheers!
schabrolles commented at 2020-03-21 08:03:¶
@mmetts
You should add the following into your local.conf
AUTOEXCLUDE_MULTIPATH=n
BOOT_OVER_SAN=y
There is some configuration example here: https://www.slideshare.net/sebastienchabrolles/relax-and-recover-on-power-updated-052017
pcahyna commented at 2020-03-21 11:10:¶
and ISO creation worked just fine. so no need for my earlier hack.
I still intend to add it to the script, so that it works for everyone without having to specify it in the configuration. Unless there is a potential problem with it - @schabrolles @rmetrich is there a risk that OpenFirmware would not understand DVDs created with this option?
schabrolles commented at 2020-03-21 11:40:¶
@pcahyna, Honestly, I don't know ... but I would say if it works fo @mmetts, then I think it is OK and means that OpenFirmware is able to handle it ... As it is a ISO image used as VirtualDVD there is no need to limit it to 4GB because we don't have to burn it on real physical media ...
pcahyna commented at 2020-03-21 12:09:¶
@schabrolles well anyway the limit applies to the size of files in the filesystem, not to the filesystem itself, IIUC. (The big file in this case is backup.tar.gz)
mmetts commented at 2020-03-21 16:09:¶
thanks @schabrolles ! I'll likely give those options a try. there has been a super helpful discussion. I really appreciate everyones comments.
pcahyna commented at 2020-03-21 20:58:¶
@mmetts this is POWER7 ? Or POWER8 ?
mmetts commented at 2020-03-21 22:04:¶
Power8. Source systems are dedicated LPARs (i.e. - no VIOs) on a pair of IBM S822s. the target test LPAR is on an IBM 824. the source systems each have a pair of in-frame SAS drives with one dangling doing nothing in ... hence the reason for this exercise ... we're going to wipe them one at a time creating a hardware-based RAID1 device and then restore. The test LPAR has one 300 GB LUN on one of our storage devices since we didn't have any in-frame disk space on the S824 to provision for testing.
jsmeix commented at 2020-03-23 10:23:¶
@pcahyna
regarding your
https://github.com/rear/rear/issues/2344#issuecomment-601949828
I did
https://github.com/rear/rear/pull/2346
jsmeix commented at 2020-03-23 10:47:¶
A general comment regarding
https://github.com/rear/rear/issues/2344#issuecomment-602032249
I guess nowadays ISO images are in most cases no longer
burnt on physical medium but used as a virtual medium
so the default ReaR options should be for the
usual "ISO used as virtual medium" use case
BUT
in general I do not like hardcoded options in the scripts
except standard options that always work reliably
because editing scripts is not the preferred way
how users should set up their ReaR things
SO
in general default options should be in default.conf
where the user is expected to recognize them
and where he could more easily adjust them if needed
(compared to editing scripts).
The question for -iso-level 3 is whether or not that is
a standard option that always works reliably.
The next question is what "always works reliably"
should mean regarding the particular -iso-level 3 option:
I think it does not belong to ReaR to guarantee
that its ISO image can be burnt on a physical medium.
I think it is the user's task to check whether or not
ReaR's ISO image can be burnt on his particular physical medium
because ReaR cannot know what the user's particular physical medium
will be e.g. a traditional CD or a DVD or something more modern.
I think in particular it is the user's task to check whether or not
ReaR's ISO image is too big to be burnt on his particular physical
medium.
pcahyna commented at 2020-03-23 10:59:¶
Is -iso-level 3 in any way related to the ability to burn the image on
a medium? The option overcomes the limit which is there for single
files, not for the whole image IIUC, and maximum DVD size is usually
4.70 GB, so there are situations where one would benefit from the option
even when using a physical DVD. So I believe that the question whether
ReaR should guarantee that the ISO image can be burnt on a DVD is mostly
unrelated to the topic of -iso-level 3.
Moreover, from this conversation it appears that an ISO on ppc64 can be used for hard disks and USB media, not only optical media. Which supports the view that "it does not belong to ReaR to guarantee that its ISO image can be burnt on a physical medium".
To enforce that the image fits onto a medium, ISO_MAX_SIZE seems
better suited.
jsmeix commented at 2020-03-23 12:02:¶
@pcahyna
I played around with ISO_MAX_SIZE and found its implementation
seems to be somewhat "quick and ditry hack style", see the code for it
in
https://github.com/rear/rear/blob/master/usr/share/rear/backup/NETFS/default/500_make_backup.sh#L56
which means that for ISO_MAX_SIZE <= 200 things behave really
unexpected
but also for ISO_MAX_SIZE > 200 things do not work exactly as
expected.
I did "rear mkbackup" with ISO_MAX_SIZE=100 and got a 232M first ISO.
The reason is that actually ISO_MAX_SIZE does not implement
the maximum size of the ISO.
Instead it implements the size of the chunks when a backup.tar.gz is
split
cf.
https://github.com/rear/rear/pull/287
and
https://github.com/rear/rear/pull/292
The first ISO contains additionally the recovery system kernel and
initrd
plus the ISO bootloader stuff so it gets bigger than ISO_MAX_SIZE.
When I do "rear mkrescue" the ISO is 131M.
This matches what I get for "rear mkbackup" with ISO_MAX_SIZE=100
because 131M recovery system + 100M backup chunk results 232M.
In general related to that is
https://github.com/rear/rear/issues/2265
mmetts commented at 2020-03-23 14:18:¶
I just want to reiterate that this got started when I tried to create a bootable USB device for PPC64le but could not and not having or wanting NFS as an option needed for a "big ISO" to work for me since I was running out of handy options. I think there still might be something to resolve WRT creating of bootable USB devices of any size for PPC64le. Thanks.
jsmeix commented at 2020-03-23 14:48:¶
As a first step regarding
https://github.com/rear/rear/issues/2344#issuecomment-602550299
I explained in more detail in default.conf how ISO_MAX_SIZE actually
works
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L644
pcahyna commented at 2020-03-23 16:41:¶
@jsmeix sorry, I did not realize the limitations of ISO_MAX_SIZE.
@mmetts
I just want to reiterate that this got started when I tried to create a bootable USB device for PPC64le but could not
yes, I realize we are hijacking an issue which was intended for something else.
mmetts commented at 2020-03-23 16:46:¶
no worries, I'm as much to blame for changing the subject as anyone since I just wanted to get my work done but it seems that the issue itself remains.
jsmeix commented at 2020-03-23 19:18:¶
One more of the "hijacking this issue" things resulted in my
https://github.com/rear/rear/pull/2347
@mmetts
don't worry about us "hijacking your issue".
I appreciate it that by the way your issue made us aware
of some other minor things that do not work good enough
so that we could "just improve them by the way".
jsmeix commented at 2020-03-27 13:50:¶
@mmetts
regarding your
"there still might be something to resolve WRT creating
of bootable USB devices of any size for PPC64le" in
https://github.com/rear/rear/issues/2344#issuecomment-602624831
and
"it seems that the issue itself remains" in
https://github.com/rear/rear/issues/2344#issuecomment-602721504
I did
https://github.com/rear/rear/issues/2348
Please have a look there if I described the issue correctly.
I am not a PPC user so I may misunderstand things.
github-actions commented at 2020-06-26 01:39:¶
Stale issue message
[Export of Github issue for rear/rear.]