#1767 Issue `closed`: trying to exclude other mount points and swap from rear recovery¶

Labels: fixed / solved / done, minor bug

pspatola101 opened issue at 2018-04-03 18:50:¶

Relax-and-Recover (ReaR) Issue Template¶

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

rear version (/usr/sbin/rear -V):
Relax-and-Recover 2.3-git.2932.d6865ed.master.changed / 2018-03-23
OS version (cat /etc/rear/os.conf or lsb_release -a):
LSB Version: :core-4.1-noarch:core-4.1-ppc64le
Distributor ID: RedHatEnterpriseServer
Description: Red Hat Enterprise Linux Server release 7.3 (Maipo)
Release: 7.3
Codename: Maipo
rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):

OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL="nfs://172.19.5.99/stuff/unix/iowa"
SSH_ROOT_PASSWORD="reardemo"
BOOT_FROM_SAN=y
AUTOEXCLUDE_MULTIPATH=n
REAR_INITRD_COMPRESSION=lzma
USE_STATIC_NETWORKING=y
ONLY_INCLUDE_VG=( "rhel_ssstfs03" )

Are you using legacy BIOS or UEFI boot?
PPC PRep Boot
Brief description of the issue:
Even though Im using the ONLY_INCLUDE_VG option the disklayout.conf is included the swap lvol on another vg. Im also looking to exclude any disks managed by gpfs software.

[root@ssstfs03 layout]# cat disklayout.conf
#lvmdev /dev/datavg /dev/mapper/mpathbe Kbe3rY-2Lhl-IYYm-IsKP-grv8-GMQB-UUV41T 209715200
lvmdev /dev/rhel_ssstfs03 /dev/mapper/mpatha3 aTXusm-kzGQ-14r7-myOi-mNsj-Iar7-pxVHKj 41959424
#lvmdev /dev/vgswap /dev/mapper/mpathbd AcAIgK-g0S4-wEav-bbQB-KLka-q0z1-FENNKb 52428800
#lvmgrp /dev/datavg 4096 25599 104853504
lvmgrp /dev/rhel_ssstfs03 4096 5121 20975616
#lvmgrp /dev/vgswap 4096 6399 26210304
#lvmvol /dev/datavg lvzimon 24320 199229440
lvmvol /dev/rhel_ssstfs03 root 256 2097152
lvmvol /dev/rhel_ssstfs03 usr 1024 8388608
lvmvol /dev/rhel_ssstfs03 opt 1024 8388608
lvmvol /dev/rhel_ssstfs03 home 512 4194304
lvmvol /dev/rhel_ssstfs03 tmp 512 4194304
lvmvol /dev/rhel_ssstfs03 var 512 4194304
lvmvol /dev/rhel_ssstfs03 var_log 768 6291456
lvmvol /dev/rhel_ssstfs03 swap 512 4194304
#lvmvol /dev/vgswap swapvol02 512 4194304
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
#fs /dev/mapper/datavg-lvzimon /opt/IBM/zimon/data xfs uuid=68029364-d304-4a2e-b994-44213dc7bc0d label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/mpatha2 /boot xfs uuid=3df7e785-acac-42b4-b35b-73568426ad4e label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/rhel_ssstfs03-home /home xfs uuid=5af3aaa8-7cfd-448b-a112-2ac933f93347 label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/rhel_ssstfs03-opt /opt xfs uuid=e6ca811a-f69c-45b3-b99c-3c814d1c8b28 label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/rhel_ssstfs03-root / xfs uuid=665632df-0cc4-46d6-9fea-a0d05a81aa7a label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/rhel_ssstfs03-tmp /tmp xfs uuid=5e7f9312-de14-4319-be49-35c34a0b2830 label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/rhel_ssstfs03-usr /usr xfs uuid=f082af39-e5eb-4df4-bd60-8f3288650fc7 label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/rhel_ssstfs03-var /var xfs uuid=54085803-3e12-4d97-87e3-77acf0ecc85f label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/mapper/rhel_ssstfs03-var_log /var/log xfs uuid=b040c636-c008-43d1-bd37-d023c096f19a label=  options=rw,relatime,attr2,inode64,noquota
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/mapper/rhel_ssstfs03-swap uuid=6fe88eca-5b91-4d38-b7e7-3c85c5d15536 label=
**swap /dev/mapper/vgswap-swapvol02 uuid=ab161f52-9488-4b07-a25a-7d89dadd48b8 label=**
**multipath /dev/mapper/mpathbp 42949672960 /dev/sdbp,/dev/sdem
part /dev/mapper/mpathbp 42949623808 24576 GPFS none /dev/mapper/mpathbp1**
#multipath /dev/mapper/mpathak 10737418240 /dev/sdap,/dev/sddm
multipath /dev/mapper/mpathr 32212254720 /dev/sdaa,/dev/sdcx
part /dev/mapper/mpathr 32212205568 24576 GPFS none /dev/mapper/mpathr1
multipath /dev/mapper/mpathbc 10737418240 /dev/sdbb,/dev/sddy
part /dev/mapper/mpathbc 10737369088 24576 GPFS none /dev/mapper/mpathbc1
multipath /dev/mapper/mpathe 10737418240 /dev/sdck,/dev/sdn
part /dev/mapper/mpathe 10737369088 24576 GPFS none /dev/mapper/mpathe1
multipath /dev/mapper/mpathaw 75161927680 /dev/sdcd,/dev/sdg
part /dev/mapper/mpathaw 75161878528 24576 GPFS none /dev/mapper/mpathaw1
multipath /dev/mapper/mpathbo 42949672960 /dev/sdbq,/dev/sden
part /dev/mapper/mpathbo 42949623808 24576 GPFS none /dev/mapper/mpathbo1
multipath /dev/mapper/mpathaj 42949672960 /dev/sdak,/dev/sddh
part /dev/mapper/mpathaj 42949623808 24576 GPFS none /dev/mapper/mpathaj1

Work-around, if any:

gozora commented at 2018-04-04 14:39:¶

Your vgswap is on /dev/mapper/mpathbd which is multipath device, hence applies AUTOEXCLUDE_MULTIPATH=n which is IMHO correct.
I can't tell you from top of my head what is the best/correct way to get rid of unwanted multipath devices, but my best guess would be EXCLUDE_COMPONENTS=(), maybe ?

V.

gozora commented at 2018-04-04 14:42:¶

I'm not sure if

#lvmdev /dev/vgswap /dev/mapper/mpathbd AcAIgK-g0S4-wEav-bbQB-KLka-q0z1-FENNKb 52428800

was modified by you, or is it original copy of your disklayout.conf, so just a site note:
disklayout.conf entries starting by '#' are ignored by ReaR ...

V.

pspatola101 commented at 2018-04-04 17:52:¶

In the script , I see the follwoing entry:
parted -s /dev/mapper/mpathf mkpart "'GPFS'" 24576B 10737393663B >&2

I'm concerned that this could over-write data on that gpfs disk. How can that be excluded in the local.conf?
What I'm really trying to do here is just try and create a recovery of the root vg in case it gets corrupted. All my other data is on the san and would be imported in when I recover the rootvg.

gozora commented at 2018-04-04 18:07:¶

That command you've posted would indeed re-partition your disk.
I'm however missing context, as I don't see mpathf being mentioned in layout.conf you've posted previously.

Maybe you could upload some more complex information (disklayout.conf, diskrestore.sh, ..) so that people who understand this whole multipath ppc64le thingy ~~much better that I do~~, have better understanding what is going on ...

V.

pspatola101 commented at 2018-04-04 18:09:¶

I apologize for that. I'll upload a new set.

pspatola101 commented at 2018-04-04 18:16:¶

My current local.conf:

[root@ssstfs03 rear]# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL="nfs://172.19.5.99/stuff/unix/iowa"
SSH_ROOT_PASSWORD="reardemo"
BOOT_FROM_SAN=y
AUTOEXCLUDE_MULTIPATH=n
REAR_INITRD_COMPRESSION=lzma
USE_STATIC_NETWORKING=y
ONLY_INCLUDE_VG=( "rhel_ssstfs03" )
EXCLUDE_RECREATE=( "swap:/dev/mapper/vgswap-swapvol02" "fs:/gpfs/*" )
EXCLUDE_COMPONENTS=( "swap:/dev/mapper/vgswap-swapvol02" "fs:/gpfs/*")
BACKUP_PROG_EXCLUDE=( "${BACKUP_PROG_EXCLUDE[@]}" '/usr/lpp/mmfs/4.2.3.1' '/usr/lpp/mmfs/4.2.3.2' )
AUTOEXCLUDE_DISKS=y

pspatola101 commented at 2018-04-04 18:38:¶

Even though I'm excluding the swap on vgswap, I see it being excluded in the disklayout.conf except for the last line. When I'm in the rear recover, it goes into the "Confirm or edit the disk layout file" and I have to manually comment out that last line for it to continue.

[root@ssstfs03 layout]# cat disklayout.conf |grep vgswap
#lvmdev /dev/vgswap /dev/mapper/mpathbd AcAIgK-g0S4-wEav-bbQB-KLka-q0z1-FENNKb 52428800
#lvmgrp /dev/vgswap 4096 6399 26210304
#lvmvol /dev/vgswap swapvol02 512 4194304
swap /dev/mapper/vgswap-swapvol02 uuid=ab161f52-9488-4b07-a25a-7d89dadd48b8 label=

In the disklayout.conf: I commented out the second line myself. What else do I need to add to my local.conf to exclude that?

# Swap partitions or swap files
# Format: swap  uuid= label=
swap /dev/mapper/rhel_ssstfs03-swap uuid=6fe88eca-5b91-4d38-b7e7-3c85c5d15536 label=
##swap /dev/mapper/vgswap-swapvol02 uuid=ab161f52-9488-4b07-a25a-7d89dadd48b8 label=

pspatola101 commented at 2018-04-04 19:39:¶

disklayout.txt

pspatola101 commented at 2018-04-04 19:52:¶

It looks as if I'm having 2 different issues. One is the additional swap space it looks as if its trying to format under the "#Swap partitions or swap file" stanza. The second is it appears that its trying to reformat the disks that GPFS maintains. That would wipe all of my data.

jsmeix commented at 2018-04-05 12:29:¶

Since
https://github.com/rear/rear/commit/72810e9594d121ee12b5d8394c9fc9daa31ce749
there is a bug in usr/share/rear/layout/save/default/330_remove_exclusions.sh
so that since that commit excluded swap entries are no longer disabled
in disklayout.conf because since that commit swap entries get disabled
by remove_second_component which does not work for swap entries
because swap entries need to get disabled with remove_component
as it was before.

jsmeix commented at 2018-04-05 12:43:¶

Fixed the issue in the above
https://github.com/rear/rear/issues/1767#issuecomment-378919232
via
https://github.com/rear/rear/commit/89a53792e0dc9cb3f20d36719203e89b2e1823cf

jsmeix commented at 2018-04-05 12:52:¶

@pspatola101
with the recent fix
https://github.com/rear/rear/commit/89a53792e0dc9cb3f20d36719203e89b2e1823cf
in current ReaR upstream GitHub master code EXCLUDE_COMPONENTS
works well for me. Regarding EXCLUDE_COMPONENTS syntax see
usr/share/rear/conf/default.conf

For example I have in my etc/rear/local.conf

EXCLUDE_COMPONENTS+=( "fs:/sdb3" "swap:/dev/sdb1" )

and during "rear -D mkrescue" I see (excerpt):

Creating disk layout
Excluding component fs:/sdb3.
Excluding component swap:/dev/sdb1.

and afterwards I got those results (excerpts):

# egrep '^swap |^fs |#swap |#fs ' var/lib/rear/layout/disklayout.conf

fs /dev/sda2 / btrfs uuid=83a3ba1c-7c87-4249-a0cd-a60c3e41ac8a label= options=rw,relatime,space_cache,subvolid=259,subvol=/@/.snapshots/1/snapshot
fs /dev/sdb2 /sdb2 xfs uuid=778a49a1-7f11-4a95-a19b-da6e504f2f5b label=  options=rw,relatime,attr2,inode64,noquota
#fs /dev/sdb3 /sdb3 btrfs uuid=8fc4c3e0-533f-4c54-b5bc-c32daec41015 label= options=rw,relatime,space_cache,subvolid=5,subvol=/
swap /dev/sda1 uuid=b754c8dc-cd7d-42fa-8edc-a69c02c54f68 label=
#swap /dev/sdb1 uuid=d6500016-41c3-48e0-9846-914ba6cc43b3 label=

and

# egrep 'fs:|swap:' var/lib/rear/layout/disktodo.conf

todo fs:/ fs
todo fs:/sdb2 fs
done fs:/sdb3 fs
todo swap:/dev/sda1 swap
done swap:/dev/sdb1 swap

jsmeix commented at 2018-04-06 09:46:¶

@pspatola101
does EXCLUDE_COMPONENTS together with
https://github.com/rear/rear/commit/89a53792e0dc9cb3f20d36719203e89b2e1823cf
also work for your use case?

jsmeix commented at 2018-04-09 13:15:¶

According to
https://github.com/rear/rear/commit/72810e9594d121ee12b5d8394c9fc9daa31ce749#commitcomment-28455084
it works now again to exclude swap partitions.

According to
https://github.com/rear/rear/commit/72810e9594d121ee12b5d8394c9fc9daa31ce749#commitcomment-28455118
a remaining question is how to exclude partitions like

part /dev/mapper/mpathr 32212205568 24576 GPFS none /dev/mapper/mpathr1
part /dev/mapper/mpathbc 10737369088 24576 GPFS none /dev/mapper/mpathbc1

jsmeix commented at 2018-04-09 13:49:¶

One can exlude a partition like "/dev/sdb2" via

EXCLUDE_COMPONENTS=( "${EXCLUDE_COMPONENTS[@]}" "/dev/sdb2" )

and during "rear -D mkrescue" one can see

Excluding component /dev/sdb2.

but then all other partitions on the parent device "/dev/sdb"
will be also automatically excluded because of that code in
usr/share/rear/layout/save/default/330_remove_exclusions.sh

        part)
            ### find the immediate parent
            name=$(grep "^$name " "$LAYOUT_DEPS" | cut -d " " -f 2)
            remove_component "$type" "$name"
            ;;

so that one gets in disklayout.conf

root@host# grep dev/sdb var/lib/rear/layout/disklayout.conf

# Disk /dev/sdb
disk /dev/sdb 2147483648 msdos
# Partitions on /dev/sdb
#part /dev/sdb 103809024 1048576 primary none /dev/sdb1
#part /dev/sdb 104857600 209715200 primary none /dev/sdb2
#part /dev/sdb 1728053248 419430400 primary none /dev/sdb3
#fs /dev/sdb2 /sdb2 xfs uuid=778a49a1-7f11-4a95-a19b-da6e504f2f5b label=  options=rw,relatime,attr2,inode64,noquota
fs /dev/sdb3 /sdb3 btrfs uuid=8fc4c3e0-533f-4c54-b5bc-c32daec41015 label= options=rw,relatime,space_cache,subvolid=5,subvol=/
# Btrfs default subvolume for /dev/sdb3 at /sdb3
btrfsdefaultsubvol /dev/sdb3 /sdb3 5 /
btrfsmountedsubvol /dev/sdb3 /sdb3 rw,relatime,space_cache,subvolid=5,subvol=/ /
swap /dev/sdb1 uuid=d6500016-41c3-48e0-9846-914ba6cc43b3 label=

which shows a bug because only all other partitions
on the parent device will be also automatically excluded
but neither their affected filesystems nor swap entries.

I assume it is no bug but exactly right to also automatically all other partitions
on the parent device "/dev/sdb" because I assume somehow "rear recover"
may fail when it should skip an excluded partition "in between" other partitions.

I guess the reason behind is how the create_partitions() function in
usr/share/rear/layout/prepare/GNU/Linux/100_include_partition_code.sh
works but I don't know the actual reason behind because according to the output of

git log -p --follow usr/share/rear/layout/save/default/330_remove_exclusions.sh

the initial commit where the whole parent device gets exluded
when one of its partitions was exluded is
https://github.com/rear/rear/commit/6a502f6190e1cd04ce7b9b21658452ee489c41bb
where the code was

        part)
            name=$( echo "$name" | sed -r 's/(.*)[0-9]$/\1/')
            if [ "${name/cciss/}" != "$name" ] ; then
                name=${name%p}
            fi
            remove_component $type $name
            ;;

with an evil echo "$name" | sed -r 's/(.*)[0-9]$/\1/' that just breaks
when there are more than 9 partitions:

# name=/dev/sdb2
# echo "$name" | sed -r 's/(.*)[0-9]$/\1/'
/dev/sdb

# name=/dev/sdb12
# echo "$name" | sed -r 's/(.*)[0-9]$/\1/'
/dev/sdb1

which was (hopefully) fixed by that commit
https://github.com/rear/rear/commit/3101421f66d2be65d20f6304a3359944afbe2fdf
to the current code

         part)
            ### find the immediate parent
            name=$(grep "^$name " $LAYOUT_DEPS | cut -d " " -f 2)
             remove_component $type $name
             ;;

jsmeix commented at 2018-04-09 14:14:¶

@schabrolles @gozora
do you perhaps have an idea how to exclude specifically partitions like

part /dev/mapper/mpathr 32212205568 24576 GPFS none /dev/mapper/mpathr1
part /dev/mapper/mpathbc 10737369088 24576 GPFS none /dev/mapper/mpathbc1

without also automatically exclude all other partitions on the same disk?

I do not use multipath so that I cannot reproduce
how ReaR actually behaves in this case.

Perhaps in case of multipath things are different to that it may not matter
when also automatically all other partitions on the same disk get excluded?

pspatola101 commented at 2018-04-09 14:52:¶

I would be ok with a global exclude of any disk that has a "GPFS" label on it. Those disks are managed exclusively by the cluster application and wouldn't have any other partitions on them.

pspatola101 commented at 2018-04-09 15:39:¶

I did try a few things with the local.conf. I removed every exclude and went with the 'ONLY_INCLUDE_VG' and added the name of my rootvg. That works with the fix you gave me and only includes the partitions on my rootvg. The swap on the other vg is commented out on the disklayout.conf.

[root@ssstfs03 rear]# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL="nfs://172.19.5.99/stuff/unix/iowa"
SSH_ROOT_PASSWORD="reardemo"
BOOT_FROM_SAN=y
AUTOEXCLUDE_MULTIPATH=n
REAR_INITRD_COMPRESSION=lzma
USE_STATIC_NETWORKING=y
**ONLY_INCLUDE_VG=( "${ONLY_INCLUDE_VG[@]}" "rhel_ssstfs03" )**

[root@ssstfs03 layout]# cat disklayout.conf |grep swap
#lvmdev /dev/vgswap /dev/mapper/mpathbd AcAIgK-g0S4-wEav-bbQB-KLka-q0z1-FENNKb 52428800
#lvmgrp /dev/vgswap 4096 6399 26210304
lvmvol /dev/rhel_ssstfs03 swap 512 4194304
#lvmvol /dev/vgswap swapvol02 512 4194304
# Swap partitions or swap files
# Format: swap  uuid= label=
swap /dev/mapper/rhel_ssstfs03-swap uuid=6fe88eca-5b91-4d38-b7e7-3c85c5d15536 label=
**#swap /dev/mapper/vgswap-swapvol02 uuid=ab161f52-9488-4b07-a25a-7d89dadd48b8 label=**

Unfortunately, the GPFS disks are still being included.
If I change the multipath flag to yes, it excludes the gpfs disks but also excludes my boot drive as well which is npiv attached. The disklayout.conf has everything excluded.

pspatola101 commented at 2018-04-09 16:31:¶

rear-ssstfs03.log
I see at 335_remove_excluded_multipath_vgs.sh that its excluding the multpath disks that belong to the other volume groups and a few that are not being used.

schabrolles commented at 2018-05-07 13:10:¶

@jsmeix

As GPFS is a very specific filesystem, I think we should exclude it by default.
What about defining an AUTOEXCLUDE_GPFS=yvariable in default.conf

pspatola101 commented at 2018-05-07 20:34:¶

That would be great!

pspatola101 commented at 2018-05-07 21:01:¶

How would I go about doing that?

jsmeix commented at 2018-05-08 09:42:¶

@schabrolles
in usr/share/rear/layout/save/GNU/Linux/230_filesystem_layout.sh there is

supported_filesystems="ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs"

so that the GPFS filesystem is not supported by ReaR.

@pspatola101
when a filesystem is not supported by ReaR it means
ReaR has no code to recreate such filesystems.

Accordingly it does not make sense to have any fs entries
for GPFS filesystems in disklayout.conf (which do not happen)
BUT
it does make sense to have part entries for partitions
that have a GPFS partition type/name in disklayout.conf

Reason:
On one same disk there can be part entries for partitions
that have a filesystem that is supported by ReaR
plus (on the same disk) there can be part entries for partitions
that have a filesystem that is not supported by ReaR e.g. like:

part /dev/sda ... ... ext2 none /dev/sda1
part /dev/sda ... ... GPFS none /dev/sda2
part /dev/sda ... ... XFS none /dev/sda3

During "rear recover" all three partitions must be recreated
because otherwise the XFS partition would become /dev/sda2
cf. https://github.com/rear/rear/issues/1793

Even if in parctice a partition with a GPFS partition type/name
cannot be on the same disk together with partitions
that have a filesystem that is supported by ReaR
my gut feeling is that AUTOEXCLUDE_GPFS=y
looks like a quick and dirty hack that may cause
other issues in the future.

I fear the actual solution is to overhaul the current EXCLUDE_COMPONENTS
implementation to make it usable even for special needs like this one.
I think the current EXCLUDE_COMPONENTS implementation needs to be
enhanced so that user can specify more exactly what he wants to get excluded.

@pspatola101
Note that usually things will go wrong during "rear recover"
when you exclude to recreate one particular partition
that is between other partitions on a disk (see my reason above).
I think it should work to only exclude one or more last partitions
on a disk but I think this is currently not implemented in ReaR.

schabrolles commented at 2018-05-08 12:42:¶

The PR #1801 could help here.
As no fs: GPFS are stored in the disklayout.conf, all GPFS multipath devices and partitions should be commented.

pspatola101 commented at 2018-05-08 15:09:¶

Thank you all for all your work.
@schabrolles
When would I be able to access the new build?
@jsmeix
I understand what you mean. The gpfs exclusion would fix my issue it may open up the group to other single fix issues.

I do have one more question. I'm looking at this as using a mksysb for aix. If I have the following statement in my local.conf.
ONLY_INCLUDE_VG=( "${ONLY_INCLUDE_VG[@]}" "rhel_ssstfs03" )
Why would it be trying to do anything else with the gpfs disks or any of the other vg? Shouldn't it be excluding everything but the included vg?

schabrolles commented at 2018-05-08 15:24:¶

@pspatola101,

I prefer to have a review from @jsmeix and @gozora before merging it into master branch.
If you can't wait, you can evaluate it from here:

git clone http://github.com/schabrolles/rear -b exclude_mpath_no_fs
mv rear rear.github.exclude_mpath_no_fs
cd rear.github.exclude_mpath_no_fs
vi etc/rear/local.conf
usr/sbin/rear -D mkbackup

schabrolles commented at 2018-05-08 15:54:¶

@pspatola101,
ONLY_INCLUDE_VG=( "rootvg" ) means exclude ALL the other VGs ... But you can have FS directly on disks.
Normally, all non-monted or unsupported FS are excluded by default. ReaR should exclude GPFS disks as it already exclude the fs (it is not in disklayout.conf). But you need the fix #1801 in order to have it work properly with multipath and not only pure disk.

pspatola101 commented at 2018-05-08 17:20:¶

Thank you for the response. I will be patient and wait.

jsmeix commented at 2018-05-09 09:50:¶

@schabrolles
regarding your https://github.com/rear/rear/issues/1767#issuecomment-387389332

As no fs: GPFS are stored in the disklayout.conf, all GPFS multipath devices
and partitions should be commented.

I think this is not right in general.

I think the opposite is true:
When there is a part entry without a matching fs entry in disklayout.conf
the part entry cannot be commented in general because that may lead
to a wrong recreated partitioning, see my "Reason" in
https://github.com/rear/rear/issues/1767#issuecomment-387346382

schabrolles commented at 2018-05-09 10:26:¶

@jsmeix,

So, I maybe miss something... if AUTOEXCLUDE_DISKS=y (which is the default) all disks (and multipath if #1801 is applied) should be excluded if they are not a parent of a mounted fs:
This should prevent RAW devices disk to be recreated (partition etc ... ) and other non-supported FS (like GPFS)

extract from default.conf

# Automatically exclude disks that are not used by mounted filesystems
# Explicitly excluding/including devices is generally a safer option.
# (layout code)
AUTOEXCLUDE_DISKS=y

Please, correct me if I'm wrong.

schabrolles commented at 2018-05-09 10:28:¶

@jsmeix , forget my last comment... I just get your point (with multiple partitions) ...

schabrolles commented at 2018-05-10 15:00:¶

@pspatola101
#1801 is now merged into rear master branch. Could you make a test and tell us if it solves you issue?
Thanks.

pspatola101 commented at 2018-05-14 15:21:¶

Hey guys... I ran it this morning and I received the following error message.
[root@ssstfs03 ~]# rear -D mkrescue
Relax-and-Recover 2.3-git.2991.e4eb0e1.master.changed / 2018-05-14
Using log file: /var/log/rear/rear-ssstfs03.log
Using backup archive '/tmp/rear.fNJ8lRBPtGLi2Sh/outputfs/ssstfs03/backup.tar.gz'
Creating disk layout
ERROR: Partition number '' of partition datavg-lvzimon is not a valid number.
ERROR: Partition datavg-lvzimon is numbered ''. More than 128 partitions is not supported.

pspatola101 commented at 2018-05-14 15:21:¶

rear-ssstfs03.log

pspatola101 commented at 2018-05-14 15:28:¶

looks like someone had a similar issue with Oracle ASM disks before.
https://github.com/rear/rear/issues/373

schabrolles commented at 2018-05-14 15:54:¶

@pspatola101 my PR #1802 introduces a new BUG when you create LVM physical Volume directly on the disk (without any partition). I'm gonna look at it.

schabrolles commented at 2018-05-14 17:19:¶

@pspatola101, I've isolated the issue. It is now working for me with a Redhat and VG created directly on the disk (like you).
I would like to perform some additional test with other distribution (Suse/Ubuntu) before creating a new PR.
If you have time and want to test the fix, it is here:

git clone http://github.com/schabrolles/rear -b get_only_partition_from_holders

pspatola101 commented at 2018-05-15 16:29:¶

It worked beautifully!!!!! I really appreciate your work on this. The mkrescue finished successfully. This is the layout of the the todo file. The GPFS multipath disks are commented out in the disklayout.conf! Please let me know when it merges into the master branch.
[root@ssstfs03 layout]# cat disktodo.conf
done pv:/dev/mapper/mpathbe lvmdev
todo pv:/dev/mapper/mpatha3 lvmdev
done pv:/dev/mapper/mpathbd lvmdev
done /dev/datavg lvmgrp
todo /dev/rhel_ssstfs03 lvmgrp
done /dev/vgswap lvmgrp
done /dev/mapper/datavg-lvzimon lvmvol
todo /dev/mapper/rhel_ssstfs03-root lvmvol
todo /dev/mapper/rhel_ssstfs03-usr lvmvol
todo /dev/mapper/rhel_ssstfs03-opt lvmvol
todo /dev/mapper/rhel_ssstfs03-home lvmvol
todo /dev/mapper/rhel_ssstfs03-tmp lvmvol
todo /dev/mapper/rhel_ssstfs03-var lvmvol
todo /dev/mapper/rhel_ssstfs03-var_log lvmvol
todo /dev/mapper/rhel_ssstfs03-swap lvmvol
done /dev/mapper/vgswap-swapvol02 lvmvol
done fs:/opt/IBM/zimon/data fs
todo fs:/boot fs
todo fs:/home fs
todo fs:/opt fs
todo fs:/ fs
todo fs:/tmp fs
todo fs:/usr fs
todo fs:/var fs
todo fs:/var/log fs
todo swap:/dev/mapper/rhel_ssstfs03-swap swap
done swap:/dev/mapper/vgswap-swapvol02 swap
done /dev/mapper/mpathbp multipath
done /dev/mapper/mpathbp1 part
done /dev/mapper/mpathak multipath
done /dev/mapper/mpathr multipath
done /dev/mapper/mpathr1 part
done /dev/mapper/mpathbc multipath
done /dev/mapper/mpathbc1 part
done /dev/mapper/mpathe multipath
done /dev/mapper/mpathe1 part
done /dev/mapper/mpathaw multipath
done /dev/mapper/mpathaw1 part
done /dev/mapper/mpathbo multipath
done /dev/mapper/mpathbo1 part
done /dev/mapper/mpathaj multipath
done /dev/mapper/mpathaj1 part
done /dev/mapper/mpathq multipath
done /dev/mapper/mpathq1 part
done /dev/mapper/mpathd multipath
done /dev/mapper/mpathd1 part
done /dev/mapper/mpathbb multipath
done /dev/mapper/mpathbb1 part
done /dev/mapper/mpathav multipath
done /dev/mapper/mpathav1 part
done /dev/mapper/mpathbn multipath
done /dev/mapper/mpathbn1 part
done /dev/mapper/mpathai multipath
done /dev/mapper/mpathai1 part
done /dev/mapper/mpathp multipath
done /dev/mapper/mpathp1 part
done /dev/mapper/mpathba multipath
done /dev/mapper/mpathba1 part
done /dev/mapper/mpathc multipath
done /dev/mapper/mpathc1 part
done /dev/mapper/mpathbz multipath
done /dev/mapper/mpathau multipath
done /dev/mapper/mpathau1 part
done /dev/mapper/mpathbm multipath
done /dev/mapper/mpathbm1 part
done /dev/mapper/mpathah multipath
done /dev/mapper/mpatho multipath
done /dev/mapper/mpatho1 part
done /dev/mapper/mpathb multipath
done /dev/mapper/mpathb1 part
done /dev/mapper/mpathby multipath
done /dev/mapper/mpathat multipath
done /dev/mapper/mpathat1 part
done /dev/mapper/mpathbl multipath
done /dev/mapper/mpathbl1 part
done /dev/mapper/mpathag multipath
done /dev/mapper/mpathag1 part
done /dev/mapper/mpathn multipath
done /dev/mapper/mpathn1 part
todo /dev/mapper/mpatha multipath
todo /dev/mapper/mpatha1 part
todo /dev/mapper/mpatha2 part
todo /dev/mapper/mpatha3 part
done /dev/mapper/mpathbx multipath
done /dev/mapper/mpathas multipath
done /dev/mapper/mpathas1 part
done /dev/mapper/mpathz multipath
done /dev/mapper/mpathz1 part
done /dev/mapper/mpathbk multipath
done /dev/mapper/mpathbk1 part
done /dev/mapper/mpathaf multipath
done /dev/mapper/mpathm multipath
done /dev/mapper/mpathm1 part
done /dev/mapper/mpathbw multipath
done /dev/mapper/mpathbw1 part
done /dev/mapper/mpathar multipath
done /dev/mapper/mpathar1 part
done /dev/mapper/mpathy multipath
done /dev/mapper/mpathy1 part
done /dev/mapper/mpathbj multipath
done /dev/mapper/mpathbj1 part
done /dev/mapper/mpathae multipath
done /dev/mapper/mpathl multipath
done /dev/mapper/mpathl1 part
done /dev/mapper/mpathbv multipath
done /dev/mapper/mpathbv1 part
done /dev/mapper/mpathaq multipath
done /dev/mapper/mpathaq1 part
done /dev/mapper/mpathx multipath
done /dev/mapper/mpathx1 part
done /dev/mapper/mpathbi multipath
done /dev/mapper/mpathbi1 part
done /dev/mapper/mpathad multipath
done /dev/mapper/mpathk multipath
done /dev/mapper/mpathk1 part
done /dev/mapper/mpathbu multipath
done /dev/mapper/mpathbu1 part
done /dev/mapper/mpathap multipath
done /dev/mapper/mpathap1 part
done /dev/mapper/mpathw multipath
done /dev/mapper/mpathw1 part
done /dev/mapper/mpathbh multipath
done /dev/mapper/mpathbh1 part
done /dev/mapper/mpathac multipath
done /dev/mapper/mpathj multipath
done /dev/mapper/mpathj1 part
done /dev/mapper/mpathbt multipath
done /dev/mapper/mpathbt1 part
done /dev/mapper/mpathao multipath
done /dev/mapper/mpathao1 part
done /dev/mapper/mpathv multipath
done /dev/mapper/mpathv1 part
done /dev/mapper/mpathbg multipath
done /dev/mapper/mpathbg1 part
done /dev/mapper/mpathab multipath
done /dev/mapper/mpathab1 part
done /dev/mapper/mpathi multipath
done /dev/mapper/mpathi1 part
done /dev/mapper/mpathbs multipath
done /dev/mapper/mpathbs1 part
done /dev/mapper/mpathan multipath
done /dev/mapper/mpathan1 part
done /dev/mapper/mpathu multipath
done /dev/mapper/mpathu1 part
done /dev/mapper/mpathbf multipath
done /dev/mapper/mpathbf1 part
done /dev/mapper/mpathaa multipath
done /dev/mapper/mpathaa1 part
done /dev/mapper/mpathh multipath
done /dev/mapper/mpathh1 part
done /dev/mapper/mpathaz multipath
done /dev/mapper/mpathaz1 part
done /dev/mapper/mpathbr multipath
done /dev/mapper/mpathbr1 part
done /dev/mapper/mpatham multipath
done /dev/mapper/mpatham1 part
done /dev/mapper/mpatht multipath
done /dev/mapper/mpatht1 part
done /dev/mapper/mpathbe multipath
done /dev/mapper/mpathg multipath
done /dev/mapper/mpathg1 part
done /dev/mapper/mpathay multipath
done /dev/mapper/mpathay1 part
done /dev/mapper/mpathbq multipath
done /dev/mapper/mpathbq1 part
done /dev/mapper/mpathal multipath
done /dev/mapper/mpathal1 part
done /dev/mapper/mpaths multipath
done /dev/mapper/mpaths1 part
done /dev/mapper/mpathbd multipath
done /dev/mapper/mpathf multipath
done /dev/mapper/mpathf1 part
done /dev/mapper/mpathax multipath
done /dev/mapper/mpathax1 part

schabrolles commented at 2018-05-15 17:05:¶

@pspatola101 thanks for the prompt feedback, it really help us.
PR #1805 is already created. I'm just waiting for other member to have a quick review on it before merging.... (hopfully tomorrow).

schabrolles commented at 2018-05-18 07:26:¶

@pspatola101, #1805 is now merged into master branch.

Please confirm everything is working now to close this issue.

Thanks.

schabrolles commented at 2018-05-23 08:14:¶

@pspatola101, any news ?

pspatola101 commented at 2018-05-29 15:17:¶

Apologize for the delay Sebastien. The new package worked like a charm!. Everything is commented out as it should be. I really appreciate all your hard work with this. If you ever need a tester for redhat power linux please do not hesitate to contact me. I would be happy to help.

schabrolles commented at 2018-05-29 17:28:¶

@pspatola101 You're welcome.
could you please drop me a mail at s.chabrolles@fr.ibm.com. As I would like to discuss more with you offline about your activity around linux, redhat, power and rear. ... Thanks if you have time to do that.

jsmeix commented at 2018-08-01 07:30:¶

I fixed an evil logical typo in my
https://github.com/rear/rear/issues/1767#issuecomment-387346382

Before it was (wrong):

... it does not make sense to have any fs entries
for GPFS filesystems in disklayout.conf (which do not happen)
BUT
it does not make sense to have part entries for partitions
that have a GPFS partition type/name in disklayout.conf

Now it is (right):

... it does not make sense to have any fs entries
for GPFS filesystems in disklayout.conf (which do not happen)
BUT
it does make sense to have part entries for partitions
that have a GPFS partition type/name in disklayout.conf

i.e. for fs entries it still does not make sense
but for part entries it must be it does make sense

[Export of Github issue for rear/rear.]

#1767 Issue closed: trying to exclude other mount points and swap from rear recovery¶

pspatola101 opened issue at 2018-04-03 18:50:¶

Relax-and-Recover (ReaR) Issue Template¶

gozora commented at 2018-04-04 14:39:¶

gozora commented at 2018-04-04 14:42:¶

pspatola101 commented at 2018-04-04 17:52:¶

gozora commented at 2018-04-04 18:07:¶

pspatola101 commented at 2018-04-04 18:09:¶

pspatola101 commented at 2018-04-04 18:16:¶

pspatola101 commented at 2018-04-04 18:38:¶

pspatola101 commented at 2018-04-04 19:39:¶

pspatola101 commented at 2018-04-04 19:52:¶

jsmeix commented at 2018-04-05 12:29:¶

jsmeix commented at 2018-04-05 12:43:¶

jsmeix commented at 2018-04-05 12:52:¶

jsmeix commented at 2018-04-06 09:46:¶

jsmeix commented at 2018-04-09 13:15:¶

jsmeix commented at 2018-04-09 13:49:¶

jsmeix commented at 2018-04-09 14:14:¶

pspatola101 commented at 2018-04-09 14:52:¶

pspatola101 commented at 2018-04-09 15:39:¶

pspatola101 commented at 2018-04-09 16:31:¶

schabrolles commented at 2018-05-07 13:10:¶

pspatola101 commented at 2018-05-07 20:34:¶

pspatola101 commented at 2018-05-07 21:01:¶

jsmeix commented at 2018-05-08 09:42:¶

schabrolles commented at 2018-05-08 12:42:¶

pspatola101 commented at 2018-05-08 15:09:¶

schabrolles commented at 2018-05-08 15:24:¶

schabrolles commented at 2018-05-08 15:54:¶

pspatola101 commented at 2018-05-08 17:20:¶

jsmeix commented at 2018-05-09 09:50:¶

schabrolles commented at 2018-05-09 10:26:¶

schabrolles commented at 2018-05-09 10:28:¶

schabrolles commented at 2018-05-10 15:00:¶

pspatola101 commented at 2018-05-14 15:21:¶

pspatola101 commented at 2018-05-14 15:21:¶

pspatola101 commented at 2018-05-14 15:28:¶

schabrolles commented at 2018-05-14 15:54:¶

schabrolles commented at 2018-05-14 17:19:¶

pspatola101 commented at 2018-05-15 16:29:¶

schabrolles commented at 2018-05-15 17:05:¶

schabrolles commented at 2018-05-18 07:26:¶

schabrolles commented at 2018-05-23 08:14:¶

pspatola101 commented at 2018-05-29 15:17:¶

schabrolles commented at 2018-05-29 17:28:¶

jsmeix commented at 2018-08-01 07:30:¶

#1767 Issue `closed`: trying to exclude other mount points and swap from rear recovery¶