#1958 Issue closed: Support by-path PVs¶
Labels: enhancement, fixed / solved / done
schlomo opened issue at 2018-11-08 10:59:¶
ReaR 2.4 on SLES12 SP3 with local disk in VMware VM with LVM
Problem
PVs are addressed by /dev/disk/by-path/ like this:
# pvs
PV VG Fmt Attr PSize PFree
/dev/disk/by-id/dm-name-local-lvm-modbtsbl-114v_vol00_dg-001 modbtsbl-114v_vol00_dg lvm2 a-- 550.10g 104.00m
/dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0-part2 storage lvm2 a-- 19.99g 1.49g
# pvdisplay -c
/dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0-part2:storage:41924608:-1:8:8:-1:4096:5117:381:4736:HcNHiV-SSq8-yvzK-3q21-p8ND-qGah-J1eb6J
/dev/disk/by-id/dm-name-local-lvm-modbtsbl-114v_vol00_dg-001:modbtsbl-114v_vol00_dg:1153654784:-1:8:8:-1:4096:140826:26:140800:JDWfR1-feeC-Eznp-x1Qo-7TAb-j5KK-JMc1Bj
The resulting disklayout.conf contains broken lines like this:
lvmdev /dev/03 /dev/disk/by-path/pci-0000 -1 00.0-scsi-0
And consequently ReaR doesn't work.
Cause
Root cause is
220_lvm_layout.sh
using : to separate the output. : also appears as part of the PV
physical block device name so that our code breaks.
Solution
Don't use : as a separator in stuff that contains file names because
: is valid and used in file names.
Further TODO
This fix alone is not enough because ReaR does not recognize the
dependency between the PV and the /dev/sda disk and therefore excludes
that disk from formatting and recovery.
schlomo commented at 2018-11-08 11:32:¶
@skrueger8 FYI
jsmeix commented at 2018-11-08 11:36:¶
I added @gdha because according to the
git log -p --follow usr/share/rear/layout/save/GNU/Linux/220_lvm_layout.sh
output layout/save/GNU/Linux/220_lvm_layout.sh was initially creted
by
@gdha via
https://github.com/rear/rear/commit/c7b79dd8ed4f3c653e223fee482feb740edea7a4
that already uses : as filed separator but the commit message
'added patch of Jeroen to check/save-layout workflows'
indicates it originates from @jhoekx
I think the 'pvdisplay -c' output is the root cause because
dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0-part2:storage:...
is already broken because the 'pvdisplay -c' output uses : as field
separator
but that character appears in the first field value in this case.
jsmeix commented at 2018-11-08 11:39:¶
I need to set up a test system with LVM to be able to actually work on it...
schlomo commented at 2018-11-08 11:43:¶
@skrueger8 can you please check that the fix in https://github.com/rear/rear/commit/ea2b700ef16ed1dc46d46ad472c6ee91fc38ba30 reflects what we tested?
schlomo commented at 2018-11-08 11:48:¶
@jsmeix for testing some more info:
- GPT disk with BIOS boot
- 1st partition is
bios_grub - 2nd partition is PV on which resides the LV with the btrfs volume for the system
/bootis on the btrfs volume that has/- PV is defined via its
by-pathblock device file
Please add additional fixes to the issue-1958 branch and we will also
try them out.
jsmeix commented at 2018-11-08 15:28:¶
@schlomo
there are so many ways how to setup storage in YaST
that I need more details how to setup what you have
so that I can reproduce their issue.
What is the output of commands like
# parted -s /dev/sda unit MiB print # lsblk -i -p -o NAME,KNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT /dev/sda
on a relatively small system that has this issue?
FYI
what I have on my SLES12 test system with the default LVM setup that one
gets
when one selects during installation in YaST the "LVM-based Proposal"
(but not the "Encrypted LVM-based Proposal")
# parted -s /dev/sda unit MiB print Model: ATA QEMU HARDDISK (scsi) Disk /dev/sda: 20480MiB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 1.00MiB 20480MiB 20479MiB primary boot, lvm, type=8e # lsblk -i -p -o NAME,KNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT /dev/sda NAME KNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT /dev/sda /dev/sda ata disk 20G `-/dev/sda1 /dev/sda1 part LVM2_member 20G |-/dev/mapper/system-swap /dev/dm-0 lvm swap 1.4G [SWAP] `-/dev/mapper/system-root /dev/dm-1 lvm btrfs 18.6G /
Meanwhile I found out how to enforce GPT in YaST (somewhat hidden) and
how to manually create a BIOS Grub partition in YaST (Googling helped
me)
perhaps I should even read some SUSE manuals (my very last resort ;-)
...
jsmeix commented at 2018-11-08 16:14:¶
I get closer but up to now I failed to setup by-path PVs.
What I have now via manual setup in YaST:
# parted -s /dev/sda unit MiB print Model: ATA QEMU HARDDISK (scsi) Disk /dev/sda: 20480MiB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: pmbr_boot Number Start End Size File system Name Flags 1 1.00MiB 16.0MiB 15.0MiB bios_grub 2 16.0MiB 19469MiB 19453MiB lvm, legacy_boot # lsblk -i -p -o NAME,KNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT NAME KNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT /dev/sda /dev/sda ata disk 20G |-/dev/sda1 /dev/sda1 part 15M `-/dev/sda2 /dev/sda2 part LVM2_member 19G |-/dev/mapper/system-system--swap /dev/dm-0 lvm swap 1G [SWAP] `-/dev/mapper/system-system--root /dev/dm-1 lvm btrfs 18G / /dev/sr0 /dev/sr0 ata rom 1024M # pvs PV VG Fmt Attr PSize PFree /dev/sda2 system lvm2 a-- 19.00g 4.00m # pvdisplay -c /dev/sda2:system:39839744:-1:8:8:-1:4096:4863:1:4862:7guAfh-09wj-3ivj-oH4K-kvqE-NUjJ-r9sPSu # pvdisplay --columns --separator "|" PV|VG|Fmt|Attr|PSize|PFree /dev/sda2|system|lvm2|a--|19.00g|4.00m
skrueger8 commented at 2018-11-09 07:28:¶
@schlomo
there are so many ways how to setup storage in YaST
that I need more details how to setup what you have
so that I can reproduce their issue.What is the output of commands like
parted -s /dev/sda unit MiB print¶
lsblk -i -p -o NAME,KNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT /dev/sda¶
on a relatively small that has this issue?
Hi, here's the output you requested:
# parted -s /dev/sda unit MiB print
Model: VMware Virtual disk (scsi)
Disk /dev/sda: 20480MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1.00MiB 8.00MiB 7.00MiB primary bios_grub
2 8.00MiB 20479MiB 20471MiB primary lvm, legacy_boot
# lsblk -i -p -o NAME,KNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT /dev/sda
NAME KNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/sda /dev/sda disk 20G
|-/dev/sda1 /dev/sda1 part 7M
`-/dev/sda2 /dev/sda2 part LVM2_member 20G
|-/dev/mapper/storage-root /dev/dm-0 lvm btrfs 12G /
|-/dev/mapper/storage-home /dev/dm-8 lvm xfs 5G /home
|-/dev/mapper/storage-varcrash /dev/dm-9 lvm xfs 1G /var/crash
`-/dev/mapper/storage-varlog /dev/dm-10 lvm xfs 512M /var/log
skrueger8 commented at 2018-11-09 07:32:¶
@skrueger8 can you please check that the fix in ea2b700 reflects what we tested?
Yup, that's the fix we tested yesterday.
jsmeix commented at 2018-11-09 08:10:¶
@skrueger8
thanks for the info!
@skrueger8 @schlomo
do you perhaps know how to install a SLES12 system so that it uses
by-path PVs?
I mean what I would have to do in YaST during installation to get
by-path PVs?
Or do you know a URL to documentation where that is described?
(I can find that out on my own but I don't know how long that may
take...)
Or can you describe me how you installed your system so that it uses
by-path PVs
so that I can relatively easily reproduce it?
I guess if you have many such systems you probably don't install each
of them one by one with YaST but e.g. perhaps with AutoYaST (or
whatever
other deployment tool you may use) perhaps together with even more
sophisticated software on top of that like SUSE Manager (or anything
else)
but that would be something that I cannot "relatively easily reproduce".
On the other hand - I guess - when you use AutoYaST for installation
you may provide me your AutoYaST autoinst.xml file for one of your
systems
(ideally for an as simple and small as possible system that uses by-path
PVs)
pereferably attach your AutoYaST autoinst.xml file to this issue
here.
If you provide me your AutoYaST autoinst.xml file ensure it does
no longer contain secrets like root password or things like that.
Then - I hope - I can relatively easily reproduce your system.
schlomo commented at 2018-11-09 08:17:¶
https://www.suse.com/documentation/sles11/book_autoyast/data/createprofile_partitioning.html suggests to use
<mountby config:type="symbol">path</mountby>
We will provide you with the relevant information
jsmeix commented at 2018-11-09 09:58:¶
I am afraid, today I cannot work on this issue,
I have some Ghostscript security related things to do, cf.
https://ghostscript.com/pipermail/gs-devel/2018-November/010126.html
Hopefully next week...
jsmeix commented at 2019-02-26 10:00:¶
"needs sponsorship" => for a "ReaR future" milestone
jsmeix commented at 2020-02-18 11:22:¶
I think calling the 'lvm' commands with --separator ' | ' --aligned
(for better human readability) or plain --separator '|' options
could be a (hopefully) sufficiently fail-safe way because at least on
all my systems find /dev | grep '|' results nothing so that it seems
the | character does not appear in device nodes or device symlinks
...
of course except those users who use the | character in labels
so that it appears in /dev/disk/by-label/ e.g. something like
/dev/disk/by-label/my|stuff
github-actions commented at 2020-06-30 01:33:¶
Stale issue message
github-actions commented at 2022-10-31 03:30:¶
Stale issue message
pcahyna commented at 2023-01-02 15:22:¶
I believe #2903 addresses all the issues:
This fix alone is not enough because ReaR does not recognize the dependency between the PV and the /dev/sda disk and therefore excludes that disk from formatting and recovery.
this is not an issue for some reason. Here is the layout on a two-disk
system configured to show /dev/disk/by-path PVs:
/var/lib/rear/layout/disklayout.conf
# Disk layout dated 20230102095535 (YYYYmmddHHMMSS)
# NAME KNAME PKNAME TRAN TYPE FSTYPE LABEL SIZE MOUNTPOINT UUID WWN
# /dev/sda /dev/sda sata disk 931.5G 0x5000c5007971fc8c
# |-/dev/sda1 /dev/sda1 /dev/sda part vfat 600M /boot/efi C7D3-F00C 0x5000c5007971fc8c
# |-/dev/sda2 /dev/sda2 /dev/sda part xfs 1G /boot a8512f5b-ebc6-460b-8f5e-508df68c185f 0x5000c5007971fc8c
# `-/dev/sda3 /dev/sda3 /dev/sda part LVM2_member 929.9G 2z2Dm7-R1JN-eCjn-arTJ-rQDR-YQz8-meSfPN 0x5000c5007971fc8c
# |-/dev/mapper/rhel_hpe--dl120gen9--01-root /dev/dm-0 /dev/sda3 lvm xfs 70G / 89705cff-2ff7-4b45-90be-1cc292c0ef20
# |-/dev/mapper/rhel_hpe--dl120gen9--01-swap /dev/dm-1 /dev/sda3 lvm swap 7.8G [SWAP] d5b1cc1d-f7ab-4f25-bccf-e8fb73750cb5
# `-/dev/mapper/rhel_hpe--dl120gen9--01-home /dev/dm-2 /dev/sda3 lvm xfs 1.8T /home 77b0e7ae-92c7-4137-842b-f06bdadda278
# /dev/sdb /dev/sdb sata disk 931.5G 0x5000c500798fc371
# `-/dev/sdb1 /dev/sdb1 /dev/sdb part LVM2_member 931.5G Vg2HFj-EZwJ-k1D9-UjKv-Gwkr-6dhx-d0O7mc 0x5000c500798fc371
# `-/dev/mapper/rhel_hpe--dl120gen9--01-home /dev/dm-2 /dev/sdb1 lvm xfs 1.8T /home 77b0e7ae-92c7-4137-842b-f06bdadda278
# Disk /dev/sda
# Format: disk <devname> <size(bytes)> <partition label type>
disk /dev/sda 1000204886016 gpt
# Partitions on /dev/sda
# Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
part /dev/sda 629145600 1048576 EFI%20System%20Partition boot,esp /dev/sda1
part /dev/sda 1073741824 630194176 rear-noname none /dev/sda2
part /dev/sda 998500204544 1703936000 rear-noname lvm /dev/sda3
# Disk /dev/sdb
# Format: disk <devname> <size(bytes)> <partition label type>
disk /dev/sdb 1000204886016 gpt
# Partitions on /dev/sdb
# Format: part <device> <partition size(bytes)> <partition start(bytes)> <partition type|name> <flags> /dev/<partition>
part /dev/sdb 1000203091968 1048576 rear-noname lvm /dev/sdb1
# Format for LVM PVs
# lvmdev <volume_group> <device> [<uuid>] [<size(bytes)>]
lvmdev /dev/rhel_hpe-dl120gen9-01 /dev/sda3 2z2Dm7-R1JN-eCjn-arTJ-rQDR-YQz8-meSfPN 998496010240
lvmdev /dev/rhel_hpe-dl120gen9-01 /dev/sdb1 Vg2HFj-EZwJ-k1D9-UjKv-Gwkr-6dhx-d0O7mc 1000198897664
# Format for LVM VGs
# lvmgrp <volume_group> <extentsize> [<size(extents)>] [<size(bytes)>]
lvmgrp /dev/rhel_hpe-dl120gen9-01 4096 476526 1951850496
# Format for LVM LVs
# lvmvol <volume_group> <name> <size(bytes)> <layout> [key:value ...]
lvmvol /dev/rhel_hpe-dl120gen9-01 swap 8338276352b linear
lvmvol /dev/rhel_hpe-dl120gen9-01 root 75161927680b linear
# Volume rhel_hpe-dl120gen9-01/home has multiple segments. Recreating it by 'lvcreate' will not preserve segments and properties of the other segments as well
lvmvol /dev/rhel_hpe-dl120gen9-01 home 1915194703872b linear
# Extra parameters for the 'lvmvol /dev/rhel_hpe-dl120gen9-01 home' line above not taken into account when restoring using 'lvcreate': segmentsize:1000198897664b
#lvmvol /dev/rhel_hpe-dl120gen9-01 home 1915194703872b linear
# Extra parameters for the '#lvmvol /dev/rhel_hpe-dl120gen9-01 home' line above not taken into account when restoring using 'lvcreate': segmentsize:914995806208b
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs <device> <mountpoint> <fstype> [uuid=<uuid>] [label=<label>] [<attributes>]
fs /dev/mapper/rhel_hpe--dl120gen9--01-home /home xfs uuid=77b0e7ae-92c7-4137-842b-f06bdadda278 label= options=rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota
fs /dev/mapper/rhel_hpe--dl120gen9--01-root / xfs uuid=89705cff-2ff7-4b45-90be-1cc292c0ef20 label= options=rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota
fs /dev/sda1 /boot/efi vfat uuid=C7D3-F00C label= options=rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro
fs /dev/sda2 /boot xfs uuid=a8512f5b-ebc6-460b-8f5e-508df68c185f label= options=rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota
# Swap partitions or swap files
# Format: swap <filename> uuid=<uuid> label=<label>
swap /dev/mapper/rhel_hpe--dl120gen9--01-swap uuid=d5b1cc1d-f7ab-4f25-bccf-e8fb73750cb5 label=
/var/lib/rear/layout/diskdeps.conf
/dev/sda1 /dev/sda
/dev/sda2 /dev/sda
/dev/sda3 /dev/sda
/dev/sdb1 /dev/sdb
/dev/rhel_hpe-dl120gen9-01 pv:/dev/sda3
pv:/dev/sda3 /dev/sda3
/dev/rhel_hpe-dl120gen9-01 pv:/dev/sdb1
pv:/dev/sdb1 /dev/sdb1
/dev/mapper/rhel_hpe--dl120gen9--01-swap /dev/rhel_hpe-dl120gen9-01
/dev/mapper/rhel_hpe--dl120gen9--01-root /dev/rhel_hpe-dl120gen9-01
/dev/mapper/rhel_hpe--dl120gen9--01-home /dev/rhel_hpe-dl120gen9-01
fs:/home /dev/mapper/rhel_hpe--dl120gen9--01-home
fs:/home fs:/
fs:/ /dev/mapper/rhel_hpe--dl120gen9--01-root
fs:/boot/efi /dev/sda1
fs:/boot/efi fs:/
fs:/boot/efi fs:/boot
fs:/boot /dev/sda2
fs:/boot fs:/
swap:/dev/mapper/rhel_hpe--dl120gen9--01-swap /dev/mapper/rhel_hpe--dl120gen9--01-swap
/var/lib/rear/layout/disktodo.conf
todo /dev/sda disk
todo /dev/sda1 part
todo /dev/sda2 part
todo /dev/sda3 part
todo /dev/sdb disk
todo /dev/sdb1 part
todo pv:/dev/sda3 lvmdev
todo pv:/dev/sdb1 lvmdev
todo /dev/rhel_hpe-dl120gen9-01 lvmgrp
todo /dev/mapper/rhel_hpe--dl120gen9--01-swap lvmvol
todo /dev/mapper/rhel_hpe--dl120gen9--01-root lvmvol
todo /dev/mapper/rhel_hpe--dl120gen9--01-home lvmvol
todo fs:/home fs
todo fs:/ fs
todo fs:/boot/efi fs
todo fs:/boot fs
todo swap:/dev/mapper/rhel_hpe--dl120gen9--01-swap swap
lvm pvdisplay -c
/dev/disk/by-path/pci-0000:00:11.4-ata-1-part3:rhel_hpe-dl120gen9-01:1950195712:-1:8:8:-1:4096:238060:0:238060:2z2Dm7-R1JN-eCjn-arTJ-rQDR-YQz8-meSfPN
/dev/disk/by-path/pci-0000:00:11.4-ata-2-part1:rhel_hpe-dl120gen9-01:1953521664:-1:8:8:-1:4096:238466:0:238466:Vg2HFj-EZwJ-k1D9-UjKv-Gwkr-6dhx-d0O7mc
So, /dev/disk/by-path/* does not appear in the layout, only /dev/sd*
does, and dependencies seem to be properly recorded. This is despite
pvdisplay showing /dev/disk/by-path/* paths. I believe the layout code
performs enough symlink resolution now.
Or can you describe me how you installed your system so that it uses by-path PVs
so that I can relatively easily reproduce it?
LVM configuration described in https://github.com/rear/rear/pull/2903: add
filter = [ "r|/dev/disk/by-path/.*-usb-|", "a|/dev/disk/by-path/pci-.*-nvme-|", "a|/dev/disk/by-path/pci-.*-scsi-|", "a|/dev/disk/by-path/pci-.*-ata-|", "a|/dev/disk/by-path/pci-.*-sas-|", "a|loop|", "r|.*|" ]
to /etc/lvm/lvm.conf.
https://github.com/rear/rear/commit/ea2b700ef16ed1dc46d46ad472c6ee91fc38ba30#commitcomment-31237351
PV size is human readable and not in bytes :(
solved by --units=b.
[Export of Github issue for rear/rear.]