#3455 PR merged: EXCLUDE_COMPONENTS in 200_partition_layout.sh¶
Labels: enhancement, fixed / solved / done
jsmeix opened issue at 2025-04-15 14:47:¶
- Type: Enhancement
The intent is to provide a feasible way out for the user
in particular for exceptional cases when there is
an unwanted and unneeded disk in his system
which causes trouble for ReaR - for example as in
https://github.com/rear/rear/issues/2995 and
https://github.com/rear/rear/issues/3433
where parted failed because of incorrect partitioning
which as a consequence makes ReaR fail.
-
Impact: Normal
-
Reference to related issues (URLs):
https://github.com/rear/rear/issues/2995
https://github.com/rear/rear/issues/3433
- Description of the changes in this pull request:
My current changes are described below starting at
https://github.com/rear/rear/pull/3455#issuecomment-2821188937
My initial and meanwhile outdated changes
caused a regression which I described below in
https://github.com/rear/rear/pull/3455#issuecomment-2812436548
Because of the regression
I had to completely change the implementation
from my initial dirty hack makeshift
towards a (hopefully) proper solution.
My initial and meanwhile outdated changes were:
In layout/save/GNU/Linux/200_partition_layout.sh
added a makeshift to exclude unwanted disks
from the very beginning via EXCLUDE_COMPONENTS
to help the user to avoid issues with unneeded disks
where the subsequent code fails e.g. as in
https://github.com/rear/rear/issues/2995
where parted did not recognize a partition table
that is recognized both by the kernel and fdisk
or as in
https://github.com/rear/rear/issues/3433
where also parted failed for an unused disk
which has somewhat incorrect partitioning.
- How was this pull request tested?
For the test on a SLES15 VM
I added second disk sdb with two partitions
each one with ext4 filesystem that is mounted
# lsblk -ipo NAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINTS
NAME TRAN TYPE FSTYPE SIZE MOUNTPOINTS
/dev/sda ata disk 15G
|-/dev/sda1 part 8M
|-/dev/sda2 part btrfs 13G /var
| /usr/local
| /tmp
| /root
| /srv
| /opt
| /home
| /boot/grub2/x86_64-efi
| /boot/grub2/i386-pc
| /.snapshots
| /
`-/dev/sda3 part swap 2G [SWAP]
/dev/sdb ata disk 2G
|-/dev/sdb1 part ext4 1020M /mountpoint1
`-/dev/sdb2 part ext4 512M /mountpoint2
/dev/sr0 ata rom iso9660 14.1G
My current test results are described below starting at
https://github.com/rear/rear/pull/3455#issuecomment-2821459892
and subsequent comments.
My meanwhile outdated test results from my initial changes were:
My local.conf contains
EXCLUDE_COMPONENTS+=( /dev/sdb fs:/mountpoint1 fs:/mountpoint2 )
With that I get
# usr/sbin/rear -D mkrescue
...
Running 'layout/save' stage ======================
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Skipping /dev/sdb in EXCLUDE_COMPONENTS (does not also exclude mounted filesystems on it)
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/sda2 (default subvolume path contains '@/.snapshots/')
Added /dev/sda2 to BTRFS_SUBVOLUME_SLES_SETUP in /var/tmp/rear.VDl9jDl9fKeBaAX/rootfs/etc/rear/rescue.conf
Excluding component /dev/sdb in EXCLUDE_COMPONENTS
Excluding component fs:/mountpoint1 in EXCLUDE_COMPONENTS
Marking component 'fs:/mountpoint1' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Excluding component fs:/mountpoint2 in EXCLUDE_COMPONENTS
Marking component 'fs:/mountpoint2' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /mountpoint1' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /mountpoint2' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Skip saving storage layout as 'barrel' devicegraph (no 'barrel' command)
Verifying that the entries in /root/rear.github.master/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master/var/lib/rear/layout/disklayout.conf)
...
# cat var/lib/rear/layout/disktodo.conf
todo /dev/sda disk
todo /dev/sda1 part
todo /dev/sda2 part
todo /dev/sda3 part
todo fs:/ fs
done fs:/mountpoint1 fs
done fs:/mountpoint2 fs
todo btrfsmountedsubvol:/ btrfsmountedsubvol
todo btrfsmountedsubvol:/.snapshots btrfsmountedsubvol
todo btrfsmountedsubvol:/boot/grub2/i386-pc btrfsmountedsubvol
todo btrfsmountedsubvol:/boot/grub2/x86_64-efi btrfsmountedsubvol
todo btrfsmountedsubvol:/home btrfsmountedsubvol
todo btrfsmountedsubvol:/opt btrfsmountedsubvol
todo btrfsmountedsubvol:/srv btrfsmountedsubvol
todo btrfsmountedsubvol:/root btrfsmountedsubvol
todo btrfsmountedsubvol:/tmp btrfsmountedsubvol
todo btrfsmountedsubvol:/usr/local btrfsmountedsubvol
todo btrfsmountedsubvol:/var btrfsmountedsubvol
todo swap:/dev/sda3 swap
# cat var/lib/rear/layout/disklayout.conf
...
disk /dev/sda 16106127360 gpt
...
part /dev/sda 8388608 1048576 rear-noname bios_grub /dev/sda1
part /dev/sda 13949206528 9437184 rear-noname legacy_boot /dev/sda2
part /dev/sda 2147466752 13958643712 rear-noname swap /dev/sda3
# Skipped /dev/sdb in EXCLUDE_COMPONENTS (does not also exclude mounted filesystems on it)
...
fs /dev/sda2 / btrfs uuid=bdec53c2-1ee8-4268-90f9-5ec523774035 label= ...
#fs /dev/sdb1 /mountpoint1 ext4 uuid=058cb383-20e1-4237-abf5-c53ce27325ae label= ...
#fs /dev/sdb2 /mountpoint2 ext4 uuid=670b47d9-31b3-4483-9409-43116e16b0ac label= ..
...
swap /dev/sda3 uuid=921157bc-e4d6-4869-8796-7e09207e49a9 label=
I also did "rear mkbackuponly"
and then tested "rear recover"
on another VM without a second disk sdb
and - at least for me - it "just worked"
(i.e. nothing about 'sdb' during "rear recover")
so sdb is really completely skipped for recovery.
jsmeix commented at 2025-04-17 10:24:¶
The changes in this pull request result a different behaviour
compared to how it was before.
Reason:
Regardless that before this pull request conf/default.conf reads
# You cannot exclude a device (e.g. /dev/sdg) directly. Instead you have to exclude everything
# ON that device and then the dependency tracker will automatically exclude the device from the
# recovery (because there won't be any recovery information for that "unnecessary" device).
one can exclude a device (e.g. /dev/sdg) directly
and then the dependency tracker will automatically exclude
also at least partitions and filesystems on that device.
On my same SLES15 test VM as above
with current GitHub master code
I get with EXCLUDE_COMPONENTS+=( /dev/sdb ) in local.conf
the following behaviour
# usr/sbin/rear -D mkrescue
...
Running 'layout/save' stage ======================
Creating disk layout
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/sda2 (default subvolume path contains '@/.snapshots/')
Added /dev/sda2 to BTRFS_SUBVOLUME_SLES_SETUP in /var/tmp/rear.IeF7CQWDVk4rxWe/rootfs/etc/rear/rescue.conf
Excluding component /dev/sdb in EXCLUDE_COMPONENTS
Marking component '/dev/sdb' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb1 is a child of component /dev/sdb
Marking component '/dev/sdb1' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb2 is a child of component /dev/sdb
Marking component '/dev/sdb2' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint1 is a child of component /dev/sdb
Marking component 'fs:/mountpoint1' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint2 is a child of component /dev/sdb
Marking component 'fs:/mountpoint2' as done in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Automatically excluding disk /dev/sdb (not used by any mounted filesystem)
Component '/dev/sdb' is marked as 'done /dev/sdb' in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb1 is a child of component /dev/sdb
Component '/dev/sdb1' is marked as 'done /dev/sdb1' in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb2 is a child of component /dev/sdb
Component '/dev/sdb2' is marked as 'done /dev/sdb2' in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint1 is a child of component /dev/sdb
Component 'fs:/mountpoint1' is marked as 'done fs:/mountpoint1' in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint2 is a child of component /dev/sdb
Component 'fs:/mountpoint2' is marked as 'done fs:/mountpoint2' in /root/rear.github.master/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdb' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdb' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /mountpoint1' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /mountpoint2' in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Skip saving storage layout as 'barrel' devicegraph (no 'barrel' command)
Verifying that the entries in /root/rear.github.master/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master/var/lib/rear/layout/disklayout.conf)
...
# cat var/lib/rear/layout/disktodo.conf
todo /dev/sda disk
todo /dev/sda1 part
todo /dev/sda2 part
todo /dev/sda3 part
done /dev/sdb disk
done /dev/sdb1 part
done /dev/sdb2 part
todo fs:/ fs
done fs:/mountpoint1 fs
done fs:/mountpoint2 fs
todo btrfsmountedsubvol:/ btrfsmountedsubvol
todo btrfsmountedsubvol:/.snapshots btrfsmountedsubvol
todo btrfsmountedsubvol:/boot/grub2/i386-pc btrfsmountedsubvol
todo btrfsmountedsubvol:/boot/grub2/x86_64-efi btrfsmountedsubvol
todo btrfsmountedsubvol:/home btrfsmountedsubvol
todo btrfsmountedsubvol:/opt btrfsmountedsubvol
todo btrfsmountedsubvol:/srv btrfsmountedsubvol
todo btrfsmountedsubvol:/root btrfsmountedsubvol
todo btrfsmountedsubvol:/tmp btrfsmountedsubvol
todo btrfsmountedsubvol:/usr/local btrfsmountedsubvol
todo btrfsmountedsubvol:/var btrfsmountedsubvol
todo swap:/dev/sda3 swap
# grep sdb var/lib/rear/layout/disklayout.conf
# /dev/sdb /dev/sdb ata disk 2G
# |-/dev/sdb1 /dev/sdb1 /dev/sdb part ext4 1020M /mountpoint1 058cb383-20e1-4237-abf5-c53ce27325ae
# `-/dev/sdb2 /dev/sdb2 /dev/sdb part ext4 512M /mountpoint2 670b47d9-31b3-4483-9409-43116e16b0ac
# Disk /dev/sdb
#disk /dev/sdb 2147483648 gpt
# Partitions on /dev/sdb
#part /dev/sdb 1069547520 4194304 other none /dev/sdb1
#part /dev/sdb 536870912 1073741824 other none /dev/sdb2
#fs /dev/sdb1 /mountpoint1 ext4 uuid=058cb383-20e1-4237-abf5-c53ce27325ae label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
#fs /dev/sdb2 /mountpoint2 ext4 uuid=670b47d9-31b3-4483-9409-43116e16b0ac label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
So with current GitHub master code
EXCLUDE_COMPONENTS+=( /dev/sdb )
perfectly excludes /dev/sdb plus automatically
also at least partitions and filesystems on it.
I guess it won't automatically exclue higher level
storage objects like LVM or things like that.
In contrast with the changes in this pull request I get
(same EXCLUDE_COMPONENTS+=( /dev/sdb ) in local.conf)
# usr/sbin/rear -D mkrescue
...
Running 'layout/save' stage ======================
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Skipping /dev/sdb in EXCLUDE_COMPONENTS (does not automatically exclude mounted filesystems on it)
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/sda2 (default subvolume path contains '@/.snapshots/')
Added /dev/sda2 to BTRFS_SUBVOLUME_SLES_SETUP in /var/tmp/rear.QNG4GunBgtzxOLm/rootfs/etc/rear/rescue.conf
Excluding component /dev/sdb in EXCLUDE_COMPONENTS
Disabling excluded components in /root/rear.github.master/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Skip saving storage layout as 'barrel' devicegraph (no 'barrel' command)
Verifying that the entries in /root/rear.github.master/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master/var/lib/rear/layout/disklayout.conf)
...
# cat var/lib/rear/layout/disktodo.conf
todo /dev/sda disk
todo /dev/sda1 part
todo /dev/sda2 part
todo /dev/sda3 part
todo fs:/ fs
todo fs:/mountpoint1 fs
todo fs:/mountpoint2 fs
todo btrfsmountedsubvol:/ btrfsmountedsubvol
todo btrfsmountedsubvol:/.snapshots btrfsmountedsubvol
todo btrfsmountedsubvol:/boot/grub2/i386-pc btrfsmountedsubvol
todo btrfsmountedsubvol:/boot/grub2/x86_64-efi btrfsmountedsubvol
todo btrfsmountedsubvol:/home btrfsmountedsubvol
todo btrfsmountedsubvol:/opt btrfsmountedsubvol
todo btrfsmountedsubvol:/srv btrfsmountedsubvol
todo btrfsmountedsubvol:/root btrfsmountedsubvol
todo btrfsmountedsubvol:/tmp btrfsmountedsubvol
todo btrfsmountedsubvol:/usr/local btrfsmountedsubvol
todo btrfsmountedsubvol:/var btrfsmountedsubvol
todo swap:/dev/sda3 swap
# grep sdb var/lib/rear/layout/disklayout.conf
# /dev/sdb /dev/sdb ata disk 2G
# |-/dev/sdb1 /dev/sdb1 /dev/sdb part ext4 1020M /mountpoint1 058cb383-20e1-4237-abf5-c53ce27325ae
# `-/dev/sdb2 /dev/sdb2 /dev/sdb part ext4 512M /mountpoint2 670b47d9-31b3-4483-9409-43116e16b0ac
# Skipped /dev/sdb in EXCLUDE_COMPONENTS (does not automatically exclude mounted filesystems on it)
fs /dev/sdb1 /mountpoint1 ext4 uuid=058cb383-20e1-4237-abf5-c53ce27325ae label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
fs /dev/sdb2 /mountpoint2 ext4 uuid=670b47d9-31b3-4483-9409-43116e16b0ac label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16384 default_mount_options=user_xattr,acl options=rw,relatime
So with the changes in this pull request
EXCLUDE_COMPONENTS+=( /dev/sdb )
only excludes /dev/sdb and the partitions on it
because with the changes in this pull request
handling /dev/sdb is skipped in 200_partition_layout.sh
so neither the disk nor its partitions appear in disklayout.conf
(because extract_partitions() is not called for /dev/sdb)
but e.g. 230_filesystem_layout.sh runs normally and includes
the mounted filesystems on /dev/sdb in disklayout.conf
which also appear as 'todo' in disktodo.conf
so with the changes in this pull request plain
EXCLUDE_COMPONENTS+=( /dev/sdb )
does no longer work as before and one must use e.g.
EXCLUDE_COMPONENTS+=( /dev/sdb fs:/mountpoint1 fs:/mountpoint2 )
to also get the mounted filesystems on /dev/sdb excluded.
jsmeix commented at 2025-04-22 12:34:¶
My current offhanded (in particular not yet tested) idea
how to avoid the regression which I described above in
https://github.com/rear/rear/pull/3455#issuecomment-2812436548
is the following:
With e.g.
EXCLUDE_COMPONENTS+=( /dev/sdb )
do not skip '/dev/sdb' in 200_partition_layout.sh
but instead use EXCLUDE_COMPONENTS+=( /dev/sdb )
to avoid an Error() exit in 200_partition_layout.sh
in particular when extract_partitions() is called
for a disk which is listed in EXCLUDE_COMPONENTS.
My reasoning behind is:
When a disk is listed in EXCLUDE_COMPONENTS
it should not matter when 'parted' fails
to recognize the disk and/or its partitions.
When 'parted' works to recognize the disk and its partitions
then all works well with current ReaR via
EXCLUDE_COMPONENTS+=( /dev/sdb ) as in my case in
https://github.com/rear/rear/pull/3455#issuecomment-2812436548
When 'parted' fails to recognize a disk and/or partitions
(i.e. when the partitioning of the disk is "problematic")
and the "problematic" disk is listed in EXCLUDE_COMPONENTS
then e.g. EXCLUDE_COMPONENTS+=( /dev/sdb ) should work
when there is only the disk without mounted filesystems.
But when there are mounted filesystems on a "problematic" disk
the user may have to also specify the mounted filesystems like
EXCLUDE_COMPONENTS+=( /dev/sdb fs:/mountpoint1 fs:/mountpoint2 )
to get a "problematic" disk and all what there is on it excluded.
The point is that ReaR should not error out when a
"problematic" disk is listed in EXCLUDE_COMPONENTS
so that EXCLUDE_COMPONENTS can be used in particular
to exclude an unneeded "problematic" disk.
jsmeix commented at 2025-04-22 14:07:¶
With my recent changes EXCLUDE_COMPONENTS+=( /dev/sdb )
works again as before:
# lsblk -ipo NAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINTS /dev/sdb
NAME TRAN TYPE FSTYPE SIZE MOUNTPOINTS
/dev/sdb ata disk 2G
|-/dev/sdb1 part ext4 1020M /mountpoint1
`-/dev/sdb2 part ext4 512M /mountpoint2
# usr/sbin/rear -D mkrescue
...
Running 'layout/save' stage ======================
Creating disk layout
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/sda2 (default subvolume path contains '@/.snapshots/')
Added /dev/sda2 to BTRFS_SUBVOLUME_SLES_SETUP in /var/tmp/rear.YOcC0DIV8HstDBs/rootfs/etc/rear/rescue.conf
Excluding component /dev/sdb in EXCLUDE_COMPONENTS
Marking component '/dev/sdb' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb1 is a child of component /dev/sdb
Marking component '/dev/sdb1' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb2 is a child of component /dev/sdb
Marking component '/dev/sdb2' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint1 is a child of component /dev/sdb
Marking component 'fs:/mountpoint1' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint2 is a child of component /dev/sdb
Marking component 'fs:/mountpoint2' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Automatically excluding disk /dev/sdb (not used by any mounted filesystem)
Component '/dev/sdb' is marked as 'done /dev/sdb' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb1 is a child of component /dev/sdb
Component '/dev/sdb1' is marked as 'done /dev/sdb1' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb2 is a child of component /dev/sdb
Component '/dev/sdb2' is marked as 'done /dev/sdb2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint1 is a child of component /dev/sdb
Component 'fs:/mountpoint1' is marked as 'done fs:/mountpoint1' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint2 is a child of component /dev/sdb
Component 'fs:/mountpoint2' is marked as 'done fs:/mountpoint2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdb' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdb' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /mountpoint1' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /mountpoint2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Skip saving storage layout as 'barrel' devicegraph (no 'barrel' command)
Verifying that the entries in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf)
...
# grep ^done var/lib/rear/layout/disktodo.conf
done /dev/sdb disk
done /dev/sdb1 part
done /dev/sdb2 part
done fs:/mountpoint1 fs
done fs:/mountpoint2 fs
# grep sdb var/lib/rear/layout/disklayout.conf | cut -b-80
# /dev/sdb /dev/sdb ata disk
# |-/dev/sdb1 /dev/sdb1 /dev/sdb part ext4
# `-/dev/sdb2 /dev/sdb2 /dev/sdb part ext4
# Disk /dev/sdb
#disk /dev/sdb 2147483648 gpt
# Partitions on /dev/sdb
#part /dev/sdb 1069547520 4194304 other none /dev/sdb1
#part /dev/sdb 536870912 1073741824 other none /dev/sdb2
#fs /dev/sdb1 /mountpoint1 ext4 uuid=058cb383-20e1-4237-abf5-c53ce27325ae label= ...
#fs /dev/sdb2 /mountpoint2 ext4 uuid=670b47d9-31b3-4483-9409-43116e16b0ac label= ...
jsmeix commented at 2025-04-22 14:15:¶
What I cannot test is how it behaves with my recent changes
when there is a "problematic" disk that should be excluded
i.e. a disk that cannot be recognized properly by 'parted'
because I don't know how I could setup such a "problematic" disk
for my KVM/QEMU test VM with Virtual Machine Manager.
jsmeix commented at 2025-04-23 11:29:¶
With the help of
https://github.com/rear/rear/issues/2995#issuecomment-1598782245
I could setup such a "problematic" disk
that cannot be recognized properly by 'parted'
on my KVM/QEMU test VM:
I downloaded ubuntu-24.04.1-desktop-amd64.iso from
https://old-releases.ubuntu.com/releases/20.04.4/
On my KVM/QEMU test VM I created a disk 'sdc' with 1GiB size.
I did 'dd' the ubuntu-24.04.1-desktop-amd64.iso onto /dev/sdc
# dd if=ubuntu-20.04-beta-live-server-amd64.iso of=/dev/sdc bs=1MiB status=progress
880803840 bytes (881 MB, 840 MiB) copied, 1 s, 880 MB/s
918+0 records in
918+0 records out
962592768 bytes (963 MB, 918 MiB) copied, 1.79387 s, 537 MB/s
The disk cannot be properly recognized by 'parted'
# parted -s /dev/sdc unit MiB print
Warning: The driver descriptor says the physical block size is 2048 bytes, but Linux says it is 512 bytes.
Model: ATA QEMU HARDDISK (scsi)
Disk /dev/sdc: 1024MiB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
In particular 'parted' shows Partition Table: unknown
which leads to the message in ReaR
Unsupported partition table 'unknown' on /dev/sdc (must be one of 'msdos' 'gpt' 'gpt_sync_mbr' 'dasd')
But 'fdisk' can recognize it
# fdisk -l /dev/sdc
Disk /dev/sdc: 1 GiB, 1073741824 bytes, 2097152 sectors
Disk model: QEMU HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x53c95e79
Device Boot Start End Sectors Size Id Type
/dev/sdc1 * 0 1880063 1880064 918M 0 Empty
/dev/sdc2 20464 28463 8000 3.9M ef EFI (FAT-12/16/32)
Also 'lsblk' can list it
# lsblk -ipo NAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINTS /dev/sdc
NAME TRAN TYPE FSTYPE SIZE MOUNTPOINTS
/dev/sdc ata disk iso9660 1G
|-/dev/sdc1 part iso9660 918M
`-/dev/sdc2 part vfat 3.9M
With EXCLUDE_COMPONENTS+=( /dev/sdc ) in local.conf I got
# usr/sbin/rear -D mkrescue
...
Running 'layout/save' stage ======================
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Unsupported partition table 'unknown' on /dev/sdc (must be one of 'msdos' 'gpt' 'gpt_sync_mbr' 'dasd')
Invalid 'part /dev/sdc' entry (at least one value is missing)
Invalid 'part /dev/sdc' entry (at least one value is missing)
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/sda2 (default subvolume path contains '@/.snapshots/')
Added /dev/sda2 to BTRFS_SUBVOLUME_SLES_SETUP in /var/tmp/rear.QInIUCf5UvAn72D/rootfs/etc/rear/rescue.conf
Excluding component /dev/sdc in EXCLUDE_COMPONENTS
Marking component '/dev/sdc' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc1 is a child of component /dev/sdc
Marking component '/dev/sdc1' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc2 is a child of component /dev/sdc
Marking component '/dev/sdc2' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Automatically excluding disk /dev/sdc (not used by any mounted filesystem)
Component '/dev/sdc' is marked as 'done /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc1 is a child of component /dev/sdc
Component '/dev/sdc1' is marked as 'done /dev/sdc1' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc2 is a child of component /dev/sdc
Component '/dev/sdc2' is marked as 'done /dev/sdc2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdc' is disabled in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Skip saving storage layout as 'barrel' devicegraph (no 'barrel' command)
Verifying that the entries in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf)
...
# grep ^done var/lib/rear/layout/disktodo.conf
done /dev/sdc disk
done /dev/sdc1 part
done /dev/sdc2 part
# grep sdc var/lib/rear/layout/disklayout.conf
# /dev/sdc /dev/sdc ata disk iso9660 Ubuntu-Server 20.04 LTS amd64 1G 2020-04-02-13-22-33-00
# |-/dev/sdc1 /dev/sdc1 /dev/sdc part iso9660 Ubuntu-Server 20.04 LTS amd64 918M 2020-04-02-13-22-33-00
# `-/dev/sdc2 /dev/sdc2 /dev/sdc part vfat Ubuntu-Server 20.04 LTS amd64 3.9M 63A9-90E1
# Disk /dev/sdc
#disk /dev/sdc 1073741824 unknown
# Partitions on /dev/sdc
#part /dev/sdc 962592768 0 none /dev/sdc1
So with plain EXCLUDE_COMPONENTS+=( /dev/sdX )
an unneeded problematic disk which is not properly recognized
gets excluded plus automatically the partitions on it.
Then I tried how it behaves when there is a mounted filesystem
on such a problematic disk which is not properly recognized:
# mkdir /mountpoint_sdc2
# mount /dev/sdc2 /mountpoint_sdc2
# lsblk -ipo NAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINTS /dev/sdc
NAME TRAN TYPE FSTYPE SIZE MOUNTPOINTS
/dev/sdc ata disk iso9660 1G
|-/dev/sdc1 part iso9660 918M
`-/dev/sdc2 part vfat 3.9M /mountpoint_sdc2
# usr/sbin/rear -D mkrescue
...
Running 'layout/save' stage ======================
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Unsupported partition table 'unknown' on /dev/sdc (must be one of 'msdos' 'gpt' 'gpt_sync_mbr' 'dasd')
Invalid 'part /dev/sdc' entry (at least one value is missing)
Invalid 'part /dev/sdc' entry (at least one value is missing)
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/sda2 (default subvolume path contains '@/.snapshots/')
Added /dev/sda2 to BTRFS_SUBVOLUME_SLES_SETUP in /var/tmp/rear.iEnqI2gJQrwiQDg/rootfs/etc/rear/rescue.conf
Excluding component /dev/sdc in EXCLUDE_COMPONENTS
Marking component '/dev/sdc' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc1 is a child of component /dev/sdc
Marking component '/dev/sdc1' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc2 is a child of component /dev/sdc
Marking component '/dev/sdc2' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint_sdc2 is a child of component /dev/sdc
Marking component 'fs:/mountpoint_sdc2' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Automatically excluding disk /dev/sdc (not used by any mounted filesystem)
Component '/dev/sdc' is marked as 'done /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc1 is a child of component /dev/sdc
Component '/dev/sdc1' is marked as 'done /dev/sdc1' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc2 is a child of component /dev/sdc
Component '/dev/sdc2' is marked as 'done /dev/sdc2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component fs:/mountpoint_sdc2 is a child of component /dev/sdc
Component 'fs:/mountpoint_sdc2' is marked as 'done fs:/mountpoint_sdc2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdc' is disabled in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'fs ... /mountpoint_sdc2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Skip saving storage layout as 'barrel' devicegraph (no 'barrel' command)
Verifying that the entries in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf)
...
# grep sdc var/lib/rear/layout/disktodo.conf
done /dev/sdc disk
done /dev/sdc1 part
done /dev/sdc2 part
done fs:/mountpoint_sdc2 fs
# grep sdc var/lib/rear/layout/disklayout.conf | cut -b-112
# /dev/sdc /dev/sdc ata disk iso9660 Ubuntu-Server 20.04 LTS amd64 1G
# |-/dev/sdc1 /dev/sdc1 /dev/sdc part iso9660 Ubuntu-Server 20.04 LTS amd64 918M
# `-/dev/sdc2 /dev/sdc2 /dev/sdc part vfat Ubuntu-Server 20.04 LTS amd64 3.9M /mountpoint_sdc2
# Disk /dev/sdc
#disk /dev/sdc 1073741824 unknown
# Partitions on /dev/sdc
#part /dev/sdc 962592768 0 none /dev/sdc1
#part /dev/sdc 4096000 10477568 none /dev/sdc2
#fs /dev/sdc2 /mountpoint_sdc2 vfat uuid=63A9-90E1 label= options=rw,relatime,fmask=0022,dmask=0022,codepage=437
So with plain EXCLUDE_COMPONENTS+=( /dev/sdX )
a problematic disk which is not properly recognized
gets excluded plus automatically the partitions on it
and even also automatically its mounted filesystems.
I assume one has to exclude higher level storage objects
e.g. MD devices or LVM2 volume groups and things like that
separately as described in default.conf
jsmeix commented at 2025-04-23 14:00:¶
An addendum FYI:
When a disk has no mounted filesystem
the whole disk would get normally automatically excluded via
Automatically excluding disk /dev/sdX (not used by any mounted filesystem)
BUT
for a problematic disk which is not properly recognized
this does not work without EXCLUDE_COMPONENTS+=( /dev/sdX )
because without EXCLUDE_COMPONENTS+=( /dev/sdX )
ReaR errors out with
ERROR: Unsupported partition table 'unknown' on /dev/sdX (must be one of 'msdos' 'gpt' 'gpt_sync_mbr' 'dasd')
because for a disk which is not properly recognized
it is not possible to safely determine whether or not
this disk is really not used by any mounted filesystem.
To be on the safe side "rear mkrescue/mkbackup" must error out
unless the user had explicitly specified that /dev/sdX
should be excluded from the recovery, cf.
https://github.com/rear/rear/wiki/Coding-Style#try-hard-to-care-about-possible-errors
How that looks for me on my test VM
when I have no mounted filesystem
on the normal disk /dev/sdb
and on the problematic disk /dev/sdc
with EXCLUDE_COMPONENTS+=( /dev/sdc )
# lsblk -ipo NAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINTS
NAME TRAN TYPE FSTYPE SIZE MOUNTPOINTS
/dev/sda ata disk 15G
|-/dev/sda1 part 8M
|-/dev/sda2 part btrfs 13G /var
| /usr/local
| /tmp
| /root
| /srv
| /opt
| /home
| /boot/grub2/x86_64-efi
| /boot/grub2/i386-pc
| /.snapshots
| /
`-/dev/sda3 part swap 2G [SWAP]
/dev/sdb ata disk 2G
|-/dev/sdb1 part ext4 1020M
`-/dev/sdb2 part ext4 512M
/dev/sdc ata disk iso9660 1G
|-/dev/sdc1 part iso9660 918M
`-/dev/sdc2 part vfat 3.9M
/dev/sr0 ata rom iso9660 14.1G
# usr/sbin/rear -D mkrescue
...
Running 'layout/save' stage ======================
Creating disk layout
Overwriting existing disk layout file /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Unsupported partition table 'unknown' on /dev/sdc (must be one of 'msdos' 'gpt' 'gpt_sync_mbr' 'dasd')
Invalid 'part /dev/sdc' entry (at least one value is missing)
Invalid 'part /dev/sdc' entry (at least one value is missing)
SLES12-SP1 (and later) btrfs subvolumes setup needed for /dev/sda2 (default subvolume path contains '@/.snapshots/')
Added /dev/sda2 to BTRFS_SUBVOLUME_SLES_SETUP in /var/tmp/rear.qSQvOwMEG3sSmPn/rootfs/etc/rear/rescue.conf
Excluding component /dev/sdc in EXCLUDE_COMPONENTS
Marking component '/dev/sdc' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc1 is a child of component /dev/sdc
Marking component '/dev/sdc1' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc2 is a child of component /dev/sdc
Marking component '/dev/sdc2' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Automatically excluding disk /dev/sdb (not used by any mounted filesystem)
Marking component '/dev/sdb' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb1 is a child of component /dev/sdb
Marking component '/dev/sdb1' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdb2 is a child of component /dev/sdb
Marking component '/dev/sdb2' as done in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Automatically excluding disk /dev/sdc (not used by any mounted filesystem)
Component '/dev/sdc' is marked as 'done /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc1 is a child of component /dev/sdc
Component '/dev/sdc1' is marked as 'done /dev/sdc1' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Dependent component /dev/sdc2 is a child of component /dev/sdc
Component '/dev/sdc2' is marked as 'done /dev/sdc2' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disktodo.conf
Disabling excluded components in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdb' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdb' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdb' is disabled in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'disk /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Disabling component 'part /dev/sdc' in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Component 'part /dev/sdc' is disabled in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf
Using sysconfig bootloader 'grub2' for 'rear recover'
Skip saving storage layout as 'barrel' devicegraph (no 'barrel' command)
Verifying that the entries in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf are correct
Created disk layout (check the results in /root/rear.github.master.jsmeix-disk-EXCLUDE_COMPONENTS/var/lib/rear/layout/disklayout.conf)
...
The excluding happens primarily
first via layout/save/default/310_include_exclude.sh
and then via layout/save/default/320_autoexclude.sh
and layout/save/default/330_remove_exclusions.sh
jsmeix commented at 2025-04-24 08:24:¶
@rear/contributors
I would like to merge it tomorrow afternoon
provided there are no severe objections.
[Export of Github issue for rear/rear.]