#2229 Issue closed: Including components in disklayout.conf¶
Labels: enhancement, documentation, cleanup,
no-issue-activity
adatum opened issue at 2019-09-05 18:02:¶
Relax-and-Recover 2.5-git.3350.aa82834d.unknown / 2019-05-10
For some reason there is a filesystem that is detected by ReaR but excluded (commented out) automatically in disklayout.conf. Its lvmvol is included though. Why is this and how can I make sure that filesystem is included and not commented out?
A workaround would be to uncomment it in a restore situation, but of course I'd prefer to avoid such surprises.
The documentation mentions "Including/Excluding components" and shows examples of exclusions, but nothing about inclusions.
This question is in the same vein as #2215 but whereas that issue is about excluding temporary mounts, this is about including permanent filesystems which happen to be on a different filesystem and volume.
jsmeix commented at 2019-09-10 14:11:¶
@adatum
I am currently and for some more weeks not in the office
so that I cannot try out how ReaR actually behaves.
Therefore some info off the top of my head:
By default ReaR includes what is mounted
while "rear mkrescue/mkbackup" is running
except some automatically excluded things,
see in particular the scripts
usr/share/rear/layout/save/default/310_autoexclude_usb.sh
and
usr/share/rear/layout/save/default/320_autoexclude.sh
Run "rear -D mkrescue" (-D is debugscript mode)
and inspect the log file what exactly happens while
usr/share/rear/layout/save/default/320_autoexclude.sh
is running - that should (hopefully) show why a particular
component is automatically excluded in your particular case,
cf. the section "Debugging issues with Relax-and-Recover" in
https://en.opensuse.org/SDB:Disaster_Recovery
Also see default.conf for some AUTOEXCLUDE_* variables
FYI:
To see the exact syntax of components in your particular case
it should help to inspect after "rear mkrescue" was run
the var/lib/rear/layout/diskdeps.conf file that lists the
components for example something like
/dev/sda1 /dev/sda
/dev/sda2 /dev/sda
/dev/sda3 /dev/sda
/dev/sda5 /dev/sda
fs:/ /dev/sda5
swap:/dev/sda1 /dev/sda1
Also var/lib/rear/layout/disktodo.conf lists the components.
adatum commented at 2019-09-13 02:26:¶
Thanks @jsmeix that was it: AUTOEXCLUDE_PATH by default excludes
/media which is where my disappearing filesystem is mounted.
# Automatically exclude filesystems mounted under directories given here
# The default is /media to exclude USB devices mounted there.
# This is different from EXCLUDE_MOUNTPOINTS, which accepts only mountpoints.
AUTOEXCLUDE_PATH=( /media )
Incidentally, this functionality is exactly what I was asking for in
#2215. On my system, USB devices are mounted at /run/media.
Changing to AUTOEXCLUDE_PATH=( /run/media /mnt ) avoids commenting out
components I want, and also automatically excludes any temporary USB
devices. Perfect.
The only detail is that wildcards again seem not supported here. For
example, Veracrypt mounts volumes as /media/veracrypt1,
/media/veracrypt2, etc which I would also want to exclude with
/media/veracrypt* but that does not work.
Perhaps an alternative could be to exclude /media and explicitly
include /media/directory-i-want.
However, I still do not see how to explicitly include components despite the documentation and even defaults.conf mentioning it:
# Explicitly excluding/including devices is generally a safer option.
jsmeix commented at 2019-09-13 14:43:¶
@adatum
From my experience with other issues the backup include/exclude
functionality
versus the components include/exclude functionality in ReaR is "hairy",
cf.
https://github.com/rear/rear/issues/2236#issuecomment-531204474
Off the top of my head I don't know if something like
AUTOEXCLUDE_PATH=( /media /run/media /mnt )
cf.
https://github.com/rear/rear/issues/2239
plus some other setting to explicitly include some
/media/directory-i-want
actually works in practice.
adatum commented at 2019-09-13 16:42:¶
@jsmeix Note that this discussion of include/exclude functionality relates to mkrescue. Is that also "hairy"? I am not using ReaR for backups.
While changing the arguments for automatic exclusion works, I have not
tested if explicitly including some /media/directory-i-want succeeds
because I have found no way to explicitly specify inclusions. Is this
described anywhere?
jsmeix commented at 2019-09-14 12:13:¶
@adatum
the include/exclude functionality related to "mkrescue"
is the disk layout components include/exclude functionality.
From my experience the components include/exclude functionality
alone is also "hairy" when you need extraordinary things
but it is less "hairy" than both the backup include/exclude
functionality
together with the components include/exclude functionality ;-)
The reson why in my experience the components include/exclude
functionality
is "hairy" is that I did not find a way how to explicitly specify what
to include.
From my experience things work well as long as ReaR's harcoded default
behaviour what disk layout components get included is what you need
but I do not know how to enforce ReaR to include a disk layout
component
that is not included by default.
As far as I know from my experience ReaR includes by default all what
belongs
to mounted filesystems (and automatically excludes some "known to not
needed"
things).
But assume you have a second harddisk /dev/sdb with partitions
/dev/sdb1
and /dev/sdb2 with some filesystems on them but none of them is
mounted
(for example because you do not have files in them or for whatever
reason)
but you want ReaR to also recreate /dev/sdb with its partitions and
filesystems.
Off the top of my head I do not know how to tell ReaR to do that.
As far as I know the reason behind is that the basic disk layout code
is meanwhile somewhat old and I think at the time when it was made
the main intent was to make ReaR "do what is needed automatically"
(often as requested by particular companies or organizations who paid
for what they specifically need) but the intent was not so much
to let arbitrary users specify everything.
So certain config options are currently missing.
This is against my personal preference that is
"provide final power to the user"
which means that basically for all and everything there must be a config
variable
that is usually empty which means ReaR does its automated default
behaviour
but when needed the user can specify what he wants and then ReaR must
obey
even if that means things fail - what the user specified is sacrosanct -
but when
things fail the user will see that his settings do not work.
adatum commented at 2019-09-14 16:46:¶
ReaR's automatic behavior is typically quite sensible. Personally, I too prefer the approach of also "providing final power to the user" but it is understandable if this is not fully developed.
What is difficult to comprehend is how a documented feature that is also recommended in the official default configuration file somehow refers to behavior (explicit inclusions) that is not implemented??
I have to first assume the problem is my ignorance of ReaR, but if it really does not exist... I am shocked if RTFM actually misleads. Manual inclusion, as with exclusion, seems like a fundamental feature.
Either way, I think some clarification on this matter is needed in the documentation.
gdha commented at 2019-09-16 15:33:¶
@adatum Examples are always useful and yes, documentation is running
behind the facts. However, always keep in mind that all rear
contributors are doing this in their FREE time (read: we are not paid
for this at all). Therefore, never make the assumption that we deliver
enterprise support as it is done completely voluntary. That being said
we are always extremely happy with (little) contributions around
documentation (even grammar corrections are gladly accepted).
Even better is writing missing parts, and we are willing to contribute
where-ever we can.
Therefore, next time please do not use the abbreviation "RTFM" in your
comments - it is really frustrating for us to read this and not really
motivating our job. Just say what needs to be said and will try to deal
with it. Can we agree on this?
adatum commented at 2019-09-16 18:07:¶
@gdha I'm not sure my comment was understood. The "RTFM" would be directed towards me. I suppose some prefer not to see it in any direction; I can keep that in mind.
Could you weigh in on whether it is possible to explicitly include components?
I can understand if documentation is running behind the fact. I would find it shocking if it is running ahead of facts.
jsmeix commented at 2019-09-17 11:36:¶
I will try to improve things - as time permits.
I am not in the office currently and for some more weeks
so that I cannot do much for ReaR.
In particular I cannot try out or test anything for ReaR.
I expect to be back in the office at about beginning of October.
But I also expect that I have to do first and foremost other stuff with
higher priority.
Cf.
https://github.com/rear/rear/issues/799#issuecomment-531247109
jsmeix commented at 2019-09-17 12:18:¶
Via
https://github.com/rear/rear/commit/baa7336e08356451a678bc1e63a6a48e04173c34
I removed in default.conf at AUTOEXCLUDE_DISKS the two comment lines
# Explicitly excluding/including devices is generally a safer option.
# (layout code)
because they are not acually helpful and even misleading because
there is currently no config variable to explicitly include devices.
jsmeix commented at 2019-09-17 12:22:¶
@adatum
when you know about other places in the documentation that are
misleading
please tell me about them, ideally via exact links to the line for
example like
https://github.com/rear/rear/blob/master/usr/share/rear/conf/default.conf#L2390
I can do minor fixes in the documentation but what I cannot do
currently
is implementing code e.g. to support explicit includes of disk
components.
adatum commented at 2019-09-17 17:09:¶
@jsmeix Thanks for your responsiveness. No worries, take your time, there is no big rush.
Another mention of include:
https://github.com/rear/rear/blame/master/doc/user-guide/06-layout-configuration.adoc#L77
jsmeix commented at 2019-09-17 19:45:¶
I knew about that section title but I did not see how it was
misleading.
Now I think I understand what can be misunderstood and
made it simpler and more to the point via
https://github.com/rear/rear/commit/dbdfb5fa066fd719f4a9d5db4dd297c7c23cb189
jsmeix commented at 2019-10-01 15:46:¶
I think the misleading parts (i.e. the minor bugs) in the documentation are now fixed.
github-actions commented at 2020-06-27 01:33:¶
Stale issue message
[Export of Github issue for rear/rear.]