060_compare_files.sh¶

Labels: enhancement, fixed / solved / done

jsmeix opened issue at 2023-03-07 12:46:¶

Type: Enhancement
Impact: Normal
Reference to related issue (URL):
https://github.com/rear/rear/issues/2952#issuecomment-1458094671
How was this pull request tested?
see https://github.com/rear/rear/pull/2954#issuecomment-1463864769
Brief description of the changes in this pull request:

In finalize/default/060_compare_files.sh show what will be done so the user understands what possibly subsequent error messages mean

jsmeix commented at 2023-03-09 13:19:¶

With my latest (but not yet tested) changes
it should now look like:

# rear -D recover
...
Recreating directories ...
Checking if certain restored files are consistent with the recreated system
See /var/lib/rear/layout/config/files.md5sum what files are checked
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /etc/rear/local.conf: FAILED
  /etc/rear/os.conf: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files ...

Now it is no longer judged to be an actual 'Error' and
'do not fully match' instead of hard 'do not match'
and it is explained what the actual facts are
'files in the backup are not same as ...' and
it is made explicit that it is the user's task to decide
if those changed files cause issues in his recreated system.

jsmeix commented at 2023-03-10 14:14:¶

I tested it on KVM/QEMU VMs:

On the original system VM:

I work on a GitHub master checkout

# git clone https://github.com/rear/rear.git

# mv rear rear.github.master

# cd rear.github.master

# git checkout jsmeix-show-when-checking-files

# vi etc/rear/local.conf

In etc/rear/local.conf I need

CHECK_CONFIG_FILES+=( "$CONFIG_DIR" )

because in default.conf there is hardcoded '/etc/rear/' in

CHECK_CONFIG_FILES=( '/etc/drbd/' '/etc/drbd.conf' '/etc/lvm/lvm.conf' '/etc/multipath.conf' '/etc/rear/' '/etc/udev/udev.conf' )

but on a GitHub master checkout the ReaR config
is not in '/etc/rear/' but in '$(pwd)/etc/rear/'
so this hardcoded '/etc/rear/' in default.conf
should be replaced by "$CONFIG_DIR".

Then I did:

# usr/sbin/rear -D mkrescue
...

# echo '# comment' >>etc/rear/local.conf

# usr/sbin/rear -D mkbackuponly
...

On the replacement VM:

RESCUE localhost:~ # rear -D recover
...
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Running 'finalize' stage ======================
Checking if certain restored files are consistent with the recreated system
See /var/lib/rear/layout/config/files.md5sum what files are checked
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /root/rear.github.master/etc/rear/local.conf: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
...

RESCUE localhost:~ # cat /var/lib/rear/layout/config/files.md5sum
9129df91cf65ad4f6b3e5459aa4b92d9  /etc/lvm/lvm.conf
bf60be80a4cc51271a1618edf5a6d66f  /etc/udev/udev.conf
89e4602238c8f6d3aec2ba7adea8dafe  /root/rear.github.master/etc/rear/local.conf
0a37bb12518797a3cc2ef76105dfc626  /root/rear.github.master/etc/rear/os.conf
b631a22b67e2315a8e3ef4073d6883aa  /boot/grub2/grub.cfg
a1f277cc217edd984cb4bc6bf6ac2e88  /etc/sysconfig/network/ifcfg-eth0
f0fc7dd78da5a4b54e18c4b6948770e2  /etc/sysconfig/network/ifcfg-lo
649dafa97b51bee5f5b9fa3657c4dc8e  /etc/grub.d/00_header
55d58a46aeb002408fea67ed9a0a7d5c  /etc/grub.d/10_linux
5386ae9dde85d00d298eb4a24510034e  /etc/grub.d/20_linux_xen
0dfad6405eeeaedb2bf96f11e5d52dc0  /etc/grub.d/20_memtest86+
8b3cb2149b06a1f1b6500fb612f28beb  /etc/grub.d/30_os-prober
babe7de352fe18de5a238569cf4b8a11  /etc/grub.d/40_custom
7bb4eb43ee7cb6d6f52df3dd2bde84e9  /etc/grub.d/41_custom
cd81a9cad06de59dfc33f8c49596f60f  /etc/grub.d/90_persistent
b326c91e90280bf45d2fc8564d14fb4f  /etc/grub.d/95_textmode
be58f42dfe74feb6eeb98c6a843c743f  /etc/grub.d/README
ea7b2372453a311a6f14f08051d74ed3  /etc/grub.d/80_suse_btrfs_snapshot
942086a4a6784497ed863ab154eb0767  /etc/sysconfig/bootloader
3e4eabe3bae31838cb02b4bbebd72955  /etc/fstab
8fd7fce8dbb757bc622c7c214d2cd721  /etc/smartd.conf
c7277b87f798aac207a5d25e6baa67e9  /etc/sysconfig/smartmontools

RESCUE localhost:~ # md5sum /mnt/local/root/rear.github.master/etc/rear/local.conf
93cb24f14a6665f3e4c353e5791200cf  /mnt/local/root/rear.github.master/etc/rear/local.conf

Excerpts from /var/log/rear/rear-localhost.log

+ source /usr/share/rear/finalize/default/060_compare_files.sh
++ test -s /var/lib/rear/layout/config/files.md5sum
++ LogPrint 'Checking if certain restored files are consistent with the recreated system'
...
++ DebugPrint 'See /var/lib/rear/layout/config/files.md5sum what files are checked'
...
++ local md5sum_stdout
+++ chroot /mnt/local md5sum -c --quiet
md5sum: WARNING: 1 computed checksum did NOT match
++ md5sum_stdout='/root/rear.github.master/etc/rear/local.conf: FAILED'
++ LogPrintError 'Restored files do not fully match the recreated system in /mnt/local'
...
++ LogPrintError '(files in the backup are not same as when the ReaR rescue/recovery system was made)'
+++ sed -e 's/^/  /'
++ LogPrintError '  /root/rear.github.master/etc/rear/local.conf: FAILED'
...
++ LogPrintError 'Manually check if those changed files cause issues in your recreated system'
...
++ return 1
+ source_return_code=1
+ test 0 -eq 1
+ Debug 'Source function: '\''source /usr/share/rear/finalize/default/060_compare_files.sh'\'' returns 1'

jsmeix commented at 2023-03-13 14:00:¶

How it looks with the current changes here including
https://github.com/rear/rear/pull/2954/commits/37e87cbf07e1696662d648cb7e867dd09e94196d
with on the original system only
i.e. without CHECK_CONFIG_FILES+=( "$CONFIG_DIR" ) in local.conf

# rear -D mkrescue
...

# echo '# comment' >>etc/rear/local.conf

# echo '# comment' >>etc/rear/os.conf

# rear -D mkbackuponly
...

and then on the replacement system:

RESCUE localhost:~ # rear -D recover
...
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Running 'finalize' stage ======================
Checking if certain restored files are consistent with the recreated system
See /var/lib/rear/layout/config/files.md5sum what files are checked
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /root/rear.github.master/etc/rear/local.conf: FAILED
  /root/rear.github.master/etc/rear/os.conf: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
...

jsmeix commented at 2023-03-13 14:11:¶

@rear/contributors
I would like to merge it tomorrow afternoon
unless there are objections.

Perhaps this one is not a final solution
because it does not change the actual functionality
but at least it should make it more clear to the user
what the actual functionality behind is meant to provide.

pcahyna commented at 2023-03-13 14:29:¶

Hi @jsmeix , can you please show what the output looks like with rear recover (without -D) ? The original idea was that the errors appear out of context, we should check how they look in the default mode.

jsmeix commented at 2023-03-14 08:31:¶

How it looks with plain 'rear recover'

RESCUE localhost:~ # rear recover
...
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Checking if certain restored files are consistent with the recreated system
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /root/rear.github.master/etc/rear/local.conf: FAILED
  /root/rear.github.master/etc/rear/os.conf: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
...

jsmeix commented at 2023-03-14 08:40:¶

I see now a possible issue:

It shows

(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /root/rear.github.master/etc/rear/local.conf: FAILED
  /root/rear.github.master/etc/rear/os.conf: FAILED

but the shown files are not those from the backup.

Within the ReaR recovery system those files from the backup are

/mnt/local/root/rear.github.master/etc/rear/local.conf
/mnt/local/root/rear.github.master/etc/rear/os.conf

so I suggest that I adapt finalize/default/060_compare_files.sh to

LogPrintError "$( sed -e "s|^|  $TARGET_FS_ROOT|" <<< "$md5sum_stdout" )"

to make the "rear recover" output look as follows

...
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Checking if certain restored files are consistent with the recreated system
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /mnt/local/root/rear.github.master/etc/rear/local.conf: FAILED
  /mnt/local/root/rear.github.master/etc/rear/os.conf: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
...

@pcahyna @schlomo
what do you think?

jsmeix commented at 2023-03-14 08:43:¶

Alternatively I could remove the leading slash via

LogPrintError "$( sed -e 's|^/|  |' <<< "$md5sum_stdout" )"

to make the "rear recover" output look as follows

...
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Checking if certain restored files are consistent with the recreated system
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  root/rear.github.master/etc/rear/local.conf: FAILED
  root/rear.github.master/etc/rear/os.conf: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
...

so that it shows a relative path (i.e. relative to /mnt/local)

jsmeix commented at 2023-03-14 08:55:¶

The issue is that within the ReaR recovery system
there exist two kind of these files:

The ones in the ReaR recovery system

/etc/rear/local.conf
/etc/rear/os.conf

and the ones from the backup restore
which are in my case

/mnt/local/root/rear.github.master/etc/rear/local.conf
/mnt/local/root/rear.github.master/etc/rear/os.conf

but usually those are

/mnt/local/etc/rear/local.conf
/mnt/local/etc/rear/os.conf

so usually the only difference is the '/mnt/local' prefix.

Those in the ReaR recovery system match the stored md5sums
in /var/lib/rear/layout/config/files.md5sum.
Only those from the backup restore do not match the stored
md5sums in /var/lib/rear/layout/config/files.md5sum:

RESCUE localhost:~ # grep 'etc/rear' /var/lib/rear/layout/config/files.md5sum 
0c26e503364b45315699c6ea5f3a7281  /root/rear.github.master/etc/rear/local.conf
0a37bb12518797a3cc2ef76105dfc626  /root/rear.github.master/etc/rear/os.conf

RESCUE localhost:~ # md5sum /etc/rear/local.conf /etc/rear/os.conf
0c26e503364b45315699c6ea5f3a7281  /etc/rear/local.conf
07cc16f9a6296665e9631941f6f7904d  /etc/rear/os.conf

RESCUE localhost:~ # md5sum /mnt/local/root/rear.github.master/etc/rear/local.conf /mnt/local/root/rear.github.master/etc/rear/os.conf 
b9a4490739fa9d0e5f6a05a0f99ae036  /mnt/local/root/rear.github.master/etc/rear/local.conf
50221671982b1ee304a265cc466c5e2f  /mnt/local/root/rear.github.master/etc/rear/os.conf

So I think it is important to show exactly those
that do not match the stored md5sums
with their full path as valid in the user's
currently running ReaR recovery system as in
https://github.com/rear/rear/pull/2954#issuecomment-1467645338

jsmeix commented at 2023-03-14 09:05:¶

Showing

  /root/rear.github.master/etc/rear/local.conf: FAILED
  /root/rear.github.master/etc/rear/os.conf: FAILED

is plain wrong because those files do not exist
in the ReaR recovery system:

RESCUE localhost:~ # ls /root/rear.github.master/etc/rear/local.conf /root/rear.github.master/etc/rear/os.conf 
ls: cannot access '/root/rear.github.master/etc/rear/local.conf': No such file or directory
ls: cannot access '/root/rear.github.master/etc/rear/os.conf': No such file or directory

This is because of how '$CONFIG_DIR/.' gets copied into
the ReaR recovery system into hardcoded 'etc/rear' by
build/GNU/Linux/100_copy_as_is.sh

cp $v -r -L $CONFIG_DIR/. $ROOTFS_DIR/etc/rear/

https://github.com/rear/rear/blob/rear-2.7/usr/share/rear/build/GNU/Linux/100_copy_as_is.sh#L165

schlomo commented at 2023-03-14 09:20:¶

@jsmeix I'd prefer to show the full path in the Rescue system, .e. /mnt/local/... so that users can simply copy-paste that to look at it.

FYI,
in #2950 I made a change to the COPY_AS_IS copy function
https://github.com/rear/rear/pull/2950/files#diff-e3614c8ae81bb8a1d792f1a3016e99ff7f12295d47a41d3a21f7f4faca14830c
and in #2937 too at
https://github.com/rear/rear/pull/2937/files#diff-e3614c8ae81bb8a1d792f1a3016e99ff7f12295d47a41d3a21f7f4faca14830c.

My goal is to

support running from checkout while using the real configuration in /etc/rear to make it simple to try a different ReaR from the installed one without first installing it.
support creating the VAR_DIR outside of the checkout area so that my git checkout won't be polluted by files that only root can delete - and to make it simpler to run the same git checkout on multiple VMs without interference.

jsmeix commented at 2023-03-14 09:34:¶

Yes, the full path in the recovery system
is the only correct way and it also matches
what the code actually does via 'chroot /mnt/local'
i.e. the code is working on files in '/mnt/local'.

jsmeix commented at 2023-03-14 10:15:¶

FYI:

Now I see in
https://github.com/rear/rear/pull/2954#issuecomment-1467664845
that /etc/rear/os.conf in the ReaR recovery system
does not match the stored md5sums in
/var/lib/rear/layout/config/files.md5sum

This is because the stored md5sum during "rear mkrescue" time
for /root/rear.github.master/etc/rear/os.conf is for this content

OS_VENDOR=SUSE_LINUX
OS_VERSION=15.3

but the /etc/rear/os.conf in the ReaR recovery system is

OS_VENDOR=SUSE_LINUX
OS_VERSION=15.3
# The following information was added automatically by the mkrescue workflow:
ARCH='Linux-i386'
OS='GNU/Linux'
OS_VERSION='15.3'
OS_VENDOR='SUSE_LINUX'
OS_VENDOR_VERSION='SUSE_LINUX/15.3'
OS_VENDOR_ARCH='SUSE_LINUX/i386'
OS_MASTER_VENDOR='SUSE'
# End of what was added automatically by the mkrescue workflow.

which is done by
usr/share/rear/build/default/975_update_os_conf.sh
that adds only to "$ROOTFS_DIR/etc/rear/os.conf"
so /etc/rear/os.conf in the ReaR recovery system
always differs from what is in the backup.

"Fortunately" the md5sum check does not compare
what is in the backup with what is in the recovery system.
The md5sum check compares what is in the backup
with what there was on the original system
at the time when "rear mkrescue" was run.

pcahyna commented at 2023-03-14 10:37:¶

@jsmeix

How it looks with plain 'rear recover'

RESCUE localhost:~ # rear recover
...
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Checking if certain restored files are consistent with the recreated system
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /root/rear.github.master/etc/rear/local.conf: FAILED
  /root/rear.github.master/etc/rear/os.conf: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
...

Thanks, I am now thinking about another issue - is it clear what "Restored files do not fully match the recreated system in /mnt/local" means? It does not seem obvious what is the difference between "restored system" and "recreated files". Especially when "restored files" are also restored under /mnt/local.
I know, the idea is that the system (including disk layout etc) was restored using different files than those in the backup, so the various IDs embedded in the storage components do not match, but is it understandable? (The "(files in the backup are not same as when the ReaR rescue/recovery system was made)" note clarifies it a bit.)

schlomo commented at 2023-03-14 11:02:¶

About /etc/rear/os.conf, I really would like to move this file to exists only in the Rescue system, see https://github.com/rear/rear/pull/1713#issuecomment-1456020891 as I believe that it causes much more problems than it solves.

So for me, feel free to move the creation of this file to the build stage or such and moving it to the Rescue system only.

jsmeix commented at 2023-03-14 11:07:¶

@pcahyna
nice to see that also you get confused here (same as I do :-)
because you wrote "restored system" and "recreated files"
but actually it is "restored files" and "recreated system" ;-)

I think we should not show lengthy explanations on the
user's terminal because they let other info scroll away
(e.g. I don't know how things behave on a serial console)
so I tried to keep the messages not too long.

In the three information message lines

Checking if certain restored files are consistent with the recreated system
Restored files do not fully match the recreated system in /mnt/local
(files in the backup are not same as when the ReaR rescue/recovery system was made)

I tried to show various different keywords that apply here
to the user to trigger some (hopefully) right idea in his mind
what this stuff actually is about:
Consistency of what was recreated with his restored backup.

I.e. consistency of "rear mkrescue" with "rear mkbackuponly"
(or with whatever external backup method).

The "certain" in "certain restored files" is crucial
because this is not a global consistency check
which is why there is the DebugPrint message

See /var/lib/rear/layout/config/files.md5sum what files are checked

pcahyna commented at 2023-03-14 11:18:¶

@jsmeix would it make sense to change
Restored files do not fully match the recreated system in /mnt/local
to
Restored files in /mnt/local do not fully match the recreated system
because it is the restored files that can be found under /mnt/local, the "recreated system" is more nebulous concept (the stuff that may not match is usually not in any files but in filesystem/LVM/RAID/whatever metadata, or even disk firmware in case of disk IDs)?

jsmeix commented at 2023-03-14 11:32:¶

Regarding /etc/rear/os.conf:

I don't know the reason what it exists at all.
I.e. the reason why such variables are stored in a file?

Why is it not sufficient to call SetOSVendorAndVersion in sbin/rear
so ReaR runs according to the current actual runtime environment
(and not according to something possibly outdated stored in a file).

In my ReaR recovery system I have /etc/os-release.

I changed bin/rear in my ReaR recovery system to

set -x
SetOSVendorAndVersion
exit 99

With that I did

RESCUE localhost:~ # rear -D help
Exiting rear help (PID 10662) and its descendant processes ...
Running exit tasks
rear help failed, check /var/log/rear/rear-localhost.log.lockless for details

and got in /var/log/rear/rear-localhost.log.lockless

+ source /etc/rear/os.conf
++ OS_VENDOR=SUSE_LINUX
++ OS_VERSION=15.3
++ ARCH=Linux-i386
++ OS=GNU/Linux
++ OS_VERSION=15.3
++ OS_VENDOR=SUSE_LINUX
++ OS_VENDOR_VERSION=SUSE_LINUX/15.3
++ OS_VENDOR_ARCH=SUSE_LINUX/i386
++ OS_MASTER_VENDOR=SUSE
+ source_return_code=0
+ test 0 -eq 0
+ cd /root
+ test 1
+ Debug 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test 1
+ Log 'Leaving debugscript mode (back to previous bash flags and options settings).'
+ test -w /var/log/rear/rear-localhost.log.lockless
+ echo '2023-03-14 12:26:44.572808888 Leaving debugscript mode (back to previous bash flags and options settings).'
2023-03-14 12:26:44.572808888 Leaving debugscript mode (back to previous bash flags and options settings).
+ SetOSVendorAndVersion
+ test SUSE_LINUX = generic -o 15.3 = none
+ OS_VENDOR_VERSION=SUSE_LINUX/15.3
+ OS_VENDOR_ARCH=SUSE_LINUX/i386
+ OS_VENDOR_VERSION_ARCH=SUSE_LINUX/15.3/i386
+ case "$( echo $OS_VENDOR_VERSION | tr '[A-Z]' '[a-z]' )" in
++ tr '[A-Z]' '[a-z]'
++ echo SUSE_LINUX/15.3
+ OS_MASTER_VENDOR=SUSE
+ OS_MASTER_VERSION=15
+ '[' SUSE ']'
+ OS_MASTER_VENDOR_VERSION=SUSE/15
+ OS_MASTER_VENDOR_ARCH=SUSE/i386
+ OS_MASTER_VENDOR_VERSION_ARCH=SUSE/15/i386
+ exit 99

Then I did

RESCUE localhost:~ # mv /etc/rear/os.conf /etc/rear/os.conf.away

RESCUE localhost:~ # rear -D help

and got in /var/log/rear/rear-localhost.log.lockless

+ SetOSVendorAndVersion
+ test generic = generic -o none = none
+ [[ -f /etc/os-release ]]
+ grep -q -i fedora /etc/os-release
+ grep -q -i -E '(centos|redhat|scientific|oracle)' /etc/os-release
+ grep -q -i suse /etc/os-release
+ OS_VENDOR=SUSE_LINUX
+ grep -q -i debian /etc/os-release
+ grep -q -i -E '(ubuntu|linuxmint)' /etc/os-release
+ grep -q -i arch /etc/os-release
++ cut '-d"' -f2
++ grep '^VERSION_ID=' /etc/os-release
+ OS_VERSION=15.3
+ test SUSE_LINUX = generic
+ test SUSE_LINUX = generic
+ test SUSE_LINUX = generic
+ OS_VENDOR_VERSION=SUSE_LINUX/15.3
+ OS_VENDOR_ARCH=SUSE_LINUX/i386
+ OS_VENDOR_VERSION_ARCH=SUSE_LINUX/15.3/i386
+ case "$( echo $OS_VENDOR_VERSION | tr '[A-Z]' '[a-z]' )" in
++ tr '[A-Z]' '[a-z]'
++ echo SUSE_LINUX/15.3
+ OS_MASTER_VENDOR=SUSE
+ OS_MASTER_VERSION=15
+ '[' SUSE ']'
+ OS_MASTER_VENDOR_VERSION=SUSE/15
+ OS_MASTER_VENDOR_ARCH=SUSE/i386
+ OS_MASTER_VENDOR_VERSION_ARCH=SUSE/15/i386
+ exit 99

ARCH is always set in default.conf regardless
what is stored in etc/rear/os.conf

jsmeix commented at 2023-03-14 11:54:¶

@schlomo
why is /etc/rear/os.conf needed in the recovery system
when it is not needed on the original system?

I.e. what is missing in the recovery system
that running sbin/rear in the recovery system
needs a /etc/rear/os.conf file?

schlomo commented at 2023-03-14 12:08:¶

I agree with you @jsmeix, I also don't know what for we need the os.conf in the original system. For the Rescue system I can only imagine that it helps on systems where /etc/os-release is not present, which seems to be only CentOS 6:

@gdha introduced it in #1713 so maybe he can say more about it.

I'd hope that moving it to exist in the Rescue system only would be a step towards getting rid of it.

jsmeix commented at 2023-03-14 12:13:¶

@pcahyna
thank you for your better wording in
https://github.com/rear/rear/pull/2954#issuecomment-1467910938

I changed it via
https://github.com/rear/rear/pull/2954/commits/11fd6f6b61333b4463c7b239841a70519f5cae78

Only a side note FYI:
Usually I prefer to have human readable text at the beginning
and the technical stuff at the end of a message like

Cannot access mystuff.conf in /longer/path/to/some/deeply/nested/directory

instead of

mystuff.conf in /longer/path/to/some/deeply/nested/directory inaccessible

so usually I try to evaluate variables at the end of a message
but $TARGET_FS_ROOT evaluates to a short value so it is
perfectly fine to have it in the middle of the message.

jsmeix commented at 2023-03-14 12:23:¶

@schlomo
I assume the function SetOSVendorAndVersion()
in lib/config-functions.sh supports CentOS 6
(e.g. via /etc/system-release or other files or via lsb_release).

Currently I fail to see how files like /etc/os-release
and other ...-release files get copied into the recovery system?

jsmeix commented at 2023-03-14 12:26:¶

Oh!
I see now basically the same was already discussed in
https://github.com/rear/rear/pull/1713
and
https://github.com/rear/rear/issues/1639

schlomo commented at 2023-03-14 12:28:¶

Back in the days the original idea was to do an "expensive" calculation of the OS_* variables during mkrescue and then use that during recover as well. I'd still say that instead of copying all kinds of OS-specific files to the rescue system I'd rather store the result in a config file and that is it - hence the thought to create the /etc/rear/os.conf in the rescue system only. If it is present and filled with meaningful variable content then the OS discovery will be skipped.

jsmeix commented at 2023-03-14 15:28:¶

@schlomo
let's do further /etc/rear/os.conf related things
outside of this pull request as needed.

schlomo commented at 2023-03-14 15:31:¶

Yes, please merge this one here

jsmeix commented at 2023-03-14 15:34:¶

With the latest code in this pull request I get:

On my original system VM:

# usr/sbin/rear -D mkrescue
...

# cat var/lib/rear/layout/config/files.md5sum
9129df91cf65ad4f6b3e5459aa4b92d9  /etc/lvm/lvm.conf
bf60be80a4cc51271a1618edf5a6d66f  /etc/udev/udev.conf
0c26e503364b45315699c6ea5f3a7281  /root/rear.github.master/etc/rear/local.conf
0a37bb12518797a3cc2ef76105dfc626  /root/rear.github.master/etc/rear/os.conf
b631a22b67e2315a8e3ef4073d6883aa  /boot/grub2/grub.cfg
a1f277cc217edd984cb4bc6bf6ac2e88  /etc/sysconfig/network/ifcfg-eth0
f0fc7dd78da5a4b54e18c4b6948770e2  /etc/sysconfig/network/ifcfg-lo
649dafa97b51bee5f5b9fa3657c4dc8e  /etc/grub.d/00_header
55d58a46aeb002408fea67ed9a0a7d5c  /etc/grub.d/10_linux
5386ae9dde85d00d298eb4a24510034e  /etc/grub.d/20_linux_xen
0dfad6405eeeaedb2bf96f11e5d52dc0  /etc/grub.d/20_memtest86+
8b3cb2149b06a1f1b6500fb612f28beb  /etc/grub.d/30_os-prober
babe7de352fe18de5a238569cf4b8a11  /etc/grub.d/40_custom
7bb4eb43ee7cb6d6f52df3dd2bde84e9  /etc/grub.d/41_custom
cd81a9cad06de59dfc33f8c49596f60f  /etc/grub.d/90_persistent
b326c91e90280bf45d2fc8564d14fb4f  /etc/grub.d/95_textmode
be58f42dfe74feb6eeb98c6a843c743f  /etc/grub.d/README
ea7b2372453a311a6f14f08051d74ed3  /etc/grub.d/80_suse_btrfs_snapshot
942086a4a6784497ed863ab154eb0767  /etc/sysconfig/bootloader
3e4eabe3bae31838cb02b4bbebd72955  /etc/fstab
8fd7fce8dbb757bc622c7c214d2cd721  /etc/smartd.conf
c7277b87f798aac207a5d25e6baa67e9  /etc/sysconfig/smartmontools

# echo '# comment' >>etc/rear/local.conf

# echo '# comment' >>/etc/grub.d/README

# usr/sbin/rear -D mkbackuponly
...

On my replacement VM:

RESCUE localhost:~ # rear -D recover
...
Recreating directories (with permissions) from /var/lib/rear/recovery/directories_permissions_owner_group
Running 'finalize' stage ======================
Checking if certain restored files are consistent with the recreated system
See /var/lib/rear/layout/config/files.md5sum what files are checked
Restored files in /mnt/local do not fully match the recreated system
(files in the backup are not same as when the ReaR rescue/recovery system was made)
  /mnt/local/root/rear.github.master/etc/rear/local.conf: FAILED
  /mnt/local/etc/grub.d/README: FAILED
Manually check if those changed files cause issues in your recreated system
Migrating disk-by-id mappings in certain restored files in /mnt/local to current disk-by-id mappings ...
...

jsmeix commented at 2023-03-14 15:36:¶

@rear/contributors
I would like to merge it tomorrow afternoon
unless there are objections.

jsmeix commented at 2023-03-15 14:19:¶

No time today for it - postponing...

jsmeix commented at 2023-03-16 10:00:¶

What is the actual effect of 'return 1' in a ReaR script:

First and foremost in general
no piece of code that does not finish successfully
should lie to its caller and return a success code
regardless if a caller of a piece of code
cares about the return code.

In particular regaring the return code of a ReaR script see
https://github.com/rear/rear/pull/1970

pcahyna commented at 2023-03-16 10:10:¶

Return code was discussed here: https://github.com/rear/rear/pull/2608#issuecomment-831929130

jsmeix commented at 2023-03-16 10:29:¶

@pcahyna
thank you for the link to another place where I also explained it.

jsmeix commented at 2023-03-16 10:30:¶

Phew!
Now - first things first - it's brunch time,
afterwards it's merge time ;-)

jsmeix commented at 2023-03-16 12:14:¶

Only FYI:

During brunch time I was wondering
if I may improve the current user info message

files in the backup are not same as when the ReaR rescue/recovery system was made

where the part "when the ReaR rescue/recovery system was made"
might be perceived by the user as somewhat "nebulous", cf.
https://github.com/rear/rear/pull/2954#issuecomment-1467910938
to a shorter and more explicit text

files in the backup are not same as when 'rear mkrescue' was run

because when the user did "rear mkbackup" then there is consistency
of what is in the recovery system with what is in the backup
at least with sufficent high probability that we can sufficently
safely assume that with "rear mkbackup" there will be no such
user messages about inconsistencies.

BUT
then I remembered
https://github.com/rear/rear/issues/2951#issuecomment-1456051265
that reads (excerpt)

... 'mkrescue' and 'mkbackup' and 'mkopalpba'
... make a ReaR recovery system

so such user messages about inconsistencies
may (I don't know any TCG Opal 2 details)
also appear when TCG Opal 2 disks are used
which means the generic but somewhat "nebulous" wording

files in the backup are not same as when the ReaR rescue/recovery system was made

is the better one.

pcahyna commented at 2023-03-16 13:35:¶

@jsmeix this wording is totally OK for me. What seemed "nebulous" to me was the "recreated system" concept as distinct from restored files, not the "when the ReaR rescue/recovery system was made" part.

jsmeix commented at 2023-03-16 14:37:¶

The longer one stares at an issue the more likely ghosts appear,
or in other worlds: "Don't dig too deep!"
https://en.wikipedia.org/wiki/Moria,_Middle-earth

[Export of Github issue for rear/rear.]

#2954 PR merged: Better user messages for finalize/default/060_compare_files.sh¶