#3018 Issue `closed`: BACKUP=SESAM default excludes dysfunctional¶

Labels: bug, fixed / solved / done, external tool

abbbi opened issue at 2023-06-26 10:27:¶

hi,

it seems as the default excludes we read from sesam configuration file:

https://github.com/rear/rear/blob/master/usr/share/rear/lib/sesam-functions.sh#L25

are disfunctional because the pathes from the configuration file contain an trailing /.
As such, tar wont match the pathes from the exclude file and they end up in the generated
ISO image.

A simple fix would be to remove the pathes via something like:

 FOO="/var/opt/sesam/var/log/lgc/"
 echo ${FOO%%+(/)}

Better alternative would probably be to add * add the end of pathes (lgc/*) so directory
itself is considered but no contents.

are there any rules for "bashisms" like these in the code or is there already a function
that could be used? I would prepare an PR to fix this issue, accordingly. Thanks.

abbbi commented at 2023-06-26 11:42:¶

Proposed fix in my branch:

read only the toplevel directory from the configuration file, in case someone has changed the installation paths for the application these are the only values that change
default the rest of the directories to their location beneath the installation path, end with * so only contents are excluded not the directories itself. Not sure if duplicate / in path causes issues during exclude, so i opted to to remove if existant and append during variable definition.

jsmeix commented at 2023-06-26 11:42:¶

Via

# find usr/sbin/rear usr/share/rear -type f | xargs grep 'trailing slash'

I found only one code place that mentiones 'trailing slash'
which is in
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh

# accounting for possible trailing slashes in TARGET_FS_ROOT
esp_mountpoint_inside="${esp_mountpoint#${TARGET_FS_ROOT%%*(/)}}"

which also works
but %%*(/) looks less clear than %%+(/)
because %%*(/) removes zero or more occurrences of trailing /
while %%*(/) removes one or more occurrences of trailing /
but why should zero trailing / be removed at all?

abbbi commented at 2023-06-26 11:56:¶

see:
https://github.com/abbbi/rear/commit/420d4c8cc376f442b766ab8c3623016a5bcda087

jsmeix commented at 2023-06-26 11:58:¶

@abbbi
I am wondering if code like

SESAM_WORK_DIR="${SESAM_VAR_DIR%%+(/)}/var/work/*"

is really needed because - as far as I know - directory paths
with duplicated slashes like

//path///to/some/dir////

do not matter in Linux so ${SESAM_VAR_DIR%%+(/)}
could be unneeded and the simpler

SESAM_WORK_DIR="$SESAM_VAR_DIR/var/work/*"

could also work, for example like

# SESAM_VAR_DIR=//var///lib//rear///

# SESAM_WORK_DIR="$SESAM_VAR_DIR/layout/config/*"

# echo $SESAM_WORK_DIR
//var///lib//rear////layout/config/df.txt //var///lib//rear////layout/config/files.md5sum

ls $SESAM_WORK_DIR
//var///lib//rear////layout/config/df.txt  //var///lib//rear////layout/config/files.md5sum

# ls /var/lib/rear/layout/config
df.txt  files.md5sum

?

abbbi commented at 2023-06-26 12:08:¶

i think the problem is, that tar is pretty picky about this in its exclude list, as you can see it wont
consider the second path with duplicate slashes in its exclude list:

~ $ tar -cz - --exclude=/tmp/testme/folder1/* /tmp/testme/| tar -ztvf -
tar: Removing leading `/' from member names
-rw-r--r-- root/root   1048576 2023-03-20 16:18 -
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/folder1/
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/test/
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/folder12/
~ $ tar -cz - --exclude=/tmp/testme//folder1/* /tmp/testme/| tar -ztvf -
tar: Removing leading `/' from member names
-rw-r--r-- root/root   1048576 2023-03-20 16:18 -
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/folder1/
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/folder1/folder2/
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/test/
drwxr-xr-x abi/abi           0 2023-06-26 14:04 tmp/testme/folder12/

jsmeix commented at 2023-06-26 12:13:¶

@abbbi
ah! - yes - I forgot that 'tar' is "in between" here.

jsmeix commented at 2023-06-29 08:23:¶

With https://github.com/rear/rear/pull/3019 merged
this issue is fixed.

@abbbi
thank you for your issue report and for your fix!

Contributions for third-party backup tool support in ReaR
are much appreciated because we at ReaR upstream
usually do not have those third-party backup tools
so we totally depend on contributions from people
who use and know about third-party backup tools.

abbbi commented at 2023-06-29 12:43:¶

@jsmeix

hi,

i think we have to re-open as our internal CI has proven that now, where the exludes work and are in place
there is an issue with the sesam backup client service during ISO boot. Usually the service starts automatically
but now fails to start as the SESAM_WORK_DIR is excluded and its unable to figure out its semaphore state.

2023-06-29 14:19:18: Arguments: sm_main start
2023-06-29 14:19:18: Daemons: ['qm', 'db', 'rmi', 'passd', 'sms', 'stpd', 'ctrl', 'sshd', 'sepuler', 'ui', 'sds']
2023-06-29 14:19:18: Set file permissions...
2023-06-29 14:19:18: qm                 : offline
STATUS=ERROR MSG=semaphore is occupied by another process

This issue will only hit recent Sesam client versions, in older versions the service was started in another
way. Fix is probably to remove the SESAM_WORK_DIR from the exclude list altogether.

I have fixed this in our labs and will wait until our next build and CI cycle finishes so i can confirm and will create
another Pull Request to fix the issue introduced.

jsmeix commented at 2023-06-29 13:04:¶

@abbbi
thank you for testing it!

I reopened this issue so we can continue here
because its subject still matches.

abbbi commented at 2023-07-03 20:38:¶

@jsmeix Followup PR ready

jsmeix commented at 2023-07-04 15:24:¶

With also https://github.com/rear/rear/pull/3023 merged
this issue should be solved.

@abbbi
thank you for your continuous contributions to ReaR
in particular here for BACKUP=SESAM!

[Export of Github issue for rear/rear.]

#3018 Issue closed: BACKUP=SESAM default excludes dysfunctional¶