#1518 Issue `closed`: Recovery system libraries are insufficient (libs needed by libs are missing)¶

Labels: bug, fixed / solved / done

jsmeix opened issue at 2017-09-26 13:36:¶

Recovery system libraries can be inconsistent
because (at least for me on SLES12) by default
some libraries require other libraries that are missing, cf.
https://github.com/rear/rear/pull/1514#issuecomment-331868727

This is only a minor bug because in practice all works for me and
I assume this issue exists "since ever" also for other distributions.
I assume only via https://github.com/rear/rear/pull/1514
this issue here became visible.

jsmeix commented at 2017-09-27 12:44:¶

No longer a minor bug but a real bug because
I think I know the reason:

I analyzed in
https://github.com/rear/rear/pull/1514#issuecomment-331868727
the first case where
"/usr/lib64/libsoftokn3.so requires additional libraries".

No program (i.e. no file in /bin/ in the recovery system
requires libsoftokn3 so I wondered how that library came
into the recovery system and I found it hardcoded in
usr/share/rear/conf/GNU/Linux.conf (excerpt):

LIBS=(
...
### needed for curl HTTPS
...
/lib*/libsoftokn3.so*
/usr/lib*/libsoftokn3.so*

cf.
https://github.com/rear/rear/commit/25eb92534ebaa5120885259c0011ee35cd89aa5b

What seems to be missing from my current point of view
is that in build/GNU/Linux/390_copy_binaries_libraries.sh

# copy libraries
declare -a all_libs=( $( for lib in ${LIBS[@]} $( SharedObjectFiles "${BINARIES[@]}" ) ; do
                             echo $lib
                         done | sort -u ) )

the SharedObjectFiles function must also be called for
the libraries in ${LIBS[@]} - why the heck is there no
quotation i.e. why is it not "${LIBS[@]}" ???

jsmeix commented at 2017-09-27 12:51:¶

Using in build/GNU/Linux/390_copy_binaries_libraries.sh

# copy libraries
declare -a all_libs=( $( for lib in $( SharedObjectFiles "${LIBS[@]}" | sed -e 's#^#/#' ) $( SharedObjectFiles "${BINARIES[@]}" | sed -e 's#^#/#' ) ; do
                             echo $lib
                         done | sort -u ) )

is not yet sufficient because now I get in the log (excerpts):

e205:~/rear.master # egrep 'requires additional libraries|=> not found' /root/rear.master/var/log/rear/rear-e205.log
2017-09-27 14:46:39.981727399 /lib64/security/pam_mount.so requires additional libraries
        libcryptmount.so.0 => not found
2017-09-27 14:46:39.985795203 /lib64/security/pam_winbind.so requires additional libraries
        libwbclient.so.0 => not found
        libtalloc.so.2 => not found
2017-09-27 14:46:39.989756260 /lib64/security/pam_pwcheck.so requires additional libraries
        libcrack.so.2 => not found
2017-09-27 14:46:39.993606685 /lib64/security/pam_cracklib.so requires additional libraries
        libcrack.so.2 => not found
2017-09-27 14:46:39.997429066 /usr/lib/udev/mtp-probe requires additional libraries
        libmtp.so.9 => not found

jsmeix commented at 2017-09-27 12:52:¶

It seems the whole "needed libraries autodetection"
does not work sufficiently transitive (i.e. it does not sufficiently
find libraries needed by libraries needed by libraries needed by...)

jsmeix commented at 2017-09-29 09:07:¶

https://github.com/rear/rear/commit/4456d6424c93df024fb1a67b56262f0df76d62c5
is a minimal fix for this issue that I found after longish
digging into the binaries and libraries copying code
until I got some understanding how it works.

The main problem was that I failed to see that the
SharedObjectFiles function only returns additionally
need libraries but not the one that needs them so that
something like a plain

all_libs=( $( SharedObjectFiles "${LIBS[@]}" "${BINARIES[@]}" ) )

does not result all needed libraries for the recovery system
because those libraries in LIBS are missing that are not needed
by a binary in BINARIES (the BINARIES are copied separated).

Accordingly the LIBS need also to be added via something like:

all_libs=( "${LIBS[@]}" $( SharedObjectFiles "${LIBS[@]}" "${BINARIES[@]}" ) )

The actual code looks even more complicated, in particular
the everywhere used trailing 'sed' band-aid as in

SharedObjectFiles ... | sed -e 's#^#/#'

shows that something is not right with the SharedObjectFiles
implementation.

During my experiments while digging around inside
the binaries and libraries copying code I did already
a major cleanup of that code (which "just happened"
because I needed to understand the code ;-) and
I will provide that cleanup in a separated pull request
because such a cleanup needs some review.

jsmeix commented at 2017-09-29 09:35:¶

FYI regarding "why the heck is there no quotation" in
https://github.com/rear/rear/issues/1518#issuecomment-332508657

I think in contrast to string variables
where no quoting in 'for' loops is required
to get each word separated

# string="foo bar 'foo bar' baz"

# for word in $string ; do echo "'$word'" ; done
'foo'
'bar'
''foo'
'bar''
'baz'

# for word in "$string" ; do echo "'$word'" ; done
'foo bar 'foo bar' baz'

array variables require quoting in 'for' loops
to get the array elements correctly separated

# arr=( foo bar 'foo bar' baz )

# for e in "${arr[@]}" ; do echo "'$e'" ; done
'foo'
'bar'
'foo bar'
'baz'

# for e in ${arr[@]} ; do echo "'$e'" ; done
'foo'
'bar'
'foo'
'bar'
'baz'

cf. https://github.com/rear/rear/issues/1068

jsmeix commented at 2017-10-04 14:16:¶

With https://github.com/rear/rear/pull/1521 merged
the whole binaries and libraries copying code is now
cleaned up and simplified.

[Export of Github issue for rear/rear.]

#1518 Issue closed: Recovery system libraries are insufficient (libs needed by libs are missing)¶