#2950 PR merged: ReaR in Docker for development & fix package dependencies

Labels: enhancement, fixed / solved / done, ReaR Project

schlomo opened issue at 2023-03-01 17:53:

Tooling to quickly run commands or ReaR via Docker in all "officially supported" Linux distros and fixes for package dependencies

@rear/contributors happy to hear your thoughts on how to make tools/run-in-docker even more useful for you.

Quick intro:

  1. run tools/run-in-docker -- --patch to prepare the Docker images for ReaR
  2. run tools/run-in-docker -- to check Bash version
  3. run tools/run-in-docker -- usr/sbin/rear dump to check ReaR on all distros

Use run-in-docker -a amd64 -- to choose architecture different from your host (e.g. for working on M1 Mac)

Use run-in-docker suse -- to limit to matching subset of Docker images - or specify completely different Docker image

WIP, currently works for Ubuntu, Debian and SUSE

@jsmeix I couldn't find a working Docker image of SLES12 with repos that work for software installation so I used Leap 42 that seems to be similar (e.g. using asciidoc instead of asciidoctor)

jsmeix commented at 2023-03-02 12:55:

@schlomo
regarding a SLES12 Docker image with repos for software installation:

It was a rather long time ago (I guess about two years ago)
when I had worked a bit with Docker only to try out something.
At that time I used only SLE15 and openSUSE Leap 15.1 and 15.2.
Because all those stuff is currently continuously developing
I cannot provide you right now what works for SLE12.
I will try to play around with it as time permits.
Feel free to skip SLE12 - at least for now.

Perhaps SUSE's "Container Guide" may help you:
https://documentation.suse.com/container/all/pdf/SLES-container_color_en.pdf#book-sles-docker

schlomo commented at 2023-03-02 12:58:

@jsmeix do you think that Leap 42 is sufficiently similar to SLES12 to allow using it for testing, or is it too far away and there is no value in testing something on Leap 42?

The problem seems to be the lack of publicly available SLES12 repos

jsmeix commented at 2023-03-02 14:37:

I can use a public accessible SLE12-SP5 image repository
but there are no RPM package repopsitories defined in it
and what is installed by default is too minimal:

# docker run -it --name testing registry.suse.com/suse/sles12sp5:latest /bin/bash
Unable to find image 'registry.suse.com/suse/sles12sp5:latest' locally
latest: Pulling from suse/sles12sp5
4ffd95c779b8: Pull complete 
Digest: sha256:21777cc4d43d15a7964e63a0aa03d1dc7cf9982f4890b4394c9e05b7f737ec23
Status: Downloaded newer image for registry.suse.com/suse/sles12sp5:latest

e32e3327832d:/ # zypper lr  
Refreshing service 'container-suseconnect-zypp'.
Problem retrieving the repository index file for service 'container-suseconnect-zypp':
[container-suseconnect-zypp|file:/usr/lib/zypp/plugins/services/container-suseconnect-zypp] 
Warning: Skipping service 'container-suseconnect-zypp' because of the above error.
Warning: No repositories defined.
Use the 'zypper addrepo' command to add one or more repositories.

e32e3327832d:/ # type -a bc
bash: type: bc: not found

Now I found my (SUSE internal) documentation from the time
when I had worked a bit with Docker to try out something.
It contains (excerpt):

I did this very first proof of concept test on Leap 15.2
because for SLE15 there are no public accessible
online software repositories which would make it
needlessly hard ...
... a ... Docker container is run that is based
on a Leap 15.2 base container image
...
The following Dockerfile
...
FROM registry.opensuse.org/opensuse/leap
MAINTAINER jsmeix
ADD etc/zypp/* /etc/zypp/
RUN zypper -v refs && zypper -v refresh
# in particular with online repositories it seems reasonable
# to first update the basic system to an up-to-date state
# before installing newer packages into it:
RUN zypper -v --non-interactive up
# netcat and less are nice to have when running bash inside the container:
RUN zypper -v --non-interactive in netcat less
...

I also got stuck at the same problem
which I avoided by using Leap.

So I suggest to use Leap instead of SLE
to avoid issues with SLE restrictions
e.g. like

# docker run -it --name testing2 registry.opensuse.org/opensuse/leap:latest /bin/bash
...

22f4fd0505a6:/ # zypper lr -E
Repository priorities are without effect.
All enabled repositories share the same priority.
repo-backports-update | Update repository of openSUSE Backports
repo-non-oss          | Non-OSS Repository
repo-oss              | Main Repository
repo-sle-update       | Update repository with updates from SUSE Linux Enterprise 15
repo-update           | Main Update Repository
repo-update-non-oss   | Update Repository (Non-Oss)

22f4fd0505a6:/ # zypper refs && zypper -v refresh
...

22f4fd0505a6:/ # zypper --non-interactive up
...

22f4fd0505a6:/ # type -a bc
bash: type: bc: not found

22f4fd0505a6:/ # zypper --non-interactive in bc
...

22f4fd0505a6:/ # type -a bc
bc is /usr/bin/bc

Regarding SLE12 vs. Leap 42:

Because Leap 42.1 is based on SLE12-SP1
it is perfectly OK to use Leap >= 42.1
as a substitute for SLE12.

On
https://registry.opensuse.org/cgi-bin/cooverview
I found only 42 and 42.3 (under "leap")

I get:

# docker run -it --name testing3 registry.opensuse.org/opensuse/leap:42 /bin/bash

Unable to find image 'registry.opensuse.org/opensuse/leap:42' locally
42: Pulling from opensuse/leap
868a4212e7d9: Pull complete 
Digest: sha256:6ff4db34f99ba6f374b724077405de1eaf25ad1aa8c4e1e3197fc158d5d7f4ce
Status: Downloaded newer image for registry.opensuse.org/opensuse/leap:42

b77e78fb6c47:/ # zypper lr -E
Repository priorities are without effect. All enabled repositories share the same priority.
# | Alias          | Name           | Enabled | GPG Check | Refresh
--+----------------+----------------+---------+-----------+--------
1 | NON OSS        | NON OSS        | Yes     | ( p) Yes  | Yes    
2 | NON OSS Update | NON OSS Update | Yes     | ( p) Yes  | Yes    
3 | OSS            | OSS            | Yes     | ( p) Yes  | Yes    
4 | OSS Update     | OSS Update     | Yes     | ( p) Yes  | Yes

b77e78fb6c47:/ # cat /etc/os-release 
NAME="openSUSE Leap"
VERSION="42.3"
ID=opensuse
ID_LIKE="suse"
VERSION_ID="42.3"
PRETTY_NAME="openSUSE Leap 42.3"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:42.3"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"

b77e78fb6c47:/ # zypper refs                  
All services have been refreshed.

b77e78fb6c47:/ # zypper --non-interactive refresh
Repository 'NON OSS' is up to date.                                                                                                                     
Repository 'NON OSS Update' is up to date.                                                                                                              
Repository 'OSS' is up to date.                                                                                                                         
Retrieving repository 'OSS Update' metadata ------------------------[\]
...
Signature verification failed for file 'repomd.xml' from repository 'OSS Update'. Continue? [yes/no] (no): no
Retrieving repository 'OSS Update' metadata .................[error]
...
Skipping repository 'OSS Update' because of the above error.
Some of the repositories have not been refreshed because of an error.

b77e78fb6c47:/ # zypper --non-interactive up
Retrieving repository 'OSS Update' metadata --------------------------[\]
Signature verification failed for file 'repomd.xml' from repository 'OSS Update'.
...
Warning: Repository 'NON OSS Update' appears to be outdated. Consider using a different mirror or server.
Reading installed packages...
Nothing to do.

b77e78fb6c47:/ # type -a bc
bash: type: bc: not found

b77e78fb6c47:/ # zypper --non-interactive in bc
Retrieving repository 'OSS Update' metadata ------------------------[|]
Signature verification failed for file 'repomd.xml' from repository 'OSS Update'.
...
The following NEW package is going to be installed:
  bc
...
Continue? [y/n/...? shows all options] (y): y
Retrieving package bc-1.06.95-11.13.x86_64                                                                         (1/1), 112.4 KiB (253.8 KiB unpacked)
Retrieving: bc-1.06.95-11.13.x86_64.rpm ........................[done]
Checking for file conflicts: ...................................[done]
(1/1) Installing: bc-1.06.95-11.13.x86_64 ......................[done]

b77e78fb6c47:/ # type -a bc
bc is /usr/bin/bc

schlomo commented at 2023-04-01 20:41:

@rear/contributors I'd like to merge this branch within the next couple of days. Currently Debian doesn't work because of some strange problem with their repos, but I consider this a minor problem as long as Ubuntu does work.

I heavily reworked the Makefile here to support building without writing so much to the source tree and to base all the distro builds off the generated dist archive.

jsmeix commented at 2023-04-04 12:27:

https://github.com/rear/rear/pull/2950/files
shows several "No newline at end of file" marks.
I didn't check if this is actually true or only false alarm.

schlomo commented at 2023-04-04 13:30:

I only found one worth fixing and fixed that in #2965.

FYI, I use pcregrep -LMr '\n\Z' . to find problematic files.


[Export of Github issue for rear/rear.]