#2645 Issue closed: Missing snapshot releases - OpenSUSE Build Service packages¶
Labels: ReaR Project, no-issue-activity
hpannenb opened issue at 2021-07-05 08:10:¶
Hello,
I discovered for different distros the snapshot releases under http://relax-and-recover.org/download/ are not up-to-date (se example screenshot below). Is this by intention, problem in the build process(es) or did I miss something in the discussion(s)?
Regards,
Holger.
or
jsmeix commented at 2021-07-05 10:26:¶
@hpannenb
as far as I know those snapshots are not done automatically
(in particular no automated update after a git commit at ReaR upstream).
Perhaps @gdha could tell more about how that works?
jsmeix commented at 2021-07-05 10:30:¶
In general regarding using current ReaR GitHub master code see the
section
"Testing current ReaR upstream GitHub master code" in
https://en.opensuse.org/SDB:Disaster_Recovery
This is how I do it all the time but I run ReaR only on a few test
systems
(mostly QEMU/KVM virtual machines on my homeoffice laptop).
hpannenb commented at 2021-07-05 13:02:¶
@jsmeix Yes. On direct or indirect internet facing compute nodes
(servers, VMs, ...) I would do so as well. Unfortunately this is not the
case here. Obviously it is possible to download the GIT content and
transport it to the affected system, unpack /usr/share/rear/... etc.
But a baked/built package is much more efficient and its
installation/troubleshooting much more reproducible.
Would be appreciated very much if an automatic workflow creating the packages can be implemented to test the current "bleeding edge" version of ReaR without much manual tasks in a reproducible way.
gdha commented at 2021-07-05 13:34:¶
@jsmeix @hpannenb We used to have a nightly job to build new rpm/deb after git updates, but we re-used this box for other means. Due to lack of sponsoring we stopped this activity on a nightly basis.
hpannenb commented at 2021-07-05 13:34:¶
Not knowing much of the SuSE OBS the issue might be related to
rear.spec (as already managed in #2289). According to
https://spdx.org/licenses/ the GPL-3.0
seems to be deprecated.
hpannenb commented at 2021-07-05 13:36:¶
Due to lack of sponsoring we stopped this activity on a nightly basis.
I was under the impression this was automagically created via OBS where download links under Snapshot releases from Git are referencing to as well.
Edit: Ok. Seems to be quiet new OBS can be integrated with GitHub/GitLab as mentioned in Continuous Integration with OBS and GitHub/GitLab This has "beta" state but is heading towards the proper direction.
jsmeix commented at 2021-07-06 09:07:¶
I never used OBS source services.
So here my very first attempt:
I did not understand anything in
https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.source_service.html
so I did it in script-kiddie mode based on the examples in
https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.best-practices.scm_integration.html
I made now
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-github-master
but that builds only for some recent SUSE systems
with Fedora_33 as the only non-SUSE system.
So it seems it could get hard to make OBS source services working also
for all the other systems.
So I tried a simpler and more manual approach based on manual
wget https://github.com/rear/rear/archive/refs/heads/master.zip
which resulted
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-snapshot
that builds for all SUSE systems and most Red Hat based systems
but fails for some Red Hat based systems and
fails for all Debian based systems.
I cannot fix build on Debian based systems
because I know nothing about the Debian build system.
gdha commented at 2021-07-06 09:21:¶
@jsmeix @hpannenb It seems to work the OBS workflow after a PR, albeit,
with a new name (see picture):
Bottomline is now we have an automated manner to create ReaR snapshots after each PR - thanks @hpannenb for the golden tip
jsmeix commented at 2021-07-06 10:30:¶
@gdha
I fear it may not work as you assume because
I think that new rear-2.6.5-1.el8.x86_64.rpm RPM may come from my above
testing
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-snapshot
because this is what I get as binary RPM from that source package
# osc getbinaries Archiving:Backup:Rear:Snapshot rear-snapshot CentOS_8 x86_64
...
rear-2.6.5-1.el8.x86_64.rpm
In particular note the special version 2.6.5 which is what I specified
in
https://build.opensuse.org/package/view_file/Archiving:Backup:Rear:Snapshot/rear-snapshot/rear.spec?expand=1
# Current snapshots are after ReaR 2.6 and before ReaR 2.7 i.e. between 2.6 and 2.7
Version: 2.6.5
gdha commented at 2021-07-06 10:54:¶
@jsmeix According https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649/rear it did work.
jsmeix commented at 2021-07-06 11:09:¶
https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649/rear
results this binary RPM:
# osc getbinaries home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649 rear CentOS_8 x86_64
...
rear-2.6-146.git.4317.be8b6ed.master.el8.x86_64.rpm
I think the 'unpublished' in
# osc results -v home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649 rear | grep CentOS_8
CentOS_8 x86_64 succeeded(unpublished)
means that this binary RPM is not made available on OBS download servers.
jsmeix commented at 2021-07-06 11:42:¶
I removed my testing Archiving:Backup:Rear:Snapshot rear-snapshot
and added Archiving:Backup:Rear:Snapshot rear-manual-snapshot
which builds for all systems.
The built binary RPMs have rather simple names like
# osc getbinaries Archiving:Backup:Rear:Snapshot rear-manual-snapshot openSUSE_Leap_15.2 x86_64
...
rear-2.6.5-1.x86_64.rpm
where currently only the special version number '2.6.5' indicates
it is a state between the current latest ReaR release 2.6
and the upcoming next ReaR release 2.7.
jsmeix commented at 2021-07-06 11:48:¶
@hpannenb
if you like you may try out if RPMs from
Archiving:Backup:Rear:Snapshot rear-manual-snapshot
i.e. binary RPMs with currently special version number '2.6.5'
work for you.
Be careful - I did not at all test those RPMs.
@gdha
when your automated build works well we could remove that
Archiving:Backup:Rear:Snapshot rear-manual-snapshot
or we could keep it (perhaps with some enhancements)
so that we could - if we like - provide RPMs for a certain fixed
snapshot state
instead of a too much moving target that rebuilds for each upstream
change?
gdha commented at 2021-07-06 11:52:¶
@jsmeix sure why not - kind of pre-releases
hpannenb commented at 2021-07-06 13:21:¶
Glad to see You managed the interworking between OBS and GitHub. Honestly it was just "by accident" I found this SCM integration hint at OBS. It is rather new.
@hpannenb
if you like you may try out if RPMs from
Archiving:Backup:Rear:Snapshot rear-manual-snapshot
i.e. binary RPMs with currently special version number '2.6.5'
work for you.
Be careful - I did not at all test those RPMs.
I am currently checking the Centos_7 RPM within one of my (K)VMs from
here
https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/CentOS_7/x86_64/
I will let You know once I installed it.
Edit: Installation of the 2.6.5 RPM works perfectly fine.
rear-centos-7-rpm-check.txt
jsmeix commented at 2021-07-06 14:02:¶
@hpannenb
thank you for your prompt feedback!
It is much appreciated.
gdha commented at 2021-07-09 09:41:¶
@jsmeix I think that the PR github workflow towards OBS is user bounded, so if you, merge a PR nothing will happen. To make your OBS account linked to your GibHub account follow the procedure found at https://openbuildservice.org/2021/05/31/scm-integration/
jsmeix commented at 2021-07-12 08:43:¶
I had never before let computers act on their own on as if they were me.
https://openbuildservice.org/2021/05/31/scm-integration/
reads (excerpts):
You have to give OBS permissions
to report the build status to the pull/merge request to the SCM
on your behalf.
For this, you have to create a personal access token.
...
You also have to give your SCM permissions
to trigger a workflow inside OBS
on your behalf.
That’s why you need to create an OBS access token.
...
Tokens are like the keys to (parts of) your house.
You have to keep your token secret
to prevent someone else
from triggering operations
in your name!
This is a contradiction in itself.
Via the tokens
I give the programmers of the SCM (GitHub in my case)
and the programmers of the OBS
permissions
to do things in my name
because what actually happens is not what I myself do
but what programmers of those workflows/operations implement
where I have no control in practice what actually happens
(in practice I cannot verify their source code).
Of course when I log in at any foreign service and manually do things
there
I depend on what the programmers implemented for what I do
i.e. I let their programs run on their servers on my behalf, cf.
https://www.gnu.org/philosophy/who-does-that-server-really-serve.html
The difference - as far as I understand it - is that
with access tokens I give them some kind of key/password
while via login at any foreign service and manually do things there
I do not give them some kind of key/password
to let them do arbitrary things as if it was me
(under the assumtion they do not store my clear text login password).
Perhaps it might be acceptable for me if I had a separated robot
account
where I could specify limited permissions as I like and then
do such access token things only via the restricted robot account.
Then it could be at least clear what happened via my robot account
versus what I myself did via my normal user account.
jsmeix commented at 2021-07-15 11:59:¶
@gdha
when I made
https://github.com/rear/rear/pull/2658
I noticed that it triggered a rebuild of some rear package in OBS
home:gdha:Archiving:Backup:Rear:Snapshot...
At
https://github.com/rear/rear/pull/2658
click on Show all checks and there the Details links all point to
https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
But the commits in this pull request belong to a branch jsmeix-fix2622
and not to master.
I think only commits to master should trigger builds in OBS
but not commits to (development work in progress) branches.
On the other hand when pull requests create sepatated OBS projects
like
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658
the additional builds are separated but I wonder where the built RPMs
appear?
On
https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
the link Download package is
https://software.opensuse.org//download.html?project=home%3Agdha%3AArchiving%3ABackup%3ARear%3ASnapshot%3APR-2658&package=rear
but it only results
No data for home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658 / rear
gdha commented at 2021-07-15 14:55:¶
@jsmeix See https://build.opensuse.org/repositories/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear see the reposity rows which contains the deb/rpms to download.
jsmeix commented at 2021-07-16 07:22:¶
@gdha
thank you - now I see and understand!
https://build.opensuse.org/repositories/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
shows that all "Publish Flag"s are disabled.
But on
https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
I can click in the "Build Results" list on a repository e.g. "CentOS_7"
which leads to
https://build.opensuse.org/package/binaries/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear/CentOS_7
and there are download links to the source RPM and the binary RPM
e.g. the binary RPM links is
https://build.opensuse.org/package/binary/download/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear/CentOS_7/x86_64/rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm
When I am not logged in at OBS and
click on that binary RPM "Download" link
I get a page that shows
Please login to access the requested page.
Also
# wget https://build.opensuse.org/package/binary/download/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear/CentOS_7/x86_64/rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm
...
2021-07-16 09:04:06 (0.00 B/s) - ‘rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm’ saved [0/0]
# file rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm
rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm: empty
does not provide the actual binary RPM.
In contrast when I am logged in at OBS and
click on that binary RPM "Download" link
I get the actual binary RPM.
Conclusion:
When the "Publish Flag" is disabled it means the built binary RPM
will not appear in a commonly known public accessible repository for
binary RPMs
but the built binary RPM is still available via a direct "Download"
link
for prople who can log in at OBS.
I don't know if there are certain restrictions for those who can log in
at OBS
which binary RPMs they can access via such direct "Download" links.
Because
https://build.opensuse.org/package/users/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
lists only
https://build.opensuse.org/users/gdha
as the only user of the OBS project
"home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658"
I assume everybody who can log in at OBS can get source RPM and binary
RPM
via those direct "Download" links.
I.e. when the "Publish Flag" is disabled it does not mean
the source RPM and the binary RPM are not accessible for others.
github-actions commented at 2021-09-15 02:09:¶
Stale issue message
hpannenb commented at 2021-09-15 16:44:¶
Prevent the bot from auto-closing.
hpannenb commented at 2022-02-08 16:26:¶
@jsmeix and @gdha: Does this topic intentionally stay closed?
jsmeix commented at 2022-02-09 08:08:¶
According to
# osc search rear | grep 'home:gdha:Archiving:Backup:Rear:Snapshot'
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2650 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2655 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2656 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2659 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2660 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2661 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2662 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2664 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2665 rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2675 rear
it seems that automatism has shomehow stopped after pull request 2675.
I have no idea why because I do not understand how that works.
I gave up because I could never make it work for me, cf.
https://github.com/rear/rear/issues/2645#issuecomment-878090407
As time permits I will update my
Archiving:Backup:Rear:Snapshot rear-manual-snapshot
jsmeix commented at 2022-02-09 09:14:¶
Was much easier than expected because I made comments in
https://build.opensuse.org/package/view_file/Archiving:Backup:Rear:Snapshot/rear-manual-snapshot/rear.spec?expand=1
how to make its Source0
# How to make Source0:
# Download current ReaR upstream GitHub master.zip:
# # wget https://github.com/rear/rear/archive/refs/heads/master.zip
# Unzip it (unzips into rear-master directory):
# # unzip -q master.zip
# Rename the rear-master unzip directory:
# # mv rear-master rear-snapshot
# Tar it (use the above 'Version' value):
# # tar -czf rear-snapshot-2.6.5.tar.gz rear-snapshot
# Remove master.zip and the rear-snapshot directory:
# # rm -r master.zip rear-snapshot
Source0: rear-snapshot-%{version}.tar.gz
so I did that and - voila! - here you are:
https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/openSUSE_Leap_15.3/x86_64/
contains a new rear-2.6.5-2.x86_64.rpm dated right now 09-Feb-2022
08:53
For other distributions see what there is under
https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/
The RPM package file must be named rear-2.6.5...
in particular its version 2.6.5 indicates it is from the sources in
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-manual-snapshot
github-actions commented at 2022-04-11 02:58:¶
Stale issue message
hpannenb commented at 2022-04-18 07:18:¶
Hmppfff...the bot is bot-hering. :-)
jsmeix commented at 2022-04-20 08:04:¶
At least the bot reminds one from time to time about older issues...
jsmeix commented at 2022-04-20 08:05:¶
...that are still of interest.
github-actions commented at 2022-06-20 03:06:¶
Stale issue message
pcahyna commented at 2022-06-20 08:23:¶
@jsmeix @hpannenb I am not familiar with this build service at all, but maybe the openSUSE build targets in Packit #2823 could provide at least a partial replacement?
schlomo commented at 2023-02-27 17:14:¶
@jsmeix it seems like the snapshot builds again stopped to work, at least the last RPM in https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/15.4/x86_64/ is from last year and the last DEB in https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/Debian_11/amd64/ is also from last year.
It would be nice if we can get the snapshot builds to work again.
jsmeix commented at 2023-02-28 12:43:¶
I have no personal interest in automated builds and
in particular not in automated snapshot builds.
My personal reasons:
I do not want to get one single user issue/bug report
when something does not (yet) work properly in his case
because it is under development but a blind automatism
made a useless snapshot build where of course the user
cannot know he got a (partially) inconsistent state
(often users blindly install just the highest version).
Automated (snapshot) builds enforce us at ReaR upstream
to never merge a pull request that requires further work
by subsequent pull requests until all is properly done
which means in particular we can never again merge a
somewhat not yet complete pull request from a user
who kindly contributed something that works for him
but does not yet work sufficiently well in general.
I do not want to be enforced to enforce users who
contributed something to enhance their contribution
until it works sufficiently well in general.
In particular not because in most cases I cannot verify
what a user contributed because I don't have his environment
or I am busy with other (higher priority) things
when a user's pull request appears.
As far as I know automated builds require to let computers
act on their own on as if they were me, see my above
https://github.com/rear/rear/issues/2645#issuecomment-878090407
Users who like to test our current ReaR GitHub master code
don't need automated snapshot builds, see the section
"Testing current ReaR upstream GitHub master code" in
https://en.opensuse.org/SDB:Disaster_Recovery
See also
https://github.com/rear/rear/issues/2923#issuecomment-1427820436
(excerpt)
... everybody can use our current ReaR upstream
GitHub master code via "git clone" and from there
also get any needed state before via "git checkout"
As far as I can imagine only users who like to actually use
our current ReaR GitHub master code ask for snapshot builds
so that they can install a built software package as usual
on their systems and use that normally on their systems
but this is exactly not what they should do with some
GitHub master code from a random (snapshot) point in time.
@schlomo @rear/contributors and ReaR users:
if you personally want automated (snapshot) builds,
feel free to implement it as you think it works best and
then also continuously maintain what you implemented
for some years in the future and/or find other ReaR
upstream maintainers and/or ReaR users who also can
implement and maintain it or at least help with it.
FWIW:
I neither have special openSUSE Build Service knowledge
nor special openSUSE Build Service rights or permissions.
I am only a normal end-user of the openSUSE Build Service.
[Export of Github issue for rear/rear.]