#2941 Issue open: RFC: switch from bash 3 to bash 4 as minimum required bash version

Labels: discuss / RFC, severe improvement

jsmeix opened issue at 2023-02-21 12:15:

In https://github.com/rear/rear/wiki/Coding-Style
we state that bash 3 is the minimum required bash version.

What Linux distribution versions we support at ReaR upstream
is documented in our release notes, e.g. for ReaR 2.7 see
https://github.com/rear/rear/blob/rear-2.7/doc/rear-release-notes.txt#L3814

ReaR 2.7 is supported on the following Linux based operating systems:

-   Fedora 29, 30, 31, 32, 33, and 34
-   RHEL 6, 7, 8, and 9
-   CentOS 6, 7, and 8
-   Scientific Linux 6 and 7
-   SLES 12 and 15
-   openSUSE Leap 15.x
-   Debian 8, and 9
-   Ubuntu 16, 17, and 18

Regarding SUSE and openSUSE products:
SLES 12 SP 5 has GNU bash version 4.3.48
SLES 15 and openSUSE Leap 15.x have even newer bash versions

jsmeix commented at 2023-02-21 12:16:

@gdha @pcahyna @schlomo

could you please check what
Fedora, RHEL, and Ubuntu versions provide bash 4.x

(no extraordinary efforts - just check what you have available)

pcahyna commented at 2023-02-21 16:20:

several years ago I needed to test ReaR code in bash 3 and I needed RHEL 5 machines for that, RHEL 6 has bash 4.

schlomo commented at 2023-02-21 21:42:

I just added via d0e2ea6 a little tool to the ReaR sources to make such checks simple:

$ tests/run-in-docker : centos:5 -- "type mapfile &>/dev/null || echo no mapfile in Bash \$BASH_VERSION"
********** ubuntu:18.04                             **********
********** ubuntu:20.04                             **********
********** ubuntu:22.04                             **********
********** ubuntu:devel                             **********
********** debian:8                                 **********
********** debian:9                                 **********
********** debian:10                                **********
********** debian:11                                **********
********** debian:unstable                          **********
********** opensuse/leap:15.4                       **********
********** opensuse/leap:15                         **********
********** centos:6                                 **********
********** centos:7                                 **********
********** centos:8                                 **********
********** sl:6                                     **********
********** sl:7                                     **********
********** quay.io/centos/centos:stream8            **********
********** quay.io/centos/centos:stream9            **********
********** fedora:29                                **********
********** fedora:30                                **********
********** fedora:31                                **********
********** fedora:32                                **********
********** fedora:33                                **********
********** fedora:34                                **********
********** fedora:35                                **********
********** fedora:36                                **********
********** fedora:37                                **********
********** fedora:38                                **********
********** fedora:rawhide                           **********
********** centos:5                                 **********
no mapfile in Bash 3.2.25(1)-release

About the list you give here @jsmeix I would like to suggest that ReaR as upstream project needs only to support distros that are currently being maintained. Specifically that reduces the list for the upcoming ReaR 2.8 IMHO to the following:

Everything older I would like to consider as "good luck" and point people to the last ReaR release supporting those officially, which would be ReaR 2.7.

There is much more to old software than just bash, with older distros not support certain programs or having them in old versions that lack features.

Here is one example where I check for switch_root which is required for the ramdisk code:

$ tests/run-in-docker : centos:5 -- "type switch_root"
********** ubuntu:18.04                             **********
switch_root is /sbin/switch_root
********** ubuntu:20.04                             **********
switch_root is /usr/sbin/switch_root
********** ubuntu:22.04                             **********
switch_root is /usr/sbin/switch_root
********** ubuntu:devel                             **********
switch_root is /usr/sbin/switch_root
********** debian:8                                 **********
switch_root is /sbin/switch_root
********** debian:9                                 **********
switch_root is /sbin/switch_root
********** debian:10                                **********
switch_root is /sbin/switch_root
********** debian:11                                **********
switch_root is /sbin/switch_root
********** debian:unstable                          **********
switch_root is /usr/sbin/switch_root
********** opensuse/leap:15.4                       **********
switch_root is /usr/sbin/switch_root
********** opensuse/leap:15                         **********
switch_root is /usr/sbin/switch_root
********** centos:6                                 **********
/bin/bash: line 1: type: switch_root: not found
********** centos:7                                 **********
switch_root is /usr/sbin/switch_root
********** centos:8                                 **********
switch_root is /usr/sbin/switch_root
********** sl:6                                     **********
/bin/bash: line 1: type: switch_root: not found
********** sl:7                                     **********
switch_root is /usr/sbin/switch_root
********** quay.io/centos/centos:stream8            **********
switch_root is /usr/sbin/switch_root
********** quay.io/centos/centos:stream9            **********
/bin/bash: line 1: type: switch_root: not found
********** fedora:29                                **********
switch_root is /usr/sbin/switch_root
********** fedora:30                                **********
switch_root is /usr/sbin/switch_root
********** fedora:31                                **********
switch_root is /usr/sbin/switch_root
********** fedora:32                                **********
switch_root is /usr/sbin/switch_root
********** fedora:33                                **********
switch_root is /usr/sbin/switch_root
********** fedora:34                                **********
switch_root is /usr/sbin/switch_root
********** fedora:35                                **********
/bin/bash: line 1: type: switch_root: not found
********** fedora:36                                **********
switch_root is /usr/sbin/switch_root
********** fedora:37                                **********
switch_root is /usr/sbin/switch_root
********** fedora:38                                **********
switch_root is /usr/sbin/switch_root
********** fedora:rawhide                           **********
switch_root is /usr/sbin/switch_root
********** centos:5                                 **********
/bin/bash: line 1: type: switch_root: not found

And here is another example checking for JSON support in the ip command, which is super useful to simplify the parsing of ip output:

$ tests/run-in-docker debian -- "{ apt update ; apt install -y iproute2 ; } &>/dev/null ; ip -json link list dev lo"
********** debian:8                                 **********
Option "-json" is unknown, try "ip -help".
********** debian:9                                 **********
Option "-json" is unknown, try "ip -help".
********** debian:10                                **********
[{"ifindex":1,"ifname":"lo","flags":["LOOPBACK","UP","LOWER_UP"],"mtu":65536,"qdisc":"noqueue","operstate":"UNKNOWN","linkmode":"DEFAULT","group":"default","txqlen":1000,"link_type":"loopback","address":"00:00:00:00:00:00","broadcast":"00:00:00:00:00:00"}]
********** debian:11                                **********
[{"ifindex":1,"ifname":"lo","flags":["LOOPBACK","UP","LOWER_UP"],"mtu":65536,"qdisc":"noqueue","operstate":"UNKNOWN","linkmode":"DEFAULT","group":"default","txqlen":1000,"link_type":"loopback","address":"00:00:00:00:00:00","broadcast":"00:00:00:00:00:00"}]
********** debian:unstable                          **********
[{"ifindex":1,"ifname":"lo","flags":["LOOPBACK","UP","LOWER_UP"],"mtu":65536,"qdisc":"noqueue","operstate":"UNKNOWN","linkmode":"DEFAULT","group":"default","txqlen":1000,"link_type":"loopback","address":"00:00:00:00:00:00","broadcast":"00:00:00:00:00:00"}]

For brevity sake only on Debian, and we see that Debian 8 and Debian 9 are too old for that, but we could use ip -json if we could decide to only support Debian 10, 11 and newer with ReaR 2.8 and following.

As enterprise distros are in any case not very eager to update ReaR we can make use of that and cut off more old tails and move forward a little bit faster, making our development and testing efforts so much easier.

jsmeix commented at 2023-02-22 08:38:

Personally I fully agree that ReaR upstream needs only
to support distros that are currently being maintained
but some users may expect something different, e.g. see
https://github.com/rear/rear/issues/2923

ReaR upstream support is not and never was something
that users can demand.
ReaR upstream support is and always was voluntary
"best effort" attempts.

For ReaR 2.8 I will overhaul the "Support" section
in the release notes.

My current plan is to replace lists with version numbers
by a generic text that does not need continuous maintenance
to keep version numbers and such mostly meaningless data
continuously up to date, i.e. similar like in
https://github.com/rear/rear.github.com/pull/16

In our current ReaR 2.7 release notes
I updated what I know about SUSE and openSUSE
but I left all the other Linux distributions
basically unchanged because I cannot spend time on
non-SUSE Linux distributions specific things because
SUSE does not pay me for that and what matters most for me:
I will not spend SUSE customer's money who pay my salary
for non-SUSE Linux distributions specific things.
The crucial word is 'specific'.
When e.g. a RHEL user reports an issue that is generic
for Linux distributions I work on it because fixing it
also helps SUSE and openSUSE users.
I am even happy when it is e.g. a RHEL user who reports
a generic issue because then I could have a fix
or a workaround or at least I know about the issue
before a SUSE or openSUSE user is hit by it.

schlomo commented at 2023-02-22 08:48:

BTW, maybe we should also consider going to a 3.0 release instead of 2.8 in order to clearly signal that we cut off support for older distros.

jsmeix commented at 2023-02-22 10:47:

Yes of course.
Major backward incompatible changes
require a different major version number.

For me '2.8' is currently only a placeholder
for "the next" ReaR version.

After the ReaR upstream maintainers agreed to actually do
major backward incompatible changes for "the next" ReaR version
I will do the needed changes in GitHub to have
'3.0' officially as "the next" ReaR version.

schlomo commented at 2023-02-22 15:52:

In light of the other discussion around deprecation, e.g. in https://github.com/rear/rear/issues/2944#issuecomment-1439975174, would we try to use that for Bash 3 deprecation too or would we just decide that the next ReaR version requires Bash 4?

DEvil0000 commented at 2023-02-22 15:57:

I agree with changing to bash4 and ip. For my use case this would be fine - this however may not be the case for other users. Especially on slower updating/more stable distros like RHEL.

About the list you give here @jsmeix I would like to suggest that ReaR as upstream project needs only to support distros that are currently being maintained. Specifically that reduces the list for the upcoming ReaR 2.8 IMHO to the following:

Please define "being maintained". Ubuntu LTS versions for example are maintained for 10 years (half free and public, half paid). Our customers for example require 10 years of support for any HW and SW. Which normally means at least +2 more years until it is in the field.

schlomo commented at 2023-02-22 16:03:

@DEvil0000 I expect that "maintenance" doesn't mean getting the latest greatest versions but getting support for actual bugs in the old version that came with the distro so many years ago. So whatever we decide here about the upstream ReaR shouldn't have any impact on the LTS distros.

You mention the paid support for the long tail of Ubuntu LTS for example, I'm sure that @gdha for example would be able to provide bugfixes or whatever is required as part of the ReaR Software Subscription if a customer wants to cover ReaR improvements for their old LTS distro.

DEvil0000 commented at 2023-02-22 16:19:

I agree, I do normally not need a newer rear version in a old system and I am very hesitant to change it. This is due to the amount of testing needed with all the different HW/SW/config combinations. Also I normally have patched versions out there since PRs do not result in new versions that fast - and well testing again.
On the other side I of course would like to use a newer version of rear after PRs got merged - also in older OS versions.
As you see my issues are more with the release cycle and testing effort (due to stability concerns). Otherwise I would love to go with newer versions on older OS releases (till in theory to ubuntu 14.04 - in reality ubuntu 20.04).
He however would for sure not be happy patching big chunks just for a few machines per OS version.

edit:
Long story short: I am fine with the change but I can see how others may not be fine with it.

schlomo commented at 2023-02-22 16:24:

About the slow releases, I agree. I was once thinking about automatically doing a release after merging a PR so that we would make that code much faster available to users.

DEvil0000 commented at 2023-02-22 16:32:

also note that rear packaging of the distros is even slower since they orient on github releases nowadays and if there is no release when they check then there will be no new rear package in the distro.
some distros may check automatically by scripts while others more manually every few years in their release cycle.

edit:
if some test with the release fails (if they test) then they skip packaging the release.

jsmeix commented at 2023-02-23 10:04:

Regarding
https://github.com/rear/rear/issues/2941#issuecomment-1440295322

From my current point of view
I would call the next ReaR version '3.0' and
switch from bash 3 to bash 4 as minimum required bash version
(i.e. error out when sbin/rear is run with bash 3).

My reasoning:
We cannot verify with reasonable effort
if code changes actually require bash 4
or still also work with bash 3
so arbitrary failures at arbitrary places will happen
if we would still allow to run sbin/rear with bash 3.

Of course a user can manually change his sbin/rear and
skip this error exit to let it still run with bash 3
if that is OK for him for his particular use case.

jsmeix commented at 2023-02-23 10:31:

@schlomo
because in
https://github.com/rear/rear/pull/2847#issuecomment-1440843109
you wrote (excerpt)

... teach shellcheck to consider Bash 4.1 as "reference" ...

I wonder if any "Bash 4" (in particular also Bash 4.0)
could be perhaps a too low minimum requirement
so actually it should be at least Bash 4.1?

According to
https://tldp.org/LDP/abs/html/bashver4.html

Version 4.1 of Bash ... was primarily a bugfix update.

so it seems any "Bash 4" as minimum requirement
is sufficient in ReaR.

Of course bugfixes are good in general but I wonder
if we must enforce that in ReaR?

schlomo commented at 2023-02-23 10:55:

I would only check for the major version, like this:

(( BASH_VERSINFO[0] >= 4 )) || echo "too old"

github-actions commented at 2023-04-25 02:21:

Stale issue message

pcahyna commented at 2023-04-25 09:54:

@jsmeix

I would call the next ReaR version '3.0' and switch from bash 3 to bash 4 as minimum required bash version (i.e. error out when sbin/rear is run with bash 3).

I would not call next ReaR version '3.0' - it is not a user-visible change unless one uses a distro which still has bash 3 - and then it falls under deprecation of older distros - which, IMO, should happen continuously - i.e we should not announce a major release only because Ubuntu 14.04 or RHEL 6 or whatever support got removed.

OTOH, I think releasing '3.0' as an opportunity fort incompatible changes is a good idea, but that's a separate discussion (everyone might have different opinions on what "incompatible changes" they would like to make, if any).

schlomo commented at 2023-04-25 13:49:

For the Bash change I'd keep the major version of 2 and simply error out if somebody runs it on very old distros.

For a 3.0 version I'd love to get rid of much more painful things like supporting BIOS boot, all the older backup software (e.g. all CommVault below GALAXY11) etc. For example, I had a look at OUTPUT=ISO for UEFI boot and it seems to be a lot of complexity thanks to the combined BIOS/UEFI boot in the same code path.

And yes, I know thanks to #2944 that we will be keeping BIOS boot for much longer. Maybe I'd be happy with splitting ISO into ISO for BIOS boot and a new EISO or UEFI-ISO for just UEFI boot and use that to simplify the code. And maybe that would also warrant a 3.0 version as it would actually remove existing functionality from ISO, thereby forcing users to check their setup when they upgrade.

pcahyna commented at 2023-05-09 16:47:

One new Bash feature that we could make use of is the lastpipe option:

     If set, and job control is not active, the shell runs the last
     command of a pipeline not executed in the background in the
     current shell environment.

it would simplify some constructs, but would require newer bash than 4 (RHEL 7 has bash 4.2.46(2), which supports this feaure, while RHEL 6 has bas 4.1.2(1), which does not).

[Export of Github issue for rear/rear.]