#1449 PR merged: Add non blocking userprompt with timeout if no multipath device detected.

Labels: enhancement, cleanup, fixed / solved / done

schabrolles opened issue at 2017-08-21 07:08:

Some Linux distribution (like sles11) has a version of multipath which return an error when dm-multipath kernel module is loaded but no multipath devices are detected.

This produce an exit to rear_shell waiting for UserInput.

I propose to change it to a UserInput with 30s delay in order to avoid to block the recovery process when BOOT_ON_SAN=yes but no multipath device are found (in case of migration for example).

schabrolles commented at 2017-08-21 10:12:

@gdha UserInput message updated.
Thanks for your feedback.

schabrolles commented at 2017-08-21 11:30:

@schlomo,

  • originally, this script only activate multipath if the Backup system was multipath (which was not enough in case of V2P).
    The following line was already present is this version of script and was activated if multipath loading return a non 0 value.
rear_shell "Did you activate the multipath devices?"

My concern here is that on some older Linux distribution (like sles11) multipath return an non 0 value if no multipath device is discovered. It is not the case with rhel7 or ubuntu16.04.
It could be a normal situation (as you said P2V). In this case, during recovery sles11 will exit to rear-shell after the following LogPrompt, and wait for user action (no other explanation/guidance).

Failed to activate multipath, or no multipath device found.
rear>

Again, there is no issue with newer multipath version (rhel7 for example). multipath return 0 if it runs without error and no multipath device discovered.

I just would like to improve the user experience by having a better message for this "non critical" error (no multipath discovered) and avoid to block a recovery (with a timeout).

What about this one :

Failed to activate multipath, or no multipath device found.
Choice: 
1) multipath is not needed, please continue recovery. (you should update your rear conf to remove BOOT_ON_SAN). 
2) Enter into rear-shell to debug multipath. (manually run "multipath -r" command)
(default 1) 
(timeout 30 seconds)

gdha commented at 2017-08-21 11:43:

@schabrolles Yes - that is indeed much better to understand for me and the end-user ;-)

schabrolles commented at 2017-08-21 16:19:

Here is an updated version with menu (inspired from @jsmeix work on 300_map_disk.sh).

if multipath command failed, user will get this menu:

Activating multipath
Failed to activate multipath, or no multipath device found.

Choice:
1) Multipath is not needed, please continue recovery.
2) Run multipath with debug options.
3) Enter into rear-shell to manually debug multipath.
4) Abort 'rear recover'
(default 1 timeout 30 seconds)

If user press 1) a message advice him to remove BOOT_ON_SAN parameter if multipath is not needed on this system.

...
Continuing 'rear recover' by default
If you don't need multipath on this server, you should consider removing BOOT_ON_SAN parameter from your rear configuration file.
multipath activated
No devices found
...

Here is an example of a recovery on a sles11 which migrate from P2V (multipath => single path).

Relax-and-Recover 2.2 / Git
Using log file: /var/log/rear/rear-rear-sles11-144.log
Running workflow recover within the ReaR rescue/recovery system
Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
Started RPC portmapper 'rpcbind'.
RPC portmapper 'rpcbind' available.
RPC status rpc.statd available.
Using backup archive '/tmp/rear.sjcrkAKIQO08Uo5/outputfs/rear-sles11-144/backup.tar.gz'
Will do driver migration (recreating initramfs/initrd)
Calculating backup archive size
Backup archive size is 1.6G     /tmp/rear.sjcrkAKIQO08Uo5/outputfs/rear-sles11-144/backup.tar.gz (compressed)
Setting up multipathing
Activating multipath
Failed to activate multipath, or no multipath device found.

Choice:
1) Multipath is not needed, please continue recovery.
2) Run multipath with debug options.
3) Enter into rear-shell to manually debug multipath.
4) Abort 'rear recover'
(default 1 timeout 30 seconds)
2
starting multipath -v3 (debug mode)
Aug 21 17:44:26 | libdevmapper version 1.03.01 (2011-10-15)
Aug 21 17:44:26 | DM multipath kernel driver v1.6.0
Aug 21 17:44:26 | loading /lib64/multipath/libcheckdirectio.so checker
Aug 21 17:44:26 | loading /lib64/multipath/libprioconst.so prioritizer
Aug 21 17:44:26 | vda: blacklisted, udev property missing
Aug 21 17:44:26 | sr0: udev property ID_SCSI_VPD whitelisted
Aug 21 17:44:26 | sr0: device node name blacklisted
Aug 21 17:44:26 | unloading const prioritizer
Aug 21 17:44:26 | unloading directio checker

Choice:
1) Multipath is not needed, please continue recovery.
2) Run multipath with debug options.
3) Enter into rear-shell to manually debug multipath.
4) Abort 'rear recover'
(default 1 timeout 30 seconds)
3

Welcome to Relax-and-Recover.

rear> multipath -ll
rear> exit
Do you want to go back to 'rear recover' ? y
exit

Choice:
1) Multipath is not needed, please continue recovery.
2) Run multipath with debug options.
3) Enter into rear-shell to manually debug multipath.
4) Abort 'rear recover'
(default 1 timeout 30 seconds)
Continuing 'rear recover' by default
If you don't need multipath on this server, you should consider removing BOOT_ON_SAN parameter from your rear configuration file.
multipath activated
No devices found
Comparing disks.
Disk configuration is identical, proceeding with restore.
Start system layout restoration.
Creating partitions for disk /dev/vda (msdos)
Creating LVM PV /dev/vda3
  0 logical volume(s) in volume group "rootvg" now active
Restoring LVM VG rootvg
Sleeping 3 seconds to let udev or systemd-udevd create their devices...
Creating filesystem of type ext3 with mount point / on /dev/mapper/rootvg-lvroot.
2 bytes [53 ef] erased at offset 0x438 (ext3)
Mounting filesystem /
Creating filesystem of type ext3 with mount point /boot on /dev/vda2.
2 bytes [53 ef] erased at offset 0x438 (ext3)
Mounting filesystem /boot
Creating swap on /dev/mapper/rootvg-lv_swap
Disk layout created.
Restoring from '/tmp/rear.sjcrkAKIQO08Uo5/outputfs/rear-sles11-144/backup.tar.gz'...
Restored 4070 MiB [avg. 78645 KiB/sec] OK
Restored 4108 MiB in 54 seconds [avg. 77901 KiB/sec]
Restoring finished.
Restore the Mountpoints (with permissions) from /var/lib/rear/recovery/mountpoint_permissions
Running mkinitrd...
Recreated initrd (/sbin/mkinitrd).
Installing PPC PReP Boot partition.
Boot partion found in lilo.conf:  /dev/vda1
Running LILO ...
running on chrp
/dev/vda
/dev/vda1
Boot target is /dev/vda1
add note section for RS6K
Using built-in yaboot.conf
Prepending  '/pci@800000020000000/scsi@4' to open firmware variable boot-device
Finished recovering your system. You can explore it under '/mnt/local'.
RESCUE rear-sles11-144:~ #

schabrolles commented at 2017-08-22 10:35:

@schlomo you are absolutely right. Here is a slight update with your suggestion applied.
thanks.

jsmeix commented at 2017-08-23 14:24:

@schabrolles
I do appreciate it so much that you use the new UserInput
function because I need testing and feedback to improve it.

Ideally I would like to have the UserInput function in a stable
and well usable state for the ReaR 2.3 release but if we need
more time for testing and enhancing it until it really works
sufficiently well for us it is also perfectly o.k. for me.

jsmeix commented at 2017-08-23 14:53:

@schlomo
I think I found something for you:

LogPrint "WARNING: ..."

;-)

@schabrolles
for background information what I am talking about see
http://blog.schlomo.schapiro.org/2015/04/warning-is-waste-of-my-time.html

schabrolles commented at 2017-08-23 15:04:

@jsmeix @schlomo

The objective here was to really "warn" the user about a possible configuration cleanup. (after a P2V for example). It is a "literally" warning :-) ... But I'm open to suggestion.

schlomo commented at 2017-08-23 15:07:

@schabrolles simple: remove the word WARNING:

schabrolles commented at 2017-08-23 15:16:

Ok, but I just found it was not visible enough.

schlomo commented at 2017-08-23 15:25:

Messages to the user should be either informational or actionable. WARNING is neither nor so it doesn't help the user. IMHO in this specific case we should mention it but not make a fuss. If the user gets tired of the manual step in his recovery he will also notice the message and act. If he doesn't care then we also don't care.

schlomo commented at 2017-08-23 16:41:

If it is important then add this log output to the UserInput function for
everybody

Am 23.08.2017 6:31 nachm. schrieb "Johannes Meixner" <
notifications@github.com>:

@jsmeix commented on this pull request.

In usr/share/rear/layout/prepare/GNU/Linux/210_load_multipath.sh
https://github.com/rear/rear/pull/1449#discussion_r134803306:

  • choices[1]="Run multipath with debug options."

  • choices[2]="Enter into rear-shell to manually debug multipath."

  • choices[3]="Abort '$rear_workflow'"

  • prompt="Choice:"

  • choice=""

  • wilful_input=""

    • looping on this menu while multipath failed to list device.

  • Note: On sles11/rhel6, multipath failed if no multipath device is found.

  • while ! multipath ; do

  • echo

  • choice="$( UserInput -t 30 -p "$prompt" -D "${choices[0]}" "${choices[@]}")"&& wilful_input="yes" || wilful_input="no"

  • case "$choice" in

  • (${choices[0]})

  • continue recovery without multipath

  • is_true "$wilful_input" && LogPrint "User confirmed continuing without multipath" || LogPrint "Continuing '$rear_workflow' by default"

I do care ;-)
I like to have it visible in the log file
whether or not the user actively confirmed the default action
or if the user basically ignored the input request and the default
action "just happened".
A precondition is that there is plenty of time so that the user
can read and understand what he is asked and make a
delibarate decision (sooner than the automated timeout acts).


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rear/rear/pull/1449#discussion_r134803306, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAGMCJtx2fmHwT06PqH8OipY-vAHcJ5Rks5sbFPCgaJpZM4O89WF
.

schabrolles commented at 2017-08-23 16:50:

@jsmeix @schlomo
changes (use default timeout 300s) and (remove WARING) applied and tested.

jsmeix commented at 2017-08-23 17:09:

@schlomo
thanks for your
https://github.com/rear/rear/pull/1449#issuecomment-324393351
I will enhance the UserInput function so that no "wilful_input" stuff
is needed for each and every call of the UserInput function.

jsmeix commented at 2017-08-23 17:17:

The outer

    multipath >&2
        if [ $? -ne 0 ] ; then
            ...
            LogPrint "multipath activated"
            dmsetup ls --target multipath
            multipath -l
        else
            LogPrint "multipath activated"
            dmsetup ls --target multipath

still looks somehow suspicious to me, cf.
https://github.com/rear/rear/pull/1449/files#r134810570

Simply put:
I think currently the "Multipath is not needed" case
does not behave well, see also in the "rear recover" log in
https://github.com/rear/rear/pull/1449#issuecomment-323789500

Choice:
1) Multipath is not needed, please continue recovery.
...
Continuing 'rear recover' by default
...
multipath activated

i.e. the "multipath activated" message that does not match
the decision that "Multipath is not needed" indicates
that things do not work perfectly clean in this case.

schabrolles commented at 2017-08-23 22:31:

@jsmeix @schlomo
Last update for today, I've re-written a bit the UserPrompt while loop.
Now the "Multipath activated" is only print on purpose.

jsmeix commented at 2017-08-24 09:07:

@schabrolles
many thanks for all your efforts with all our various
comments and improvement requests.
Not it looks o.k. from my point of view
so that you could merge it if you like.

Unfortunately (for you) I cannot restrain
to add one more comment:

I wonder about the outermost 'if' clause.
Currently the script has the form

if CONDITION1 || CONDITION2 ; then
    # Setting up multipathing
fi
function create_multipath() {
   ...
}

The trailing function definition looks somewhat strange to me.

What looks even more strange to me is that I can find
'create_multipath' only in that function definition
but I cannot find where it is ever called.

@schabrolles
could you add a comment why the create_multipath function
is defined (i.e. needed) in any case even if neither CONDITION1
nor CONDITION2 is fulfilled or remove that create_multipath
function definition if it is really nowhere needed.

jsmeix commented at 2017-08-24 09:17:

@schlomo
for bigger changes I always look at the whole resulting file
because usually my understanding gets lost in bigger diffs ;-)

jsmeix commented at 2017-08-24 09:25:

@gdha
I assume your request for changes in
https://github.com/rear/rear/pull/1449#pullrequestreview-57441577
got meanwhile perfecty fulfilled so that you should also accept it.

jsmeix commented at 2017-08-24 11:59:

@schabrolles see
https://github.com/rear/rear/pull/1449#discussion_r134994017

Summary how to output stuff and have it also in the log:

For all commands where you like to show
only the command STDOUT to the user
when ReaR is run in verbose mode (rear -v)
and have both STDOUT and STDERR also in the log use

LogPrint "$( COMMAND )"

For all commands where you like to show
both the command STDOUT and STDERR to the user
when ReaR is run in verbose mode (rear -v)
and have both STDOUT and STDERR also in the log use

LogPrint "$( COMMAND 2>&1 )"

For all commands where you like to show
only the command STDOUT to the user in any case
(i.e. regardless if ReaR is run in verbose mode)
and have both STDOUT and STDERR also in the log use

LogUserOutput "$( COMMAND )"

For all commands where you like to show
both the command STDOUT and STDERR to the user
in any case (i.e. regardless if ReaR is run in verbose mode)
and have both STDOUT and STDERR also in the log use

LogUserOutput "$( COMMAND 2>&1 )"

jsmeix commented at 2017-08-24 12:21:

@schlomo
I tried to also use LogPrint/LogUserOutput as wrapper
for commands that need unser input with

touch /tmp/foo
LogUserOutput "$( rm -vi /tmp/qqq /tmp/foo /tmp/QQQ 0<&6 2>&1 )"

but that let ReaR fail (it just exits at that rm command).
I have to use

touch /tmp/foo
LogUserOutput "$( rm -vi /tmp/qqq /tmp/foo /tmp/QQQ 0<&6 1>/dev/tty 2>&1 )"

which works on the terminal

e205:~/rear.master # usr/sbin/rear mkrescue
rm: cannot remove '/tmp/qqq': No such file or directory
rm: remove regular empty file '/tmp/foo'? y
removed '/tmp/foo'
rm: cannot remove '/tmp/QQQ': No such file or directory

but that does of course no longer produce agruments
for the LogUserOutput function so that there is
nothing in the log.

Do you have an idea how to get STDOUT and STDERR
also in the log when commands need user input?

schabrolles commented at 2017-08-24 18:30:

@jsmeix you are right regarding creat_multipath() function.
This function was there before I start working on ReaR... But no script is calling it....

I remove it and made a test with sles11-sp4 (V2P: non-multipath to multipath) => OK

schlomo commented at 2017-08-25 06:39:

@jsmeix I would have expected something like 1>&7 2>&8 or such in your rm example. Although for me this doesn't make much sense, I would expect us to use LogPrint only for stuff that doesn't need user input because that would be user input that ReaR cannot automate.

If we have the need to LogPrint the result of a command more often then I would suggest to create a new function LogPrintExec or LogPrintCommand or LogPrintRun that simply runs $@

jsmeix commented at 2017-08-25 09:53:

As usual sleeping over it helped:

Caution with 'create_...' functions!

Many of them look as if they were only defined
because one cannot find where they are callled,
for example also the create_disk function:

# find usr/sbin/rear usr/share/rear/ | xargs grep 'create_disk'

usr/share/rear/layout/prepare/GNU/Linux
/100_include_partition_code.sh:create_disk() {

Many 'create_...' functions are not called by their full name.
Instead the are called indirectly via things like 'create_$type'
where $type evaluates e.g. to 'disk' 'part' 'fs' 'swap'
(i.e. $type is a component type in disklayout.conf)
by more generic 'create_...' functions
in lib/layout-functions.sh for example like

create_device() {
    local device="$1"
    local type="$2"
    ...
    if type -t create_$type >/dev/null ; then
        create_$type "$device"
    fi

By the way:
This is another nice example for RFC 1925 item 6a
cf. https://www.ietf.org/rfc/rfc1925.txt
"It is always possible to add another level of indirection."
Personally I think that is the ultimate root cause
of 90% of the problems and the other ultimate root cause
for also 90% of the problems is RFC 1925 item 5
which means (assuming stochastical independence)
that 81% of the problems have RFC 1925 items 5 and 6a
together as their ultimate root cause...

In a "rear -d -D recover" log file things look like
(excerpts that show only the 'create_...' functions):

+ source /usr/share/rear/layout/prepare/default/540_generate_device_code.sh
++ create_device /dev/sda disk
++ type -t create_disk
++ create_disk /dev/sda
++ create_partitions /dev/sda msdos
++ create_device /dev/sda1 part
++ type -t create_part
++ create_device /dev/sda2 part
++ type -t create_part
++ create_device fs:/ fs
++ type -t create_fs
++ create_fs fs:/
++ create_device swap:/dev/sda1 swap
++ type -t create_swap
++ create_swap swap:/dev/sda1

+ source /usr/share/rear/layout/recreate/default/200_run_script.sh
+++ create_component vgchange rear
+++ create_component /dev/sda disk
+++ create_component /dev/sda1 part
+++ create_component /dev/sda2 part
+++ create_component fs:/ fs
+++ create_component swap:/dev/sda1 swap

@schabrolles
accordingly you need to carefully double-check that the
create_multipath function is really no longer ever needed
in particular on a system that uses multipath
or more generally when there are active components

multipath ...

in disklayout.conf (the line starts with 'multipath ').

Note that components in disklayout.conf can be
deactivated by commenting them out e.g. like

# multipath ...

Deactivating components in disklayout.conf can happen
automatically by "rear recover" (e.g. when the original system
had multipath but on the replacement hardware or after a
system migration multipath is no longer needed.
Deactivating components in disklayout.conf can also be
done manually by the user either by manual editing disklayout.conf
in the ReaR recovery system before he runs "rear recover" or
during "rear recover" in migration mode when there are those
dialogs that let the user edit disklayout.conf.

Deactivated components must be ignored
which is - as far as I see - a bug in

create_multipath() {
    local multipath device
    read multipath device junk < <(grep "multipath $1 " "$LAYOUT_FILE")
    create_partitions "$device"
}

because it greps for "multipath" which also finds "# multipath".
Normally I would expect it to be

    read multipath device junk < <(grep "^multipath $1 " "$LAYOUT_FILE")

to get only the active '^multipath ' components in disklayout.conf.

jsmeix commented at 2017-08-25 10:07:

@schlomo
yes, to run an interactive command with the original STDIN,
STDOUT, and STDERR one should use e.g. as in

touch /tmp/foo
rm -vi /tmp/qqq /tmp/foo /tmp/QQQ 0<&6 1>&7 2>&8

The '0<&6' is currently not needed because STDIN is not
redirected - I prefer the whole '0<&6 1>&7 2>&8' term
to be on the safe side and to make it 100% clear
that "all I/O via the original FDs" is meant.

This way it "just works" on the user's terminal

# usr/sbin/rear mkrescue
rm: cannot remove '/tmp/qqq': No such file or directory
rm: remove regular empty file '/tmp/foo'? y
removed '/tmp/foo'
rm: cannot remove '/tmp/QQQ': No such file or directory

but then nothing is in the log.

Things get somewhat tricky when one wants to have
all output both on the user's terminal and in the log.
Then a 'tee' process in between is needed.

jsmeix commented at 2017-08-25 10:22:

@schabrolles

another special thing in ReaR in verbose mode (rear -v)
is that then in usr/sbin/rear theer is

# Set the variables v and verbose to be used in called programs
# that support the '-v' or '--verbose' options like
# "cp $v ..." or "rm $verbose ..." and so on:
v=""
verbose=""
if test "$VERBOSE" ; then
    v="-v"
    verbose="--verbose"
fi

Accordingly commands that support '-v' or '--verbose'
should usually be called as

COMMAND $v

or

COMMAND $verbose

Because on my SLES11 and SLES12 systems
"man dmsetup" shows "-v" and "--verbose"
I suggest you may call dmsetup with $verbose like

    # Search and list mpath device.
    if is_true $list_mpath_device ; then
        LogPrint "Listing multipath device found"
        LogPrint "$(dmsetup ls --target multipath $verbose 2>&1)"
    fi

except dmsetup verbosity is explicitly not wanted in this case
(perhaps then dmsetup may show too much that is not helpful).

jsmeix commented at 2017-08-25 11:39:

@schabrolles
initially you had started this with 5 lines of code changed
and it had become some kind of a monstrous thing
because of all our comments, improvement requests,
and side-topics.

I do very much appreciate your endurance here
that in the end results a totally overhauled script
that looks now very nice and clear.

I hope you agree that this result is worth all those efforts.

Feel free to merge it when it is o.k. for you!

schabrolles commented at 2017-08-25 19:58:

@jsmeix no problem at all... I do appreciate your careful review / feedback (same for you @schlomo)

  • you are right regarding create_multipath() function... we finally REALLY need it.
    I do additional test which confirm it cannot work without it (partition are not created)
    In fact my server reboot properly after my previous test because the partition were already there (no need to recreate them) But If I wipe the disk before the recovery with dd, it fails to create LVM because no partition are created ... which is normal ....

  • Thanks for the info about $v and $verbose. I try dmsetup ls --target multipath -v but it seems to have no impact on the output (sles11-sp4):

# dmsetup ls --target multipath
mpatha  (252, 0)
# dmsetup ls --target multipath -v
mpatha  (252, 0)

I commit again with the function create_multipath back in place.

jsmeix commented at 2017-08-28 09:32:

@schabrolles
I think if we already had the early "cleanupdisk" script
as proposed in https://github.com/rear/rear/issues/799
such kind of accidentally positive testing results
would no longer happen.

jsmeix commented at 2017-08-28 10:30:

@schabrolles
what about the possible bug in the create_multipath()
function as I wrote in
https://github.com/rear/rear/pull/1449#issuecomment-324874100

For me the current create_multipath() function looks strange
(somehow knotted regrading $1 and $device).
As far as I understand what it should do
I would use something like

# Define the create_multipath() function in any case
# even if there is neither '^multipath' in disklayout.conf
# nor BOOT_OVER_SAN is true (cf. the above condition)
# to be on the safe side if create_multipath() might be
# also needed in whatever other circumstances:
function create_multipath () {
    local device=$1
    if grep "^multipath $device " "$LAYOUT_FILE" 1>&2 ; then
        Log "Found multipath device $device: Creating partitions on it"
        create_partitions "$device"
    fi
}

schabrolles commented at 2017-08-28 10:57:

@jsmeix

  • Agree that the way this function was written is strange and should be re-written as you propose

2 additional question:

  • Why do you redirect output of grep to stderr ? (I guess it should be related to the logging...)
  • In some circumstance, the log message "Found multipath device $device: Creating partitions on it" could be strange....

if you migrate from Multipathed to Not Multipathed system,

  • entry "multipath mpatha ....." is created in layout.conf

  • After booting to the Rescue media on the target system (not multipathed) and starting the recovery, the device mpatha in the layout file is migrated to sda by the device-migration scripts.
    Then the message "Found multipath device sda: Creating partitions on it".... The weird thing is that sda is technically NOT a multipath device ... Everything is working fine : we need to create the partition on a device (multipath or not). It is just the Log message generated that is a bit strange in this case.

jsmeix commented at 2017-08-28 11:34:

@schabrolles
in general I redirect STDOUT to STDERR to get
command output that is not really meant for the user
not on the user's terminal but instead in the log file.
I perefer '1>&2' over '1>/dev/null' because in general
ReaR should not destroy information.
In general the intent is to have all information
of a "rear ..." run in the log file so that reading the log file
(and only the log file) is sufficient to get full information
what happened.
Therefore I prefer e.g. 'grep ... 1>&2' over 'grep -q ...'.
Only if a command output is really not useful or even
misleading, then '1>/dev/null' and/or '2>/dev/null'
should be used preferably together with a comment why
redirection to /dev/null makes sense in this case, cf.
https://github.com/rear/rear/issues/1395#issuecomment-311916095

Thanks for your explanation how the Log message
could become plain wong and therefore misleading.
Feel free to adapt it as you like, e.g.

Log "Found device $device: Creating partitions on it"

or

Log "Found current or former multipath device $device: Creating partitions on it"

or whatever you prefer (ultimately it is your code).
In general misleading Log messages are not critical
because Log messages are not shown directly to the user
in contrast to LogPrint or even LogUserOutput messages.

schabrolles commented at 2017-08-30 08:20:

@jsmeix @gdha @schlomo Last review before I merge this one.

jsmeix commented at 2017-08-30 09:16:

I think this is another example how one can make "good" code
even with the bash programming language where I often
hear from colleagues when I tell them about ReaR that
the first thing they would do is to completely rewrite it
in a "real programming language" - of course they never
contributed anything to ReaR - they are blabbering rubbish
without understanding anything about why bash is perfectly
"the right thing" for ReaR.

schabrolles commented at 2017-08-30 19:29:

@gdha are you now satisfied with this PR ?


[Export of Github issue for rear/rear.]