#1366 Issue closed: Avoid 'sit-and-wait predator' and timeout bash 'select' lists with a default/fallback value¶
Labels: enhancement, fixed / solved / done
jsmeix opened issue at 2017-05-16 11:18:¶
During ReaR recovery system startup I get from
etc/scripts/system-setup.d/55-migrate-network-devices.sh
The original network device eth0 12:34... is not available. Please select another device: 1) eth0 56:78... 2) Skip replacing the network device Choose the network device to use:
and there it behaves like a 'sit-and-wait predator'
(cf.
https://en.wikipedia.org/wiki/Ambush_predator
)
and waits endlessly for my input.
For my use-case it is only a minor annoyance but
I think this could become a major hindrance
for any kind of automated "rear recover" run.
Hereby I propose to enhace it with an appropriate timeout
so that it automactially proceeds with a default/fallback value
when there is no user input.
In particular in cases like the above when there is only one
actual choice it could usually safely automactially proceed.
What else except the one actual choice could the user do
in practice - I mean in practice the user wants to proceed.
The implementation problem is that bash 'select'
has no timeout but bash 'read' has one so that
I must probably replace 'select' by 'read -t'.
schlomo commented at 2017-05-16 12:05:¶
👍 I was also already annoyed by the two question for disklayout.conf and diskrestore.sh which I would like to default to "5" (continue) at least during an automated recovery.
If you rebuild select with read then please kindly do so in a
function. Maybe something that allows easily to convert a
select ... done clause into a case ... esac clause with similar
content.
jsmeix commented at 2017-05-16 12:18:¶
@schlomo
in this case even I agree to provide such general useful
functionality in a global function.
Furthermore with 'read' one can - as needed - avoid
the additional [Enter] key, just typing a single character
and off it goes like
REPLY=x ; read -t 3 -n 1 -p 'type a char (timeout 3 sec) ' ; echo ; echo using $REPLY
schlomo commented at 2017-05-16 12:20:¶
The disadvantage of single character input is that the user has no chance to fix an error, e.g. you meant 1 but hit 2 and off it goes. With the Enter key the user can think if he really wanted to give that input.
Hence I would prefer to wait for Enter and actually time out with a sensible default or abort where appropriate.
jsmeix commented at 2017-05-16 12:31:¶
I meant the 'read -n 1' behaviour only optionally.
With 'read' we could do that, with 'select' it is impossible.
By default [Enter] is required.
Right now I tried an automated recovery and
at the end I get a 'select' dialog that waits forever
'rear recover' finished successfully 1) View Relax-and-Recover log file(s) 2) Go to Relax-and-Recover shell 3) Reboot Select what to do
I would prefer an automated reboot
(provided "rear recover" results zero exit code)
after some timeout.
gdha commented at 2017-05-16 13:59:¶
@jsmeix @schlomo Have you ever used the MIGRATION_MODE variable? That
is switched off when I run in unattended recovery mode. I also use 2
variables PXE_RECOVER_MODE or ISO_RECOVER_MODE to trigger the
behaviour.
schlomo commented at 2017-05-16 14:08:¶
The rear-workshop stuff auto-enables MIGRATION_MODE because it
recovers to a different hard disk. So yes.
jsmeix commented at 2017-05-17 12:45:¶
@gdha
I cannot find usage of MIGRATION_MODE in anything
except scripts in usr/share/rear/layout
$ find usr/sbin/rear usr/share/rear/ | xargs grep -l 'MIGRATION_MODE' usr/share/rear/layout/prepare/GNU/Linux/100_include_partition_code.sh usr/share/rear/layout/prepare/GNU/Linux/110_include_lvm_code.sh usr/share/rear/layout/prepare/default/010_prepare_files.sh usr/share/rear/layout/prepare/default/250_compare_disks.sh usr/share/rear/layout/prepare/default/300_map_disks.sh usr/share/rear/layout/prepare/default/320_apply_mappings.sh usr/share/rear/layout/prepare/default/400_autoresize_disks.sh usr/share/rear/layout/prepare/default/500_confirm_layout.sh usr/share/rear/layout/prepare/default/270_overrule_migration_mode.sh usr/share/rear/layout/recreate/default/100_ask_confirmation.sh
but here we talk in particular also about user input that is
requested during revovery system startup i.e. in the
usr/share/rear/skel/default/etc/scripts/system-setup*
scripts.
I think we should not mix up MIGRATION_MODE functionality
with new functionality how user input requests behave.
I think the latter should be implemented separated e.g.
via a new config variable like USER_INPUT_TIMEOUT
or USER_INPUT_WAIT_SECONDS.
jsmeix commented at 2017-05-17 14:49:¶
Currently I am playing around with a function
that replaces 'select' by 'read'.
In principle I got something working but the
signature (i.e. its arguments and its output)
of my current function is ugly.
My basic problem is that I would have liked to
have an array of choices as first function argument
and after that have optional other arguments to specify
things like timeout (if not the default timeout should be used)
like
choices=( 'foo and bar' 'this and that' 'something else' )
timeout=30
choice="$( selectread "${choices[@]}" $timeout )"
case "$choice" in
...
esac
Because one cannot pass an array as one function argument
cf.
http://stackoverflow.com/questions/16461656/bash-how-to-pass-array-as-an-argument-to-a-function
the choices must be the last function arguments
an all possibly optional arguments must be first
always specified as positional parameters like
choices=( 'foo and bar' 'this and that' 'something else' )
prompt="select something"
default="${choices[2]}"
timeout=30
choice="$( selectread "$prompt" "$default" $timeout "${choices[@]}" )"
case "$choice" in
...
esac
schlomo commented at 2017-05-17 15:02:¶
Why not simply use getopts and named arguments? Something like this
selectread -t 60 -p "Please choose one" "${choices[@]}"
IMHO a much cleaner interface and you can have default values for arguments not specified.
jsmeix commented at 2017-05-18 07:40:¶
@schlomo
many thanks - that helps!
Sometimes I do not see the forest for the trees.
jsmeix commented at 2017-05-19 12:07:¶
I have something that seems to work not too bad for me
but I did not yet test it very much.
Currently it is a stand-alone file choose.sh
that uses no ReaR functions therefore for now
there are things like plain "echo ... 1>&2"
that will be later replaced by appropriate
ReaR functions (e.g. Log).
During development I use "set -e -u -o pipefail"
which shows very nicely "too optimistic code"
(at least for me).
I use non-zero return codes according to
https://github.com/rear/rear/issues/1134
Simple example how it could be used:
# ( source choose.sh ; choices=( 'foo and bar' 'this and that' 'something else' ) ; choose "${choices[@]}" )
1) foo and bar
2) this and that
3) something else
Your input (default 1 timeout 60) 2
User chose 'this and that'
this and that
More complicated example:
# ( source choose.sh ; choices=( 'foo and bar' 'this and that' 'something else' ) ; choose -a myarr -d q "${choices[@]}" || true ; for e in "${myarr[@]}" ; do echo "'$e'" ; done )
1) foo and bar
2) this and that
3) something else
Your input (default 1 timeout 60) 3 foo bar bazq
User chose 'something else'
something else
'3'
'foo'
'bar'
'baz'
Finally the whole script choose.sh
#!/bin/bash
USER_INPUT_TIMEOUT=60
USER_INPUT_PROMPT="Your input"
USER_INPUT_MAX_CHARS=1000
function choose () {
set -e -u -o pipefail
# Set defaults or fallback values:
local timeout=60
# Avoid stderr if USER_INPUT_TIMEOUT is not set or empty and ignore wrong USER_INPUT_TIMEOUT:
test "$USER_INPUT_TIMEOUT" -ge 0 2>/dev/null && timeout=$USER_INPUT_TIMEOUT
local prompt="enter a choice number"
# Avoid stderr if USER_INPUT_PROMPT is not set or empty:
test "$USER_INPUT_PROMPT" 2>/dev/null && prompt="$USER_INPUT_PROMPT"
local output_array=""
local input_max_chars=1000
# Avoid stderr if USER_INPUT_MAX_CHARS is not set or empty and ignore wrong USER_INPUT_MAX_CHARS:
test "$USER_INPUT_MAX_CHARS" -ge 0 2>/dev/null && input_max_chars=$USER_INPUT_MAX_CHARS
local input_delimiter=""
local default_choice_index=0
# Get the options and their arguments:
local option=""
while getopts ":t:p:a:n:d:D:" option ; do
case $option in
(t)
test "$OPTARG" -ge 0 && timeout=$OPTARG || echo "Invalid -$option argument '$OPTARG' using fallback '$timeout'" 1>&2
;;
(p)
prompt="$OPTARG"
;;
(a)
output_array="$OPTARG"
;;
(n)
test "$OPTARG" -ge 0 && input_max_chars=$OPTARG || echo "Invalid -$option argument '$OPTARG' using fallback '$input_max_chars'" 1>&2
;;
(d)
input_delimiter="$OPTARG"
;;
(D)
test "$OPTARG" -ge 0 && default_choice_index=$OPTARG || echo "Invalid -$option argument '$OPTARG' using fallback '$default_choice_index'" 1>&2
;;
(\?)
echo "Invalid option: -$OPTARG" 1>&2
;;
(:)
echo "Option -$OPTARG requires an argument." 1>&2
;;
esac
done
# Shift away the options and arguments:
shift "$(( OPTIND - 1 ))"
# Everything that is now left in "$@" is neither an option nor an option argument
# so that now "$@" contains the trailing mass-arguments (POSIX calls them operands):
local choices=( "$@" )
test "$choices" || echo "No choices" 1>&2
test "${choices[$default_choice_index]}" || echo "No default choice" 1>&2
# When an empty prompt was specified (via -p '') do not change that:
test "$prompt" && prompt="$prompt (default $(( default_choice_index + 1 )) timeout $timeout) "
# The actual work:
# Show the choices with leading choice numbers 1) 2) 3) ... as in 'select' (i.e. starting at 1):
local choice_number=1
for choice in "${choices[@]}" ; do
echo "$choice_number) $choice" 1>&2
(( choice_number += 1 ))
done
# Prepare the 'read' call:
local read_options_and_arguments=""
# When a zero timeout was specified (via -t 0) do not use it:
test "$timeout" -ge 1 && read_options_and_arguments="$read_options_and_arguments -t $timeout"
# When no output_array was specified (via -a myarr) do not use it:
test "$output_array" && read_options_and_arguments="$read_options_and_arguments -a $output_array"
# When zero input_max_chars was specified (via -n 0) do not use it:
test "$input_max_chars" -ge 1 && read_options_and_arguments="$read_options_and_arguments -n $input_max_chars"
# When no input_delimiter was specified (via -d x) do not use it:
test "$input_delimiter" && read_options_and_arguments="$read_options_and_arguments -d $input_delimiter"
# Let the default user input match the default choice:
local user_input=$(( default_choice_index + 1 ))
# Read the user input:
if ! read $read_options_and_arguments -p "$prompt" user_input ; then
# Continue in any case because in case of errors the default choice is used:
echo -e "\n'read' finished with non-zero exit code" 1>&2
# In particular continue after a timeout that lets 'read' exit with non-zero exit code:
test "$timeout" -ge 1 && echo "probably 'read' timed out" 1>&2
fi
# When an output_array was specified it contains all user input words but we use only the first word:
test "$output_array" && user_input="${!output_array}"
# Use default choice for wrong user input:
if ! test "$user_input" -ge 1 ; then
echo "Wrong user input '$user_input' using fallback '${choices[$default_choice_index]}'" 1>&2
echo "${choices[$default_choice_index]}"
return 101
fi
local choice_index=$(( user_input - 1 ))
# Use default choice when there is no choice for the user input
# and avoid "bash: choices[$choice_index]: unbound variable":
if ! test "${choices[$choice_index]:-}" ; then
echo "No choice for '$user_input' using fallback '${choices[$default_choice_index]}'" 1>&2
echo "${choices[$default_choice_index]}"
return 102
fi
# Success:
echo "User chose '${choices[$choice_index]}'" 1>&2
echo "${choices[$choice_index]}"
return 0
}
jsmeix commented at 2017-05-19 12:32:¶
I already found some more issues (one needs to reset OPTIND), cf
http://mywiki.wooledge.org/BashFAQ/035#Complex_nonstandard_add-on_utilities
but now even automated retries until the user has explicitly
entered a valid choice work for me:
# ( source choose.sh ; choices=( 'foo and bar' 'this and that' 'something else' ) ; until choose -t 10 -D 99 "${choices[@]}" ; do echo "try again" ; done )
No default choice
1) foo and bar
2) this and that
3) something else
Your input (default 100 timeout 10)
'read' finished with non-zero exit code
probably 'read' timed out
No choice for '100' using fallback ''
try again
No default choice
1) foo and bar
2) this and that
3) something else
Your input (default 100 timeout 10) fubar
-bash: test: fubar: integer expression expected
Wrong user input 'fubar' using fallback ''
try again
No default choice
1) foo and bar
2) this and that
3) something else
Your input (default 100 timeout 10) 7
No choice for '7' using fallback ''
try again
No default choice
1) foo and bar
2) this and that
3) something else
Your input (default 100 timeout 10) 2
User chose 'this and that'
this and that
I think after some more fine-tuning it should be o.k.
to be provided as a global function in ReaR.
Then I can step by step replace existing 'read' and
'select' calls in ReaR by that new general function.
But that will happen after the ReaR 2.1 release.
jsmeix commented at 2017-06-22 07:57:¶
This one is a precondition for
https://github.com/rear/rear/issues/885
cf.
https://github.com/rear/rear/issues/885#issuecomment-310305249
and the comments before.
jsmeix commented at 2017-06-22 14:26:¶
Via
https://github.com/rear/rear/pull/1391
I added new generic UserInput and UserOutput
plus LogUserOutput functions that are intended
to replace current user input functionality that
calls select or read directly.
jsmeix commented at 2017-06-30 11:29:¶
With
https://github.com/rear/rear/pull/1396
merged
a first step is done to get this issue fixed.
jsmeix commented at 2017-07-14 09:48:¶
With
https://github.com/rear/rear/pull/1408
merged
the UserInput function should be feature-complete
now (at least from my current point of view).
Then for ReaR v 2.3 I could replace the existing 'read'
and 'select' calls that get user input by the UserInput function.
This should make ReaR prepared for running unattended
via predefined automated user input values as
USER_INPUT_VALUES[user_input_ID] array members.
Of course this does not help when interactive tools
are called in ReaR like special (external / third-party)
backup and restore software that cannot run unattended.
jsmeix commented at 2017-11-28 11:03:¶
Sufficiently done for ReaR 2.3.
Remaining things can be fixed for ReaR 2.4 or later
as time permits and as users ask for it.
[Export of Github issue for rear/rear.]