#2942 Issue open: '-c' and '-C' options are ignored with 'automatic/auto_recover' and 'unattended' modes¶

Labels: bug

jsmeix opened issue at 2023-02-21 13:19:¶

I found an issue with our current '-c' and '-C' options:

The 'automatic/auto_recover' and 'unattended' modes
currently call only

rear $rear_debug_options recover

see
https://github.com/rear/rear/blob/rear-2.7/usr/share/rear/skel/default/etc/scripts/system-setup#L155
and
https://github.com/rear/rear/blob/rear-2.7/usr/share/rear/skel/default/etc/scripts/system-setup#L192

So '-c' and '-C' are ignored / not possible
with our current 'automatic/auto_recover' and 'unattended' modes
which is a bug that needs to be fixed.

Cf.
https://github.com/rear/rear/issues/2936#issuecomment-1438462224

jsmeix commented at 2023-02-21 13:28:¶

My current offhanded idea how to fix that generically
is to store the whole 'rear [options] workflow' command line
of each last called workflow in the ReaR recovery system.

Because only "rear mkrescue" and "rear mkbackup" create
a ReaR recovery system only those workflows can store
their command line in the ReaR recovery system.

So some "rear special_option mkbackuponly" cannot
store its command line in the ReaR recovery system.

Cf.
https://github.com/rear/rear/issues/2936#issuecomment-1438475287

schlomo commented at 2023-02-21 19:59:¶

FYI, we copy the directory specified by -c (and stored as CONFIG_DIR) into the rescue system as /etc/rear so that there would be no point to remember and use -c for rear recover:
https://github.com/rear/rear/blob/ea900c4ec9589c7acb3d90c01f75ad7781fbdd8c/usr/share/rear/build/GNU/Linux/100_copy_as_is.sh#L165

It seems like primarily -C remains a problem and I totally agree with the goal that automated recovery should also work for -C users. My suggestion would be to use CONFIG_APPEND_FILES_PATHS in order to set CONFIG_APPEND_FILES in rescue.conf to again load the same files that had been loaded successfully and do that during the rescue stage. That way this will only happen for ReaR runs that actually generate a rescue image and not for others.

I can code that up quickly if we agree on the concept.

jsmeix commented at 2023-02-22 09:08:¶

@schlomo
please be patient.
There is zero time pressure here.
Basically nobody uses '-c' or '-C' plus
'automatic/auto_recover' or 'unattended'
because the current state is since it was introduced
and I cannot remember any user report about it.
Since a very long time I didn't use any of '-c' or '-C'
or 'automatic/auto_recover' or 'unattended'.

I need time to understand how '-c' and '-C' behave
when used with different values for different workflows, cf.
https://github.com/rear/rear/issues/2936#issuecomment-1434304552
which is in particular how '-C' is primarily meant to be used, see
https://github.com/rear/rear/blob/master/doc/user-guide/11-multiple-backups.adoc

Then I can make educated statements in this area
and agree or disagree on your proposal.

It is not about quick coding.
We have already way too much quickly implemented code in ReaR
for limited purpose that poorly integrates with the rest of ReaR
(e.g. the whole OUTPUT=USB functionality is an endless annoyance).

jsmeix commented at 2023-02-23 12:41:¶

The primary use case for '-C' as described in
https://github.com/rear/rear/blob/master/doc/user-guide/11-multiple-backups.adoc
is to specify things in

/etc/rear/local.conf
/etc/rear/basic_system.conf
/etc/rear/home_backup.conf
/etc/rear/opt_backup.conf

then run on the original system

# rear mkrescue
# rear -C basic_system mkbackuponly
# rear -C home_backup mkbackuponly
# rear -C opt_backup mkbackuponly

and to recreate that on replacement hardware
run there in the ReaR recovery system

# rear -C basic_system recover
# rear -C home_backup restoreonly
# rear -C opt_backup restoreonly

Currently I have no idea how to automatically
run those commands in the ReaR recovery system
i.e. how to automatically find out
while on the original system plain

# rear mkrescue

is run (which is the only comand that creates
or that can modify the ReaR recovery system)
that later for disaster recovery
in the ReaR recovery system something like

# rear -C basic_system recover && rear -C home_backup restoreonly && rear -C opt_backup restoreonly

should be run.

jsmeix commented at 2023-02-23 12:49:¶

@schlomo
I think why your proposal
https://github.com/rear/rear/issues/2942#issuecomment-1439021298
cannot work in general is that according to your

again load the same files that had been loaded successfully

it seems you implicitly assume there is always
a single recovery command (usually 'rear recover')
to recreate a system so it would be sufficient to
source (actually there is no such thing as 'load' a config file)
a single sequence of config files during recovery.

jsmeix commented at 2023-02-23 12:55:¶

I have an idea how to make a sequence of recovery commands
possible with 'automatic/auto_recover' and 'unattended' modes
but that won't happen automatically but must be setup manually
by the user.

My reasoning is:
The (experienced) user who set up the "mkbackup" procedure
knows what sequence of recovery commands (with right options)
are needed to recreate things properly so he can manually
specify the right sequence of recovery commands.
Then even an unexperienced user can do the recovery.

jsmeix commented at 2023-03-01 13:28:¶

Mainly for my own information:

How to wait for several simultaneously running background jobs
and show some info what happened to them:

# ( sleep 1 ; echo 1 started ; sleep 9 ; echo 1 succeeded ) & \
  ( sleep 2 ; echo 2 started ; sleep 6 ; echo 2 failed ; exit 2 ) & \
  ( sleep 3 ; echo 3 started ; sleep 3 ; echo 3 failed ; exit 3 ) & \
  while jobs % &>/dev/null ; \
  do echo waiting for a job to finish ; \
     wait -n && echo a job succeeded || echo a job failed with $? ; \
  done ; \
  echo all finished

[1] 27123
[2] 27124
[3] 27125
waiting for a job to finish
1 started
2 started
3 started
3 failed
[3]+  Exit 3                  ( sleep 3; echo 3 started; sleep 3; echo 3 failed; exit 3 )
a job failed with 3
waiting for a job to finish
2 failed
[2]+  Exit 2                  ( sleep 2; echo 2 started; sleep 6; echo 2 failed; exit 2 )
a job failed with 2
waiting for a job to finish
1 succeeded
[1]+  Done                    ( sleep 1; echo 1 started; sleep 9; echo 1 succeeded )
a job succeeded
all finished

So the above shown recovery command

# rear -C basic_system recover && \
  rear -C home_backup restoreonly && \
  rear -C opt_backup restoreonly

could be run with the two 'restoreonly' workflows in parallel like

# if rear -C basic_system recover ; \
  then rear -C home_backup restoreonly & \
       rear -C opt_backup restoreonly & \
       wait ; \
  fi

where the plain wait for the background jobs
could be generically enhanced to show some info
what happened to the background jobs like

# if rear -C basic_system recover ; \
  then rear -C home_backup restoreonly & \
       rear -C opt_backup restoreonly & \
       while jobs % &>/dev/null ; \
       do echo Waiting for a restoreonly workflow to finish ; \
          wait -n || echo ERROR: A restoreonly workflow failed with $? ; \
       done ; \
  fi

github-actions commented at 2023-05-01 02:21:¶

Stale issue message

---

[Export of Github issue for rear/rear.]