#2364 Issue closed: Use simpler, more fail-safe, and much faster syntax ARRAY+=( additional elements ) everywhere¶
Labels: enhancement, cleanup, fixed / solved / done,
minor bug
jsmeix opened issue at 2020-04-14 14:35:¶
This is a follow up of
https://github.com/rear/rear/issues/2220
therein in particular in its initial comment (excerpt)
replace the appending to the array
CLONE_USERS=( "${CLONE_USERS[@]:-}" daemon rpc ... dbus )
by += i.e.
CLONE_USERS+=( daemon rpc ... dbus )
I've verified that += works properly
even in bash 3 and set -ue.
and
https://github.com/rear/rear/issues/2220#issuecomment-529933383
I used the following commands to get where we use
the form ARRAY=( "${ARRAY[@]}" additional elements )
for f in $( find usr/sbin/rear usr/share/rear/ -type f ) ; \
do grep -q '[[:alpha:]_][[:alpha:]_]*=( "${[[:alpha:]_][[:alpha:]_]*\[@\]' $f || continue ; \
echo $f \
grep '[[:alpha:]_][[:alpha:]_]*=( "${[[:alpha:]_][[:alpha:]_]*\[@\]' $f ; \
done
that results 270 cases in 119 files
and for the form ARRAY=("${ARRAY[@]}" additional elements)
the command
for f in $( find usr/sbin/rear usr/share/rear/ -type f ) ; \
do grep -q '[[:alpha:]_][[:alpha:]_]*=("${[[:alpha:]_][[:alpha:]_]*\[@\]' $f || continue ; \
echo $f ; \
grep '[[:alpha:]_][[:alpha:]_]*=("${[[:alpha:]_][[:alpha:]_]*\[@\]' $f ; \
done
that results 2 cases in 2 files.
So it seems the job is doable - not nice to do - but doable ;-)
OliverO2 commented at 2020-04-16 10:40:¶
@jsmeix
Regarding the regexp: I'm using IntelliJ IDEA, which features Java
regexps.
The regexp \(\s*\S+\[@\] found 334 occurrences and seems to work quite
well for one-line and multiline assignments (we'd have to subtract 2
occurrences for your latest commit).
OliverO2 commented at 2020-04-17 15:26:¶
Based on master bec26773 (Update restoreonly-workflow.sh) here are my search results from within IntelliJ IDEA. Unfortunately, the only thing I was able to export in a readable way is a flat list of files. :-/
Targets
Occurrences of '\(\s*\S+\[@\]' in project
Found Occurrences
001_configure_workflow.sh
04-scenarios.adoc
06-layout-configuration.adoc
12-BLOCKCLONE.adoc
15-EFISTUB.adoc
200_check_rsync_relative_option.sh
200_check_rsync_relative_option.sh
200_prep_borg.sh
200_prepare_backup_installation.sh
205_inspect_tar_capabilities.sh
240_kernel_modules.sh
280_include_virtualbox.sh
300_map_disks.sh
300_map_disks.sh
340_add_isofs_module.sh
390_copy_binaries_libraries.sh
400_check_backup_special_files.sh
400_check_tape_drive.sh
400_check_tape_drive.sh
400_prep_ava.sh
400_prep_bacula.sh
400_prep_bareos.sh
400_prep_cdm.sh
400_prep_dp.sh
400_prep_fdrupstream.sh
400_prep_galaxy.sh
400_prep_galaxy.sh
400_prep_galaxy.sh
400_prep_nbkdc.sh
400_prep_nbu.sh
400_prep_nsr.sh
400_prep_sesam.sh
400_prep_tsm.sh
400_prep_yum.sh
400_prep_zypper.sh
400_restore_backup.sh
400_verify_tsm.sh
430_autoresize_all_partitions.sh
500_make_backup.sh
500_ssh.sh
540_choose_backup.sh
540_generate_device_code.sh
540_generate_device_code.sh
55-migrate-network-devices.sh
550_rebuild_initramfs.sh
550_rebuild_initramfs.sh
550_rebuild_initramfs.sh
600_check_rsync_xattr.sh
680_install_PPC_bootlist.sh
680_install_PPC_bootlist.sh
830_copy_kernel_initrd.sh
950_dsmc_save_result_files.sh
950_nsr_save_result_files.sh
checklayout-workflow.sh
dhcp-setup-functions.sh
dump-workflow.sh
finalizeonly-workflow.sh
layout-functions.sh
layoutonly-workflow.sh
linux-functions.sh
mkbackup-workflow.sh
mkrescue-workflow.sh
opaladmin-workflow.sh
rd.sh
rear-release-notes.txt
savelayout-workflow.sh
udev-workflow.sh
uefi-functions.sh
validate-workflow.sh
jsmeix commented at 2020-04-17 15:43:¶
I really would need the matching lines.
Searching in each file is not what I could do with reasonable effort.
What my regexps found was those patterns if they appear on one line:
LettersAndUnderscores=( "${LettersAndUnderscores[@]
LettersAndUnderscores=("${LettersAndUnderscores[@]
LettersAndUnderscores=( ${LettersAndUnderscores[@]
LettersAndUnderscores=(${LettersAndUnderscores[@]
There was nothing found for
LettersAndUnderscores=("${LettersAndUnderscores[@]
For LettersAndUnderscores=(${LettersAndUnderscores[*]
only one match was found:
usr/share/rear/build/OPALPBA/Linux-i386/095_exclude_non_essential_files.sh
Log "Before exclusion: PROGS=(${PROGS[*]})"
Log "After exclusion: PROGS=(${PROGS[*]})"
Log "Before exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
Log "After exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
which looks OK as is.
For LettersAndUnderscores=( ${LettersAndUnderscores[*]
only one match was found:
usr/share/rear/build/default/501_check_ssh_keys.sh
key_files=( ${SSH_UNPROTECTED_PRIVATE_KEYS[*]} )
which is intentionally as is because a comment explains it
https://github.com/rear/rear/blob/master/usr/share/rear/build/default/501_check_ssh_keys.sh#L41
Nothing is found (on one line) for
LettersAndUnderscores=( "${LettersAndUnderscores[*]
LettersAndUnderscores=("${LettersAndUnderscores[*]
jsmeix commented at 2020-04-17 15:45:¶
Next week I will think about a test that finds false
ARRAY=( additional elements )
assignments that actually should have been
ARRAY+=( additional elements )
in particular to detect possible errors I have made here.
OliverO2 commented at 2020-04-17 15:46:¶
Another export based on c5f61051 with matching lines:
Targets
Occurrences of '\(\s*\S+\[@\]' in project
Found Occurrences
001_configure_workflow.sh
if (( ${#OPAL_PBA_PROGS[@]} == 0 && ${#OPAL_PBA_COPY_AS_IS[@]} == 0)) && has_binary plymouth; then
PROGS+=( "${OPAL_PBA_PROGS[@]}" clear )
COPY_AS_IS+=( "${OPAL_PBA_COPY_AS_IS[@]}" )
LIBS+=( "${OPAL_PBA_LIBS[@]}" )
04-scenarios.adoc
COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" )
06-layout-configuration.adoc
EXCLUDE_RECREATE=( "${EXCLUDE_RECREATE[@]}" "fs:/media/backup" )
EXCLUDE_RECREATE=( "${EXCLUDE_RECREATE[@]}" "/dev/backup" )
12-BLOCKCLONE.adoc
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )
`REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone mount.ntfs-3g )`
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
BACKUP_PROG_EXCLUDE=( "${BACKUP_PROG_EXCLUDE[@]}" '/products/*' )
15-EFISTUB.adoc
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} /mnt )
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" snapper chattr lsattr xfs_repair )
COPY_AS_IS=( "${COPY_AS_IS[@]}" /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
200_prep_borg.sh
COPY_AS_IS+=( "${COPY_AS_IS_BORG[@]}" )
PROGS+=( "${PROGS_BORG[@]}" borg locale )
205_inspect_tar_capabilities.sh
echo "BACKUP_PROG_OPTIONS=( ${BACKUP_PROG_OPTIONS[@]} )" >> $ROOTFS_DIR/etc/rear/rescue.conf
210_include_dhclient.sh
dhclients=(${dhclients[@]} "$x")
COPY_AS_IS=( "${COPY_AS_IS[@]}" "/etc/localtime" "/usr/lib/dhcpcd/*" )
PROGS=( "${PROGS[@]}" arping ipcalc usleep "${dhclients[@]}" )
240_kernel_modules.sh
MODULES=(
COPY_AS_IS=(
280_include_virtualbox.sh
MODULES+=( ${VBOX_MODULES[@]} )
COPY_AS_IS+=( ${VBOX_COPY_AS_IS[@]} )
300_map_disks.sh
regular_choices=( "${possible_targets[@]}" "$skip_choice" )
300_map_disks.sh
regular_choices=( "${possible_targets[@]}" "$skip_choice" )
390_copy_binaries_libraries.sh
local all_libs=( "${LIBS[@]}" $( RequiredSharedObjects "${all_binaries[@]}" "${LIBS[@]}" ) )
400_check_backup_special_files.sh
FDRUPSTREAM ) CHECK_CONFIG_FILES+=( "${CHECK_CONFIG_FILES_FDRUPSTREAM[@]}" ) ;;
400_check_tape_drive.sh
PROGS+=( "${PROGS_OBDR[@]}" )
REQUIRED_PROGS+=( "${REQUIRED_PROGS_OBDR[@]}" )
400_check_tape_drive.sh
PROGS+=( "${PROGS_OBDR[@]}" )
REQUIRED_PROGS+=( "${REQUIRED_PROGS_OBDR[@]}" )
400_prep_ava.sh
COPY_AS_IS+=( "${COPY_AS_IS_AVA[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_AVA[@]}" )
PROGS+=( "${PROGS_AVA[@]}" )
400_prep_bacula.sh
COPY_AS_IS+=( "${COPY_AS_IS_BACULA[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_BACULA[@]}" )
PROGS+=( "${PROGS_BACULA[@]}" )
400_prep_bareos.sh
COPY_AS_IS+=( "${COPY_AS_IS_BAREOS[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_BAREOS[@]}" )
PROGS+=( "${PROGS_BAREOS[@]}" )
400_prep_cdm.sh
COPY_AS_IS+=( "${COPY_AS_IS_CDM[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_CDM[@]}" )
PROGS+=( "${PROGS_CDM[@]}" fmt )
400_prep_dp.sh
COPY_AS_IS+=( "${COPY_AS_IS_DP[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_DP[@]}" )
400_prep_fdrupstream.sh
COPY_AS_IS+=( "${COPY_AS_IS_FDRUPSTREAM[@]}" "${FDRUPSTREAM_INSTALL_PATH}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_FDRUPSTREAM[@]}" )
PROGS+=( "${PROGS_FDRUPSTREAM[@]}" )
REQUIRED_PROGS+=( "${REQUIRED_PROGS_FDRUPSTREAM[@]}" col )
400_prep_galaxy.sh
COPY_AS_IS+=( "${COPY_AS_IS_GALAXY[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_GALAXY[@]}" )
400_prep_galaxy.sh
COPY_AS_IS+=( "${COPY_AS_IS_GALAXY10[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_GALAXY10[@]}" )
400_prep_galaxy.sh
COPY_AS_IS+=( "${COPY_AS_IS_GALAXY7[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_GALAXY7[@]}" )
400_prep_nbkdc.sh
COPY_AS_IS=(
COPY_AS_IS_EXCLUDE=(
400_prep_nbu.sh
COPY_AS_IS+=( "${COPY_AS_IS_NBU[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_NBU[@]}" )
PROGS+=( "${PROGS_NBU[@]}" col )
400_prep_nsr.sh
COPY_AS_IS+=( "${COPY_AS_IS_NSR[@]}" )
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_NSR[@]}" )
PROGS+=( "${PROGS_NSR[@]}" fmt )
400_prep_sesam.sh
COPY_AS_IS=(
COPY_AS_IS_EXCLUDE=(
LIBS=(
400_prep_tsm.sh
COPY_AS_IS+=( "${COPY_AS_IS_TSM[@]}"
COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_TSM[@]}" )
PROGS+=( "${PROGS_TSM[@]}" )
400_prep_yum.sh
test "${COPY_AS_IS_YUM[*]:-}" && COPY_AS_IS+=( "${COPY_AS_IS_YUM[@]}" )
test "${COPY_AS_IS_EXCLUDE_YUM[*]:-}" && COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_YUM[@]}" )
test "${REQUIRED_PROGS_YUM[*]:-}" && REQUIRED_PROGS+=( "${REQUIRED_PROGS_YUM[@]}" )
test "${PROGS_YUM[*]:-}" && PROGS+=( "${PROGS_YUM[@]}" )
400_prep_zypper.sh
test "${COPY_AS_IS_ZYPPER[*]:-}" && COPY_AS_IS+=( "${COPY_AS_IS_ZYPPER[@]}" )
test "${COPY_AS_IS_EXCLUDE_ZYPPER[*]:-}" && COPY_AS_IS_EXCLUDE+=( "${COPY_AS_IS_EXCLUDE_ZYPPER[@]}" )
test "${REQUIRED_PROGS_ZYPPER[*]:-}" && REQUIRED_PROGS+=( "${REQUIRED_PROGS_ZYPPER[@]}" )
test "${PROGS_ZYPPER[*]:-}" && PROGS+=( "${PROGS_ZYPPER[@]}" )
400_verify_tsm.sh
DIRECTORY_ENTRIES_TO_RECOVER+=( "${excluded_mountpoints[@]}" )
430_autoresize_all_partitions.sh
if (( ${#partitions[@]} == 0 )) ; then
500_make_backup.sh
pipes_rc=( ${PIPESTATUS[@]} )
pipes_rc=( ${PIPESTATUS[@]} )
500_ssh.sh
copy_as_is_ssh_files=( "${SSH_FILES[@]}" )
contains_visible_char "${copy_as_is_ssh_files[*]}" && COPY_AS_IS+=( "${copy_as_is_ssh_files[@]}" )
540_choose_backup.sh
(( ${#backups[@]} > 0 ))
540_generate_device_code.sh
Debug "deps (${#deps[@]}): ${deps[*]}"
540_generate_device_code.sh
Debug "deps (${#deps[@]}): ${deps[*]}"
55-migrate-network-devices.sh
NEW_DEVICES=( "${NEW_DEVICES[@]/$choice/}" )
550_rebuild_initramfs.sh
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
550_rebuild_initramfs.sh
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
550_rebuild_initramfs.sh
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
600_check_rsync_xattr.sh
RSYNC_OPTIONS=( "${BACKUP_RSYNC_OPTIONS[@]}" --xattrs )
680_install_PPC_bootlist.sh
boot_list=( ${boot_list[@]:0:5} )
680_install_PPC_bootlist.sh
boot_list=( ${boot_list[@]:0:5} )
830_copy_kernel_initrd.sh
#RESULT_FILES=( "${USB_FILES[@]}" "$USB_DIR/kernel" "$USB_DIR/$REAR_INITRD_FILENAME" )
950_dsmc_save_result_files.sh
TSM_RESULT_FILES=( "${RESULT_FILES[@]}" )
950_nsr_save_result_files.sh
NSR_RESULT_FILES=( "${RESULT_FILES[@]}" )
dhcp-setup-functions.sh
((pads=4-${#target_arr[@]}))
layout-functions.sh
while (( ${#devlist[@]} )) ; do
devlist=( "${devlist[@]}" )
while (( ${#devlist[@]} )) ; do
devlist=( "${devlist[@]}" )
linux-functions.sh
if (( ${#STORAGE_DRIVERS[@]} == 0 )) ; then
opaladmin-workflow.sh
(( ${#OPALADMIN_DEVICES[@]} == 0 )) && Error "Could not detect TCG Opal 2-compliant disks."
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
rd.sh
dhclients=(${dhclients[@]} "$x")
rear-release-notes.txt
in etc/rear/local.conf MODULES=( "${MODULES[@]}" 'moduleX' 'moduleY' ) to
now MODULES=( "${MODULES[@]}" 'moduleX' 'moduleY' ) includes all kernel
uefi-functions.sh
local modules=( ${GRUB2_MODULES_LOAD:+"${GRUB2_MODULES_LOAD[@]}"} )
if (( ${#dirs[@]} )) && ! (( ${#modules[@]} )) ; then
if (( ${#dirs[@]} )) && ! (( ${#modules[@]} )) ; then
(( ${#GRUB2_MODULES[@]} )) && LogPrint "Installing only ${GRUB2_MODULES[*]} modules into $outfile memdisk"
(( ${#modules[@]} )) && LogPrint "GRUB2 modules to load: ${modules[*]}"
jsmeix commented at 2020-04-17 15:48:¶
I did not check documentation files.
I only checked find usr/sbin/rear usr/share/rear/ -type f
OliverO2 commented at 2020-04-17 15:50:¶
Obviously, the above list contains false positives (e.g.
Identifier1=("Identifier2[@]" ...)). And others.
This one seems more specific:
Targets
Occurrences of '\w\s*=\s*\(\s*\S+\[@\]' in project
Found Occurrences
04-scenarios.adoc
COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" )
06-layout-configuration.adoc
EXCLUDE_RECREATE=( "${EXCLUDE_RECREATE[@]}" "fs:/media/backup" )
EXCLUDE_RECREATE=( "${EXCLUDE_RECREATE[@]}" "/dev/backup" )
12-BLOCKCLONE.adoc
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )
`REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone mount.ntfs-3g )`
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
BACKUP_PROG_EXCLUDE=( "${BACKUP_PROG_EXCLUDE[@]}" '/products/*' )
15-EFISTUB.adoc
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} /mnt )
REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" snapper chattr lsattr xfs_repair )
COPY_AS_IS=( "${COPY_AS_IS[@]}" /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
205_inspect_tar_capabilities.sh
echo "BACKUP_PROG_OPTIONS=( ${BACKUP_PROG_OPTIONS[@]} )" >> $ROOTFS_DIR/etc/rear/rescue.conf
210_include_dhclient.sh
dhclients=(${dhclients[@]} "$x")
COPY_AS_IS=( "${COPY_AS_IS[@]}" "/etc/localtime" "/usr/lib/dhcpcd/*" )
PROGS=( "${PROGS[@]}" arping ipcalc usleep "${dhclients[@]}" )
240_kernel_modules.sh
MODULES=(
COPY_AS_IS=(
300_map_disks.sh
regular_choices=( "${possible_targets[@]}" "$skip_choice" )
300_map_disks.sh
regular_choices=( "${possible_targets[@]}" "$skip_choice" )
390_copy_binaries_libraries.sh
local all_libs=( "${LIBS[@]}" $( RequiredSharedObjects "${all_binaries[@]}" "${LIBS[@]}" ) )
400_prep_nbkdc.sh
COPY_AS_IS=(
COPY_AS_IS_EXCLUDE=(
400_prep_sesam.sh
COPY_AS_IS=(
COPY_AS_IS_EXCLUDE=(
LIBS=(
500_make_backup.sh
pipes_rc=( ${PIPESTATUS[@]} )
pipes_rc=( ${PIPESTATUS[@]} )
500_ssh.sh
copy_as_is_ssh_files=( "${SSH_FILES[@]}" )
55-migrate-network-devices.sh
NEW_DEVICES=( "${NEW_DEVICES[@]/$choice/}" )
550_rebuild_initramfs.sh
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
550_rebuild_initramfs.sh
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
550_rebuild_initramfs.sh
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
600_check_rsync_xattr.sh
RSYNC_OPTIONS=( "${BACKUP_RSYNC_OPTIONS[@]}" --xattrs )
680_install_PPC_bootlist.sh
boot_list=( ${boot_list[@]:0:5} )
680_install_PPC_bootlist.sh
boot_list=( ${boot_list[@]:0:5} )
830_copy_kernel_initrd.sh
#RESULT_FILES=( "${USB_FILES[@]}" "$USB_DIR/kernel" "$USB_DIR/$REAR_INITRD_FILENAME" )
950_dsmc_save_result_files.sh
TSM_RESULT_FILES=( "${RESULT_FILES[@]}" )
950_nsr_save_result_files.sh
NSR_RESULT_FILES=( "${RESULT_FILES[@]}" )
layout-functions.sh
devlist=( "${devlist[@]}" )
devlist=( "${devlist[@]}" )
opaladmin-workflow.sh
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
local devices=( "${@:-${OPALADMIN_DEVICES[@]}}" )
rd.sh
dhclients=(${dhclients[@]} "$x")
rear-release-notes.txt
in etc/rear/local.conf MODULES=( "${MODULES[@]}" 'moduleX' 'moduleY' ) to
now MODULES=( "${MODULES[@]}" 'moduleX' 'moduleY' ) includes all kernel
uefi-functions.sh
local modules=( ${GRUB2_MODULES_LOAD:+"${GRUB2_MODULES_LOAD[@]}"} )
jsmeix commented at 2020-04-17 15:53:¶
Your results found things as in
https://github.com/rear/rear/blob/master/usr/share/rear/prep/SESAM/default/400_prep_sesam.sh
Ahh - with inserted empty lines that separate the files
I can read it so much easier!
Next week I will have a look at those remaining issues.
@OliverO2
have a nice and relaxed weekend
and many thanks for all your help here!
OliverO2 commented at 2020-04-17 15:54:¶
Thanks, have an excellent weekend, too. Be safe, stay healthy!
OliverO2 commented at 2020-04-17 21:49:¶
Ending the day on a high note:
- There is an export option for IntelliJ IDEA search results in a readable form (context menu rules)!
- I have found an even more specific regexp.
- I have hand-edited out false positives.
So I guess this is all that's left (documentation included, would seem nicer if that would be fixed, too, for consistency):
Targets
Occurrences of '(\w+)\s*=\s*\(\s*\S+[{$]\1\[@\]' in project
Found Occurrences
06-layout-configuration.adoc (2 usages found)
152 EXCLUDE_RECREATE=( "${EXCLUDE_RECREATE[@]}" "fs:/media/backup" )
193 EXCLUDE_RECREATE=( "${EXCLUDE_RECREATE[@]}" "/dev/backup" )
12-BLOCKCLONE.adoc (8 usages found)
149 REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
159 BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )
276 `REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone mount.ntfs-3g )`
295 REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
315 BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/media/*' )
403 REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
424 REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" ntfsclone )
677 BACKUP_PROG_EXCLUDE=( "${BACKUP_PROG_EXCLUDE[@]}" '/products/*' )
15-EFISTUB.adoc (3 usages found)
138 BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} /mnt )
141 REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" snapper chattr lsattr xfs_repair )
142 COPY_AS_IS=( "${COPY_AS_IS[@]}" /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
240_kernel_modules.sh (2 usages found)
20 MODULES=(
30 COPY_AS_IS=(
400_prep_nbkdc.sh (2 usages found)
63 COPY_AS_IS=(
75 COPY_AS_IS_EXCLUDE=(
400_prep_sesam.sh (3 usages found)
14 COPY_AS_IS=(
27 COPY_AS_IS_EXCLUDE=(
39 LIBS=(
jsmeix commented at 2020-04-20 07:47:¶
@OliverO2
thank you for your precise and well readable list in your
https://github.com/rear/rear/issues/2364#issuecomment-615479355
I have now fixed those items too.
jsmeix commented at 2020-04-20 13:22:¶
Regarding
https://github.com/rear/rear/issues/2364#issuecomment-615319253
a test that finds false ARRAY=( additional elements ) assignments
I checked for all array variables that are set in default.conf
if they are also set elsewhere which could be false because
usually array variables that are set in default.conf should be
only extended by additional elements:
# arrvars="$( grep '^[[:alpha:]_][[:alpha:]_]*=(' usr/share/rear/conf/default.conf | cut -d '=' -f 1 | sort -u )" ; \
for f in $( find usr/share/rear -type f | egrep -v 'usr/share/rear/conf/default.conf|usr/share/rear/prep/GNU/Linux/210_include_dhclient.sh' ) ; \
do found="" ; \
for arrvar in $arrvars ; \
do if grep -q "[^_]*$arrvar=" $f ; \
then found=yes ; break ; fi ; \
done ; \
test $found || continue ; \
echo $f ; \
for arrvar in $arrvars ; \
do grep "[^_]*$arrvar=" $f ; \
done ; \
echo ; \
done
usr/share/rear/output/OBDR/default/810_write_image.sh
BACKUP_PROG_COMPRESS_OPTIONS=
usr/share/rear/output/USB/Linux-i386/830_copy_kernel_initrd.sh
#RESULT_FILES=( "${USB_FILES[@]}" "$USB_DIR/kernel" "$USB_DIR/$REAR_INITRD_FILENAME" )
usr/share/rear/output/TSM/default/950_dsmc_save_result_files.sh
TSM_RESULT_FILES=()
TSM_RESULT_FILES=(
TSM_RESULT_FILES=( "${RESULT_FILES[@]}" )
usr/share/rear/output/DP/default/950_dp_save_result_files.sh
#DP_RESULT_FILES=()
usr/share/rear/output/NSR/default/950_nsr_save_result_files.sh
NSR_RESULT_FILES=()
NSR_RESULT_FILES=(
NSR_RESULT_FILES=( "${RESULT_FILES[@]}" )
usr/share/rear/build/GNU/Linux/420_copy_firmware_files.sh
# The special value FIRMWARE_FILES=( 'no' ) or any value that
LogPrint "Omit copying files in /lib*/firmware/ (FIRMWARE_FILES='$FIRMWARE_FILES')"
test "${FIRMWARE_FILES[*]}" || FIRMWARE_FILES=( 'yes' )
# The special value FIRMWARE_FILES=( 'yes' ) or any value that
usr/share/rear/build/GNU/Linux/400_copy_modules.sh
# The special user setting MODULES=( 'no_modules' ) enforces that
# Since ReaR version 2.5 we have MODULES=( 'all_modules' ) in default.conf
# MODULES array contents like MODULES=( 'moduleX' 'all_modules' 'moduleY' ):
# The setting MODULES=( 'loaded_modules' ) results that only those kernel modules
# MODULES array contents like MODULES=( 'moduleX' 'loaded_modules' 'moduleY' ):
# MODULES=() which means the currently loaded kernel modules get included in the recovery system
# MODULES=( 'moduleX' 'moduleY' ) where additional kernel modules can be specified
# to be included in the recovery system in addition to the ones via an empty MODULES=() setting:
# to not lose the below added modules but with MODULES=( 'all_modules' ) in default.conf
# MODULES=( 'moduleX' 'moduleY' ) in etc/rear/local.conf to get additional kernel modules
# included in the recovery system in addition to the ones via an empty MODULES=() setting:
# Fallback is the plain module file without other needed modules (cf. the MODULES=( 'loaded_modules' ) case
usr/share/rear/build/OPALPBA/Linux-i386/095_exclude_non_essential_files.sh
Log "Before exclusion: PROGS=(${PROGS[*]})"
PROGS=( $(printf '%s\n' "${PROGS[@]}" | sed -r '/^('"$(printf '%s|' "${progs_to_exclude[@]}")"')$/d') )
Log "After exclusion: PROGS=(${PROGS[*]})"
Log "Before exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
REQUIRED_PROGS=( $(printf '%s\n' "${REQUIRED_PROGS[@]}" | sed -r '/^('"$(printf '%s|' "${progs_to_exclude[@]}")"')$/d')
Log "After exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
Log "Before exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
REQUIRED_PROGS=( $(printf '%s\n' "${REQUIRED_PROGS[@]}" | sed -r '/^('"$(printf '%s|' "${progs_to_exclude[@]}")"')$/d')
Log "After exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
usr/share/rear/conf/Linux-ppc64.conf
test "${FIRMWARE_FILES[*]}" || FIRMWARE_FILES=( 'no' )
usr/share/rear/conf/Linux-ppc64le.conf
test "${FIRMWARE_FILES[*]}" || FIRMWARE_FILES=( 'no' )
usr/share/rear/conf/examples/borg-example.conf
COPY_AS_IS_BORG=( $BORG_KEYS_DIR )
ONLY_INCLUDE_VG=( "vg00" )
usr/share/rear/conf/examples/SLE12-SP1-btrfs-example.conf
BACKUP_PROG_INCLUDE=( /var/tmp /srv /var/lib/pgsql /var/spool /var/lib/libvirt/images /var/opt/ /tmp /var/lib/named /var
#MODULES_LOAD=( )
usr/share/rear/conf/examples/RHEL7-ISO-Oracle-example.conf
#ONLY_INCLUDE_VG=( "vg00" )
usr/share/rear/conf/examples/SLE12-SP2-btrfs-example.conf
BACKUP_PROG_INCLUDE=( /var/cache /var/lib/mailman /var/tmp /var/lib/pgsql /usr/local /opt /var/lib/libvirt/images /boot/
#MODULES_LOAD=( )
usr/share/rear/conf/examples/SLE12-btrfs-example.conf
BACKUP_PROG_INCLUDE=( /home /var/tmp /var/spool /var/opt /var/log /var/lib/pgsql /var/lib/mailman /var/lib/named /usr/lo
usr/share/rear/conf/examples/SLE11-SLE12-SAP-HANA-UEFI-example.conf
EXCLUDE_VG=( vgHANA-data-HC2 vgHANA-data-HC3 vgHANA-log-HC2 vgHANA-log-HC3 vgHANA-shared-HC2 vgHANA-shared-HC3 )
usr/share/rear/layout/prepare/default/420_autoresize_last_partitions.sh
# AUTORESIZE_EXCLUDE_PARTITIONS=( boot swap efi /dev/sdb3 /dev/sdc4 )
test "$AUTORESIZE_EXCLUDE_PARTITIONS" || AUTORESIZE_EXCLUDE_PARTITIONS=( boot swap efi )
usr/share/rear/rescue/NETFS/default/610_save_capabilities.sh
is_true "$NETFS_RESTORE_CAPABILITIES" && NETFS_RESTORE_CAPABILITIES=( '/' )
usr/share/rear/rescue/GNU/Linux/240_kernel_modules.sh
# The special user setting MODULES=( 'no_modules' ) enforces that
usr/share/rear/rescue/GNU/Linux/220_load_modules_from_initrd.sh
INITRD_MODULES=
usr/share/rear/restore/RSYNC/default/200_remove_relative_rsync_option.sh
BACKUP_RSYNC_OPTIONS=( $( RmInArray "--relative" "${BACKUP_RSYNC_OPTIONS[@]}" ) )
BACKUP_RSYNC_OPTIONS=( $( RmInArray "-R" "${BACKUP_RSYNC_OPTIONS[@]}" ) )
usr/share/rear/verify/OBDR/NETFS/default/540_set_backup_compression.sh
BACKUP_PROG_COMPRESS_OPTIONS=
usr/share/rear/lib/array-functions.sh
# usage: BACKUP_RSYNC_OPTIONS=( $( RmInArray "--relative" "${BACKUP_RSYNC_OPTIONS[@]}" ) )
usr/share/rear/lib/bootloader-functions.sh
FEATURE_SYSLINUX_MODULES=
FEATURE_SYSLINUX_MODULES="y"
usr/share/rear/lib/linux-functions.sh
# The special user setting MODULES=( 'no_modules' ) enforces that
usr/share/rear/finalize/Fedora/i386/550_rebuild_initramfs.sh
OLD_INITRD_MODULES=( $(cat $VAR_DIR/recovery/initrd_modules) )
OLD_INITRD_MODULES=()
Log "Original OLD_INITRD_MODULES='${OLD_INITRD_MODULES[@]}'"
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
NEW_INITRD_MODULES=( $(tr " " "\n" <<< "${NEW_INITRD_MODULES[*]}" | sort | uniq -u) )
Log "New INITRD_MODULES='${OLD_INITRD_MODULES[@]} ${NEW_INITRD_MODULES[@]}'"
INITRD_MODULES="${OLD_INITRD_MODULES[@]} ${NEW_INITRD_MODULES[@]}"
WITH_INITRD_MODULES=$( printf '%s\n' ${INITRD_MODULES[@]} | awk '{printf "--with=%s ", $1}' )
usr/share/rear/finalize/SUSE_LINUX/i386/550_rebuild_initramfs.sh
# INITRD_MODULES=""
# # This replaces old INITRD_MODULES= variable from /etc/sysconfig/kernel
Log "Original INITRD_MODULES='$INITRD_MODULES'"
OLD_INITRD_MODULES=( $INITRD_MODULES )
NEW_INITRD_MODULES=( $INITRD_MODULES $INITRD_MODULES $( cat $TMP_DIR/storage_drivers ) )
NEW_INITRD_MODULES=( $( tr " " "\n" <<< "${NEW_INITRD_MODULES[*]}" | sort | uniq -u ) )
Log "New INITRD_MODULES='${OLD_INITRD_MODULES[@]} ${NEW_INITRD_MODULES[@]}'"
sed -i -e '/^INITRD_MODULES/s/^.*$/#&\nINITRD_MODULES="'"${OLD_INITRD_MODULES[*]} ${NEW_INITRD_MODULES[*]}"'"/' $TAR
usr/share/rear/finalize/Debian/i386/550_rebuild_initramfs.sh
INITRD_MODULES="$( sort -t " " -k 1 -u $TMP_DIR/storage_drivers $TARGET_FS_ROOT/etc/initramfs-tools/module[s] )"
usr/share/rear/prep/NETFS/default/040_inspect_configuration_files.sh
# If the user defines in local.conf for example: BACKUP_PROG_OPTIONS="--option1 --option2" then you will see
usr/share/rear/prep/NETFS/GNU/Linux/205_inspect_tar_capabilities.sh
echo "BACKUP_PROG_OPTIONS=( ${BACKUP_PROG_OPTIONS[@]} )" >> $ROOTFS_DIR/etc/rear/rescue.conf
# prep/GNU/Linux/310_include_cap_utils.sh uses NETFS_RESTORE_CAPABILITIES=( 'Yes' ) to kick in next line, and is
usr/share/rear/prep/DUPLICITY/default/250_find_all_libs.sh
sorted_unique_COPY_AS_IS=$(echo "${COPY_AS_IS[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')
eval COPY_AS_IS=${sorted_unique_COPY_AS_IS}
sorted_unique_LIBS=$(echo "${LIBS[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')
eval LIBS=${sorted_unique_LIBS[@]}
usr/share/rear/prep/GNU/Linux/310_include_cap_utils.sh
# that would store a false NETFS_RESTORE_CAPABILITIES='()' into /etc/rear/rescue.conf
# when NETFS_RESTORE_CAPABILITIES=() but '()' is not an empty array but the string "()"
NETFS_RESTORE_CAPABILITIES=( 'No' )
usr/share/rear/prep/GNU/Linux/280_include_virtualbox.sh
#VBOX_COPY_AS_IS=( /etc/init.d/vboxadd* /opt/VBoxGuestAdditions-* /usr/sbin/VBoxService )
VBOX_COPY_AS_IS=( /etc/init.d/vboxadd* /usr/sbin/VBoxService )
VBOX_MODULES=( vboxsf vboxvideo vboxguest )
usr/share/rear/prep/OPALPBA/Linux-i386/001_configure_workflow.sh
FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )
USE_RESOLV_CONF='no'
I will check those cases one by one...
jsmeix commented at 2020-04-20 13:50:¶
For the (actually not so much) fun of it:
ARRAY+=( additional elements ) is very much faster
than ARRAY=( "${ARRAY[@]}" additional elements )
in particular for bigger arrays:
# unset arr ; time for i in $( seq 10 99 ) ; do arr+=( element$i ) ; done
real 0m0.004s
user 0m0.001s
sys 0m0.001s
# unset arr ; time for i in $( seq 100 999 ) ; do arr+=( element$i ) ; done
real 0m0.008s
user 0m0.008s
sys 0m0.000s
# unset arr ; time for i in $( seq 1000 9999 ) ; do arr+=( element$i ) ; done
real 0m0.046s
user 0m0.041s
sys 0m0.005s
versus lame old code
# unset arr ; time for i in $( seq 10 99 ) ; do arr=( "${arr[@]}" element$i ) ; done
real 0m0.016s
user 0m0.013s
sys 0m0.003s
# unset arr ; time for i in $( seq 100 999 ) ; do arr=( "${arr[@]}" element$i ) ; done
real 0m1.160s
user 0m1.157s
sys 0m0.003s
# unset arr ; time for i in $( seq 1000 9999 ) ; do arr=( "${arr[@]}" element$i ) ; done
real 2m24.567s
user 2m23.277s
sys 0m0.847s
I used a not so fast Intel Core i3-4000M CPU @ 2.40GHz for that tests.
Adding a single element to large arrays in both ways:
# unset arr ; for i in $( seq 100 999 ) ; do arr+=( element$i ) ; done ; \
time arr+=( final_element1 ) ; \
time arr=( "${arr[@]}" final_element2 )
real 0m0.000s
user 0m0.000s
sys 0m0.000s
real 0m0.003s
user 0m0.003s
sys 0m0.000s
# unset arr ; for i in $( seq 1000 9999 ) ; do arr+=( element$i ) ; done ; \
time arr+=( final_element1 ) ; \
time arr=( "${arr[@]}" final_element2 )
real 0m0.000s
user 0m0.000s
sys 0m0.000s
real 0m0.026s
user 0m0.026s
sys 0m0.000s
# unset arr ; for i in $( seq 10000 99999 ) ; do arr+=( element$i ) ; done ; \
time arr+=( final_element1 ) ; \
time arr=( "${arr[@]}" final_element2 )
real 0m0.000s
user 0m0.000s
sys 0m0.000s
real 0m0.284s
user 0m0.280s
sys 0m0.004s
jsmeix commented at 2020-04-20 13:58:¶
Same for strings but less serious in practice:
Better code string+=" additional words"
# unset string ; time for i in $( seq 10 99 ) ; do string+=" word$i" ; done
real 0m0.003s
user 0m0.002s
sys 0m0.001s
# unset string ; time for i in $( seq 100 999 ) ; do string+=" word$i" ; done
real 0m0.009s
user 0m0.009s
sys 0m0.000s
# unset string ; time for i in $( seq 1000 9999 ) ; do string+=" word$i" ; done
real 0m0.361s
user 0m0.358s
sys 0m0.003s
versus lame old code string="$string additional words"
# unset string ; time for i in $( seq 10 99 ) ; do string="$string word$i" ; done
real 0m0.004s
user 0m0.004s
sys 0m0.000s
# unset string ; time for i in $( seq 100 999 ) ; do string="$string word$i" ; done
real 0m0.054s
user 0m0.051s
sys 0m0.002s
# unset string ; time for i in $( seq 1000 9999 ) ; do string="$string word$i" ; done
real 0m5.438s
user 0m5.431s
sys 0m0.007s
FYI
how long empty for loops and plain seq take:
# time for i in $( seq 10 99 ) ; do : ; done
real 0m0.004s
user 0m0.001s
sys 0m0.003s
# time for i in $( seq 100 999 ) ; do : ; done
real 0m0.005s
user 0m0.003s
sys 0m0.002s
# time for i in $( seq 1000 9999 ) ; do : ; done
real 0m0.020s
user 0m0.016s
sys 0m0.004s
# time seq 10 99 >/dev/null
real 0m0.004s
user 0m0.003s
sys 0m0.000s
# time seq 100 999 >/dev/null
real 0m0.004s
user 0m0.003s
sys 0m0.001s
# time seq 1000 9999 >/dev/null
real 0m0.004s
user 0m0.004s
sys 0m0.000s
jsmeix commented at 2020-04-20 14:55:¶
Regarding
https://github.com/rear/rear/issues/2364#issuecomment-616549422
usr/share/rear/output/OBDR/default/810_write_image.sh
BACKUP_PROG_COMPRESS_OPTIONS=
is OK because there is a comment that explains it:
# Disable compression (as tape drive does compression already)
Log "Disable compression for backup (BACKUP_PROG_COMPRESS_*)"
BACKUP_PROG_COMPRESS_OPTIONS=
BACKUP_PROG_COMPRESS_SUFFIX=
usr/share/rear/output/USB/Linux-i386/830_copy_kernel_initrd.sh
#RESULT_FILES=( "${USB_FILES[@]}" "$USB_DIR/kernel" "$USB_DIR/$REAR_INITRD_FILENAME" )
is only a comment so it is ok
usr/share/rear/output/TSM/default/950_dsmc_save_result_files.sh
TSM_RESULT_FILES=()
TSM_RESULT_FILES=(
TSM_RESULT_FILES=( "${RESULT_FILES[@]}" )
is a false positive: there is no TSM_RESULT_FILES in default.conf
usr/share/rear/output/DP/default/950_dp_save_result_files.sh
#DP_RESULT_FILES=()
is only a comment so it is ok
usr/share/rear/output/NSR/default/950_nsr_save_result_files.sh
NSR_RESULT_FILES=()
NSR_RESULT_FILES=(
NSR_RESULT_FILES=( "${RESULT_FILES[@]}" )
is a false positive: there is no NSR_RESULT_FILES in default.conf
usr/share/rear/build/GNU/Linux/420_copy_firmware_files.sh
# The special value FIRMWARE_FILES=( 'no' ) or any value that
LogPrint "Omit copying files in /lib*/firmware/ (FIRMWARE_FILES='$FIRMWARE_FILES')"
test "${FIRMWARE_FILES[*]}" || FIRMWARE_FILES=( 'yes' )
# The special value FIRMWARE_FILES=( 'yes' ) or any value that
is only a comment so it is ok
usr/share/rear/build/GNU/Linux/400_copy_modules.sh
# The special user setting MODULES=( 'no_modules' ) enforces that
# Since ReaR version 2.5 we have MODULES=( 'all_modules' ) in default.conf
# MODULES array contents like MODULES=( 'moduleX' 'all_modules' 'moduleY' ):
# The setting MODULES=( 'loaded_modules' ) results that only those kernel modules
# MODULES array contents like MODULES=( 'moduleX' 'loaded_modules' 'moduleY' ):
# MODULES=() which means the currently loaded kernel modules get included in the recovery system
# MODULES=( 'moduleX' 'moduleY' ) where additional kernel modules can be specified
# to be included in the recovery system in addition to the ones via an empty MODULES=() setting:
# to not lose the below added modules but with MODULES=( 'all_modules' ) in default.conf
# MODULES=( 'moduleX' 'moduleY' ) in etc/rear/local.conf to get additional kernel modules
# included in the recovery system in addition to the ones via an empty MODULES=() setting:
# Fallback is the plain module file without other needed modules (cf. the MODULES=( 'loaded_modules' ) case
is only a comment so it is ok
usr/share/rear/build/OPALPBA/Linux-i386/095_exclude_non_essential_files.sh
Log "Before exclusion: PROGS=(${PROGS[*]})"
PROGS=( $(printf '%s\n' "${PROGS[@]}" | sed -r '/^('"$(printf '%s|' "${progs_to_exclude[@]}")"')$/d') )
Log "After exclusion: PROGS=(${PROGS[*]})"
Log "Before exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
REQUIRED_PROGS=( $(printf '%s\n' "${REQUIRED_PROGS[@]}" | sed -r '/^('"$(printf '%s|' "${progs_to_exclude[@]}")"')$/d')
Log "After exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
Log "Before exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
REQUIRED_PROGS=( $(printf '%s\n' "${REQUIRED_PROGS[@]}" | sed -r '/^('"$(printf '%s|' "${progs_to_exclude[@]}")"')$/d')
Log "After exclusion: REQUIRED_PROGS=(${REQUIRED_PROGS[*]})"
looks ok
usr/share/rear/conf/Linux-ppc64.conf
test "${FIRMWARE_FILES[*]}" || FIRMWARE_FILES=( 'no' )
is ok because it sets FIRMWARE_FILES only if empty
usr/share/rear/conf/Linux-ppc64le.conf
test "${FIRMWARE_FILES[*]}" || FIRMWARE_FILES=( 'no' )
is ok because it sets FIRMWARE_FILES only if empty
usr/share/rear/conf/examples/borg-example.conf
COPY_AS_IS_BORG=( $BORG_KEYS_DIR )
ONLY_INCLUDE_VG=( "vg00" )
is ok because both COPY_AS_IS_BORG and ONLY_INCLUDE_VG
are set empty in default.conf
usr/share/rear/conf/examples/SLE12-SP1-btrfs-example.conf
BACKUP_PROG_INCLUDE=( /var/tmp /srv /var/lib/pgsql /var/spool /var/lib/libvirt/images /var/opt/ /tmp /var/lib/named /var
#MODULES_LOAD=( )
is ok because BACKUP_PROG_INCLUDE is set empty in default.conf
and MODULES_LOAD is only a comment
usr/share/rear/conf/examples/RHEL7-ISO-Oracle-example.conf
#ONLY_INCLUDE_VG=( "vg00" )
is only a comment so it is ok
usr/share/rear/conf/examples/SLE12-SP2-btrfs-example.conf
BACKUP_PROG_INCLUDE=( /var/cache /var/lib/mailman /var/tmp /var/lib/pgsql /usr/local /opt /var/lib/libvirt/images /boot/
#MODULES_LOAD=( )
is ok because BACKUP_PROG_INCLUDE is set empty in default.conf
and MODULES_LOAD is only a comment
usr/share/rear/conf/examples/SLE12-btrfs-example.conf
BACKUP_PROG_INCLUDE=( /home /var/tmp /var/spool /var/opt /var/log /var/lib/pgsql /var/lib/mailman /var/lib/named /usr/lo
is ok because BACKUP_PROG_INCLUDE is set empty in default.conf
usr/share/rear/conf/examples/SLE11-SLE12-SAP-HANA-UEFI-example.conf
EXCLUDE_VG=( vgHANA-data-HC2 vgHANA-data-HC3 vgHANA-log-HC2 vgHANA-log-HC3 vgHANA-shared-HC2 vgHANA-shared-HC3 )
is ok because EXCLUDE_VG is set empty in default.conf
usr/share/rear/layout/prepare/default/420_autoresize_last_partitions.sh
# AUTORESIZE_EXCLUDE_PARTITIONS=( boot swap efi /dev/sdb3 /dev/sdc4 )
test "$AUTORESIZE_EXCLUDE_PARTITIONS" || AUTORESIZE_EXCLUDE_PARTITIONS=( boot swap efi )
is ok because it sets AUTORESIZE_EXCLUDE_PARTITIONS only if empty
usr/share/rear/rescue/NETFS/default/610_save_capabilities.sh
is_true "$NETFS_RESTORE_CAPABILITIES" && NETFS_RESTORE_CAPABILITIES=( '/' )
is ok because that behaviour is explained in default.conf:
# The special value NETFS_RESTORE_CAPABILITIES=( 'Yes' ) results that 'getcap -r /'
# is used as fallback which could become unusable slow on systems with zillions of files
# in particular when tons of stuff is mounted cf. https://github.com/rear/rear/issues/1283
# In this case one must explicitly specify on what directories 'getcap -r' should be run.
usr/share/rear/rescue/GNU/Linux/240_kernel_modules.sh
# The special user setting MODULES=( 'no_modules' ) enforces that
is only a comment so it is ok
I will contiune that tomorrow...
jsmeix commented at 2020-04-21 15:19:¶
...continuing:
usr/share/rear/rescue/GNU/Linux/220_load_modules_from_initrd.sh
INITRD_MODULES=
is ok because the code is
MODULES_LOAD+=(
$(
INITRD_MODULES=
source /etc/sysconfig/kernel
echo $INITRD_MODULES
)
)
usr/share/rear/restore/RSYNC/default/200_remove_relative_rsync_option.sh
BACKUP_RSYNC_OPTIONS=( $( RmInArray "--relative" "${BACKUP_RSYNC_OPTIONS[@]}" ) )
BACKUP_RSYNC_OPTIONS=( $( RmInArray "-R" "${BACKUP_RSYNC_OPTIONS[@]}" ) )
is ok and the reason is described in a commet in the code
usr/share/rear/verify/OBDR/NETFS/default/540_set_backup_compression.sh
BACKUP_PROG_COMPRESS_OPTIONS=
is ok because the reason is described in a commet in the code
usr/share/rear/lib/array-functions.sh
# usage: BACKUP_RSYNC_OPTIONS=( $( RmInArray "--relative" "${BACKUP_RSYNC_OPTIONS[@]}" ) )
is only a comment so it is ok
usr/share/rear/lib/bootloader-functions.sh
FEATURE_SYSLINUX_MODULES=
FEATURE_SYSLINUX_MODULES="y"
is a false positive: there is no FEATURE_SYSLINUX_MODULES in default.conf
usr/share/rear/lib/linux-functions.sh
# The special user setting MODULES=( 'no_modules' ) enforces that
is only a comment so it is ok
usr/share/rear/finalize/Fedora/i386/550_rebuild_initramfs.sh
OLD_INITRD_MODULES=( $(cat $VAR_DIR/recovery/initrd_modules) )
OLD_INITRD_MODULES=()
Log "Original OLD_INITRD_MODULES='${OLD_INITRD_MODULES[@]}'"
NEW_INITRD_MODULES=( ${OLD_INITRD_MODULES[@]} ${OLD_INITRD_MODULES[@]} $( cat $TMP_DIR/storage_drivers ) )
NEW_INITRD_MODULES=( $(tr " " "\n" <<< "${NEW_INITRD_MODULES[*]}" | sort | uniq -u) )
Log "New INITRD_MODULES='${OLD_INITRD_MODULES[@]} ${NEW_INITRD_MODULES[@]}'"
INITRD_MODULES="${OLD_INITRD_MODULES[@]} ${NEW_INITRD_MODULES[@]}"
WITH_INITRD_MODULES=$( printf '%s\n' ${INITRD_MODULES[@]} | awk '{printf "--with=%s ", $1}' )
usr/share/rear/finalize/SUSE_LINUX/i386/550_rebuild_initramfs.sh
# INITRD_MODULES=""
# # This replaces old INITRD_MODULES= variable from /etc/sysconfig/kernel
Log "Original INITRD_MODULES='$INITRD_MODULES'"
OLD_INITRD_MODULES=( $INITRD_MODULES )
NEW_INITRD_MODULES=( $INITRD_MODULES $INITRD_MODULES $( cat $TMP_DIR/storage_drivers ) )
NEW_INITRD_MODULES=( $( tr " " "\n" <<< "${NEW_INITRD_MODULES[*]}" | sort | uniq -u ) )
Log "New INITRD_MODULES='${OLD_INITRD_MODULES[@]} ${NEW_INITRD_MODULES[@]}'"
sed -i -e '/^INITRD_MODULES/s/^.*$/#&\nINITRD_MODULES="'"${OLD_INITRD_MODULES[*]} ${NEW_INITRD_MODULES[*]}"'"/' $TAR
usr/share/rear/finalize/Debian/i386/550_rebuild_initramfs.sh
INITRD_MODULES="$( sort -t " " -k 1 -u $TMP_DIR/storage_drivers $TARGET_FS_ROOT/etc/initramfs-tools/module[s] )"
are false positives: there is neither OLD_INITRD_MODULES
nor OLD_INITRD_MODULES nor INITRD_MODULES in default.conf
usr/share/rear/prep/NETFS/default/040_inspect_configuration_files.sh
# If the user defines in local.conf for example: BACKUP_PROG_OPTIONS="--option1 --option2" then you will see
is only a comment so it is ok
usr/share/rear/prep/NETFS/GNU/Linux/205_inspect_tar_capabilities.sh
echo "BACKUP_PROG_OPTIONS=( ${BACKUP_PROG_OPTIONS[@]} )" >> $ROOTFS_DIR/etc/rear/rescue.conf
# prep/GNU/Linux/310_include_cap_utils.sh uses NETFS_RESTORE_CAPABILITIES=( 'Yes' ) to kick in next line, and is
is ok
usr/share/rear/prep/DUPLICITY/default/250_find_all_libs.sh
sorted_unique_COPY_AS_IS=$(echo "${COPY_AS_IS[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')
eval COPY_AS_IS=${sorted_unique_COPY_AS_IS}
sorted_unique_LIBS=$(echo "${LIBS[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')
eval LIBS=${sorted_unique_LIBS[@]}
looks ok
but that also shows an interesting question whether or not
we make the COPY_AS_IS PROGS REQUIRED_PROGS
and LIBS arrays list each element only once
(i.e. convert those arrays to sets) before we process them
to avoid duplicate work.
usr/share/rear/prep/GNU/Linux/310_include_cap_utils.sh
# that would store a false NETFS_RESTORE_CAPABILITIES='()' into /etc/rear/rescue.conf
# when NETFS_RESTORE_CAPABILITIES=() but '()' is not an empty array but the string "()"
NETFS_RESTORE_CAPABILITIES=( 'No' )
is ok because the reason is described in a commet in the code
usr/share/rear/prep/GNU/Linux/280_include_virtualbox.sh
#VBOX_COPY_AS_IS=( /etc/init.d/vboxadd* /opt/VBoxGuestAdditions-* /usr/sbin/VBoxService )
VBOX_COPY_AS_IS=( /etc/init.d/vboxadd* /usr/sbin/VBoxService )
VBOX_MODULES=( vboxsf vboxvideo vboxguest )
is a false positive: there is neither VBOX_COPY_AS_IS
nor VBOX_MODULES in default.conf
usr/share/rear/prep/OPALPBA/Linux-i386/001_configure_workflow.sh
FIRMWARE_FILES=( 'no' )
MODULES=( 'loaded_modules' )
USE_RESOLV_CONF='no'
seems to be ok - those settings seem to be needed to
"Configure the workflow for TCG Opal pre-boot authentication (PBA) image
creation"
jsmeix commented at 2020-04-21 15:21:¶
I found no false ARRAY=( additional elements ) assignments
so from my current point of view this issue seems to be done.
jsmeix commented at 2020-04-22 13:37:¶
In contrast to what I wrote above in
https://github.com/rear/rear/issues/2364#issuecomment-617246885
the item
usr/share/rear/prep/DUPLICITY/default/250_find_all_libs.sh
sorted_unique_COPY_AS_IS=$(echo "${COPY_AS_IS[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')
eval COPY_AS_IS=${sorted_unique_COPY_AS_IS}
sorted_unique_LIBS=$(echo "${LIBS[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')
eval LIBS=${sorted_unique_LIBS[@]}
is not ok because is needlessly messes around
with the LIBS and COPY_AS_IS arrays.
I tried to fix that a bit via
https://github.com/rear/rear/commit/6d25fd17b7ed2a6409362fcc33252eff4d9ce1d5
but in general the prep/DUPLICITY/default/250_find_all_libs.sh
looks as if it should be completely overhauled.
[Export of Github issue for rear/rear.]