#469 Issue closed: Suggestion: Duply/Duplicity + TMP_DIR value not handled (and optionally ARCH_DIR)

Labels: enhancement, support / question

Reiner030 opened issue at 2014-10-02 13:10:

Hello,

mysqlf I use default /tmp folder but in duply profile this could be setup:

# temporary file space. at least the size of the biggest file in backup
# for a successful restoration process. (default is '/tmp', if not set)
#TEMP_DIR=/tmp

and needs to be handeld/linked then in
verify/DUPLICITY/default/20_check_tmpfs.sh
verify/DUPLICITY/default/25_check_duply_profile.sh
and perhaps also
restore/DUPLICITY/default/10_restore_duply.sh

Also it could/should be helpful to use for duplicity tmp folder not tmpfs if possible because there could be VMs/"appliances" with low RAM but huge HDD temp space?

2014-10-02 11:45:54 Including verify/DUPLICITY/default/20_check_tmpfs.sh
2014-10-02 11:45:54 File system /tmp not present - try to mount it via tmpfs
2014-10-02 11:45:54 Including verify/DUPLICITY/default/25_check_duply_profile.sh
2014-10-02 11:46:25 Including restore/DUPLICITY/default/10_restore_duply.sh

Also it could be helpful to recognize activated ARCH_DIR settings and also activate it for ReaR recover to "buffer" the file metadata temporary to fasten the restore with smaller net links?

# Modifies archive-dir option (since 0.6.0) Defines a folder that holds 
# unencrypted meta data of the backup, enabling new incrementals without the 
# need to decrypt backend metadata first. If empty or deleted somehow, the 
# private key and it's password are needed.
# NOTE: This is confidential data. Put it somewhere safe. It can grow quite 
#       big over time so you might want to put it not in the home dir.
# default '~/.cache/duplicity/duply_<profile>/'
# if set  '\${ARCH_DIR}/<profile>'
#ARCH_DIR=/some/space/safe/.duply-cache

Bests

gdha commented at 2014-10-02 14:18:

in rear you can use TMPDIR instead (in the local.conf file)

Reiner030 commented at 2014-10-06 18:47:

My suggestions came because common/rear TMPDIR variable is overwritten by duply:

# grep -nA2 TMPDIR /usr/bin/duply 
561:DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
562- ${DUPL_ARG_ENC}"
563-}

and defaults to /tmp:

# grep -n TEMP_DIR /usr/bin/duply | grep tmp
288:#TEMP_DIR=/tmp
972:  TEMP_DIR=${TEMP_DIR:-'/tmp'}

(and the possible size problem compared to HDD and tmpfs space)

In both cases it would be nice to auto-reassign the TEMP_DIR variable of duply ?

Just checked: duplicity itself uses only TMPDIR:
http://duplicity.nongnu.org/FAQ.html#tmp_space

Reiner030 commented at 2014-10-07 08:29:

At home I got an idea howto solve it - here the short version how it for the scripts. I do it "reverse" to the duply setting of temporary folder:

# grep TEMP /etc/duply/*/conf
#TEMP_DIR=/tmp

# grep -qE "^TEMP_DIR=\\$\{TMPDIR:-" /etc/duply/*/conf || \
  sed -ie "s/^#\?TEMP_DIR=\(.*\)$/TEMP_DIR=\${TMPDIR:-\1}/" /etc/duply/*/conf

# grep TEMP /etc/duply/*/conf
TEMP_DIR=${TMPDIR:-/tmp}

This shouldn't (must not?) be used on the original conf file because its not nice to "fix" something in background without knowlede of the administrator but could be announced as "should be set" and can be used onto the conf in the rescue image ?

gdha commented at 2014-10-08 07:34:

@Reiner030 sounds logically; will you prepare a pull request? Thank you for the in-depth investigation. Gratien

Reiner030 commented at 2014-10-13 20:04:

yes... but I have small problem here: https://github.com/Reiner030/rear/compare/duply-tmpdir-fix-volsize-check

/usr/share/rear/build/DUPLICITY/default/50_patch_duply_config.sh
is created and I think it should be included automatically but it's not used when building the ISO image

gdha commented at 2014-10-16 09:25:

@Reiner030 check if the modified scripts were added to the ISO build (after booting it). perhaps you were the standard installed rear rpm instead of the local path?

Reiner030 commented at 2014-10-16 18:25:

Since I use Debian Wheezy this can't be ;) But I would check the debian folder for necessary entries.

gdha commented at 2014-12-05 17:06:

@Reiner030 can this issue be closed?

[Export of Github issue for rear/rear.]