#1664 Issue closed: Duplicity with duply waits forever (the first time)

Labels: fixed / solved / done, minor bug, external tool

gdha opened issue at 2017-12-29 10:45:

  • rear version (/usr/sbin/rear -V): 2.3
  • OS version (cat /etc/rear/os.conf or lsb_release -a): CentOs 7
  • rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):
# cat /etc/rear/duplicity.conf 
BACKUP=DUPLICITY
DUPLICITY_PROG="duplicity"
DUPLICITY_USER=root
DUPLICITY_HOST=server
DUPLICITY_PROTO="rsync"
DUPLICITY_PATH="/root/backup"
# should result e.g. in BACKUP_DUPLICITY_URL="rsync://rear-user@192.168.99.10//backup/rear"
BACKUP_DUPLICITY_URL="${DUPLICITY_PROTO}://${DUPLICITY_USER}@${DUPLICITY_HOST}/${DUPLICITY_PATH}"
# BACKUP_DUPLICITY_NETFS_URL="cifs://backupserv/md0"
# BACKUP_DUPLICITY_NETFS_OPTIONS="user=xxx,pass=xxx,vers=2.0"
# BACKUP_DUPLICITY_NETFS_MOUNTCMD="mount.cifs"
# BACKUP_DUPLICITY_NETFS_UMOUNTCMD="umount"
# BACKUP_DUPLICITY_MAX_TIME=""
BACKUP_DUPLICITY_MAX_TIME="2M"
BACKUP_DUPLICITY_GPG_OPTIONS="{ --gpg-options "--cipher-algo AES"}"
BACKUP_DUPLICITY_GPG_ENC_KEY="$(cat /root/root.gpgkey)"
# BACKUP_DUPLICITY_GPG_ENC_PASSPHRASE="vagrant"
# BACKUP_DUPLICITY_ASK_PASSPHRASE="1"
# BACKUP_DUPLICITY_GPG_SIGN_KEY=""
BACKUP_DUPLICITY_OPTIONS="--volsize 100"
BACKUP_DUPLICITY_TEMP_RAMDISK="1"
BACKUP_DUPLICITY_EXCLUDE=( '/proc' '/sys' '/run' '/var/lib/ntp/proc' "$HOME/.cache" '/tmp' '/var/tmp' '/app' '/var/app' )

OUTPUT=ISO
OUTPUT_URL=nfs://192.168.33.1/root/.config/VirtualBox/TFTP/isos
PXE_CONFIG_URL=nfs://192.168.33.1/root/.config/VirtualBox/TFTP/pxelinux.cfg

PROGS=( "${PROGS[@]}" showmount mount.nfs umount.nfs )
MODULES=( "${MODULES[@]}" nfs )
PRE_RECOVERY_SCRIPT="systemctl start rpcbind.target"

ISO_DEFAULT="automatic"
ISO_RECOVER_MODE="unattended"

USE_STATIC_NETWORKING=y
KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"

FIRMWARE_FILES=( 'no' )

SSH_ROOT_PASSWORD="vagrant"

TIMESYNC=NTPDATE
TIMESYNC_SOURCE=0.pool.ntp.org

The /etc/rear/duplicity.conf file defined DUPLICITY as backup method, but on our client VM we already had the duply executable and a valid duply conf file, therefore, automatically duply program is used instead of duplicity executable. Do not get confused about this.

  • Are you using legacy BIOS or UEFI boot? BIOS
  • Brief description of the issue:
    /sbin/rear -v -C /etc/rear/duplicity.conf mkbackup hanged. See excerpt of the log:
2017-12-29 10:15:59.069811954 Including prep/DUPLICITY/default/200_find_duply_profile.sh
Start duply v1.11.3, time is 2017-12-29 10:15:59.
Using profile '/root/.duply/duply-client'.
....
ssh: Connected (version 2.0, client OpenSSH_7.4)
The authenticity of host 'server' can't be established.
SSH-RSA key fingerprint is 09:77:25:4d:ee:33:07:9f:8c:04:45:0b:11:13:1a:a1.
Are you sure you want to continue connecting (yes/no)? Using temporary directory /tmp/duplicity-OAvEvY-tempdir
INT intercepted...exiting.
11:11:34.426 Task 'STATUS' failed with exit code '4'.
--- Finished state FAILED 'code 4' at 11:11:34.426 - Runtime 00:55:34.475 ---

It was waiting on a yes entry, which was not visible unless after pressing Ctrl-C in the log file.

  • Work-around, if any: Modified script prep/DUPLICITY/default/200_find_duply_profile.sh
    line 42 to become echo yes | duply "$DUPLY_PROFILE" status >&2 (added the echo yes |)

If we re-run then it is works from that point on.

gdha commented at 2018-01-01 08:22:

Noticed during recovery that duply cannot start the recovery due to:

2018-01-01 07:29:47.208252397 Including verify/DUPLICITY/default/250_check_duply_profile.sh
Start duply v1.11.3, time is 2018-01-01 07:29:47.

Sorry. A fatal ERROR occured:

'/etc/duply/duply-client/conf' not found.

According URL http://duply.net/wiki/index.php/Duply-documentation we should

Superuser root can place profiles under '/etc/duply'. Simply create
the folder manually before running duply as superuser.
Note:
Already existing profiles in root's home folder will cease to work
unless they are moved to the new location manually.

However, if you run duply TEST create it will create the conf file under /root/.duply/TEST/ directory and not in /etc/duply/TEST. Strange, and not according the documentation.

gdha commented at 2018-01-03 12:44:

The work-around has been implemented and works fine. Will close this issue for now.

[Export of Github issue for rear/rear.]