#2824 Issue `closed`: LFTP asks for password that is not needed (SSH and SFTP connect without password prompt) OUTPUT_URL=sftp...¶

Labels: enhancement, documentation, external tool, no-issue-activity

StereoRocker opened issue at 2022-06-19 15:52:¶

Relax-and-Recover (ReaR) Issue Template¶

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

ReaR version ("/usr/sbin/rear -V"):

root@ubusvr-test:/etc/rear# /usr/sbin/rear -V
Relax-and-Recover 2.5 / Git

OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):

root@ubusvr-test:/etc/rear# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

root@ubusvr-test:/etc/rear# cat /etc/rear/local.conf
# Default is to create Relax-and-Recover rescue media as ISO image
# set OUTPUT to change that
# set BACKUP to activate an automated (backup and) restore of your data
# Possible configuration values can be found in /usr/share/rear/conf/default.conf
#
# This file (local.conf) is intended for manual configuration. For configuration
# through packages and other automated means we recommend creating a new
# file named site.conf next to this file and to leave the local.conf as it is.
# Our packages will never ship with a site.conf.
OUTPUT=ISO
OUTPUT_URL=sftp://ubusvr-test@backupsvr-test/media/backupdata/ubusvr-test/iso

BACKUP=BORG
BORGBACKUP_HOST="backupsvr-test"
BORGBACKUP_USERNAME="ubusvr-test"
BORGBACKUP_REPO="/media/backupdata/ubusvr-test/borg"
BORGBACKUP_REMOTE_PATH="/usr/bin/borg"

# Automatic archive pruning
BORGBACKUP_PRUNE_KEEP_DAILY=10

# Archive compression
BORGBACKUP_COMPRESSION="lzma,6"

# Repository encryption
BORGBACKUP_ENC_TYPE="keyfile"
export BORG_PASSPHRASE='badphrase'
COPY_AS_IS_BORG=( "$ROOT_HOME_DIR/.config/borg/keys/" )

# Borg env vars
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
Oracle VirtualBox 6.1 VM
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86_64
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
BIOS, GRUB2
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
Local disk
Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT"):

root@ubusvr-test:/etc/rear# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME        KNAME      PKNAME   TRAN   TYPE FSTYPE   LABEL   SIZE MOUNTPOINT
/dev/loop0  /dev/loop0                 loop squashfs          55M /snap/core18/1705
/dev/loop1  /dev/loop1                 loop squashfs          69M /snap/lxd/14804
/dev/loop3  /dev/loop3                 loop squashfs        55.5M /snap/core18/2409
/dev/loop4  /dev/loop4                 loop squashfs          47M /snap/snapd/16010
/dev/loop5  /dev/loop5                 loop squashfs        61.9M /snap/core20/1518
/dev/loop6  /dev/loop6                 loop squashfs       101.3M /snap/lxd/23155
/dev/sda    /dev/sda            sata   disk                   10G
|-/dev/sda1 /dev/sda1  /dev/sda        part                    1M
`-/dev/sda2 /dev/sda2  /dev/sda        part ext4              10G /
/dev/sr0    /dev/sr0            ata    rom                  1024M

Description of the issue (ideally so that others can reproduce it):
When making a backup with rear, with the above config, I am prompted to enter a password. I have confirmed that SSH access is working appropriately with the following commands.

SSH:

stereo@ubusvr-test:~$ sudo -i
root@ubusvr-test:~# cd /etc/rear
root@ubusvr-test:/etc/rear# source local.conf
root@ubusvr-test:/etc/rear# ssh $BORGBACKUP_USERNAME@$BORGBACKUP_HOST
Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <http://wiki.alpinelinux.org/>.

You can setup the system with the command: setup-alpine

You may change this message by editing /etc/motd.

backupsvr-test:~$ exit
Connection to backupsvr-test closed.
root@ubusvr-test:/etc/rear#

SFTP:

stereo@ubusvr-test:~$ sudo -i
root@ubusvr-test:~# cd /etc/rear
root@ubusvr-test:/etc/rear# source local.conf
root@ubusvr-test:/etc/rear# sftp $BORGBACKUP_USERNAME@$BORGBACKUP_HOST
Connected to backupsvr-test.
sftp> ls
sftp> pwd
Remote working directory: /home/ubusvr-test
sftp> exit
root@ubusvr-test:/etc/rear#

As demonstrated, both SSH and SFTP connect without a password prompt. I have tried to set specific config options in /root/.ssh/config to specify the username, host + private key to use to connect - there was no difference in result. The output when I run rear -D mkbackup:

root@ubusvr-test:/etc/rear# rear -D mkbackup
Relax-and-Recover 2.5 / Git
Running rear mkbackup (PID 73531)
Using log file: /var/log/rear/rear-ubusvr-test.log
Using autodetected kernel '/boot/vmlinuz-5.4.0-110-generic' as kernel in the recovery system
Creating disk layout
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda with GPT BIOS boot partition)
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct ...
Creating root filesystem layout
Handling network interface 'enp0s3'
enp0s3 is a physical device
Handled network interface 'enp0s3'
Cannot include default keyboard mapping (no KEYMAPS_DEFAULT_DIRECTORY specified)
Cannot include keyboard mappings (neither KEYMAPS_DEFAULT_DIRECTORY nor KEYMAPS_DIRECTORIES specified)
Copying logfile /var/log/rear/rear-ubusvr-test.log into initramfs as '/tmp/rear-ubusvr-test-partial-2022-06-19T15:47:37+00:00.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/5.4.0-110-generic (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
Skip copying broken symlink '/etc/mtab' target '/proc/80531/mounts' on /proc/ /sys/ /dev/ or /run/
Symlink '/usr/share/misc/magic' -> '/usr/share/file/magic' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/share/file/magic' via the 'COPY_AS_IS' configuration variable.
Symlink '/lib/modules/5.4.0-110-generic/build' -> '/usr/src/linux-headers-5.4.0-110-generic' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/linux-headers-5.4.0-110-generic' via the 'COPY_AS_IS' configuration variable.
Testing that the recovery system in /tmp/rear.z7ghXzRPVf74TET/rootfs contains a usable system
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (417622712 bytes) in 45 seconds
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-ubusvr-test.iso (413M)
Copying resulting files to sftp location
Saving /var/log/rear/rear-ubusvr-test.log as rear-ubusvr-test.log to sftp location
Copying result files '/var/lib/rear/output/rear-ubusvr-test.iso /tmp/rear.z7ghXzRPVf74TET/tmp/VERSION /tmp/rear.z7ghXzRPVf74TET/tmp/README /tmp/rear.z7ghXzRPVf74TET/tmp/rear-ubusvr-test.log' to sftp location
Password:

The attached debug log file is from the above run, where I pressed CTRL+C to terminate the program at the point the password was prompted.

Workaround, if any:
N/A
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
rear-ubusvr-test.log

To paste verbatim text like command output or file content,
include it between a leading and a closing line of three backticks like

```
verbatim content
```

gdha commented at 2022-06-20 05:45:¶

@StereoRocker If you look carefully in the log file you will find a line:

2022-06-19 15:48:40.513033909 lftp -c open sftp://ubusvr-test@backupsvr-test/media/backupdata/ubusvr-test/iso; mput /var/lib/rear/output/rear-ubusvr-test.iso /tmp/rear.z7ghXzRPVf74TET/tmp/VERSION /tmp/rear.z7ghXzRPVf74TET/tmp/README /tmp/rear.z7ghXzRPVf74TET/tmp/rear-ubusvr-test.log

which is using lftp. That command must be asking for a password I guess

pcahyna commented at 2022-06-20 07:58:¶

I have experimented with the LFTP-dependent backup schemes (sftp is among them) some time ago and I dimly recall that it was not easy (maybe even impossible) to stop lftp from asking for a password, even if it is not really needed :-(

StereoRocker commented at 2022-07-10 16:07:¶

Thank you both. I can indeed confirm it's the lftp command that's prompting for a password, by running the command directly in a shell.

I believe I have resolved the issue by specifying a bogus password in the sftp URL. So changing this line:
OUTPUT_URL=sftp://ubusvr-test@backupsvr-test/media/backupdata/ubusvr-test/iso
to this:
OUTPUT_URL=sftp://ubusvr-test:boguspass@backupsvr-test/media/backupdata/ubusvr-test/iso

May I suggest that the requirement for a bogus password to be entered in the output URL variable for SFTP schema, where SSH keys are being used for authentication, be specified in the documentation please?

github-actions commented at 2022-09-09 03:50:¶

Stale issue message

github-actions commented at 2022-11-09 03:21:¶