#2401 PR `merged`: Create destination directory schema when using lftp¶

Labels: enhancement, fixed / solved / done

gozora opened issue at 2020-05-18 11:14:¶

Pull Request Details:¶

Type: Enhancement
Impact: Low
Reference to related issue (URL): N/A
How was this pull request tested?
Run rear mkrescue with and without destination directory schema created.
Brief description of the changes in this pull request:
When transferring ReaR recovery system ISO, create also destination directory structure (including parents). This is useful because when destination directory structure does not exist (or is incomplete), Lftp puts files into last available directory in OUTPUT_URL scheme.

jsmeix commented at 2020-05-18 12:47:¶

In rear/output/PXE/default/820_copy_to_net.sh
we also have a plain lftp ...put

lftp -c "open $OUTPUT_URL; mput $result_file"

Is there perhaps also a mkdir -p needed?

gozora commented at 2020-05-18 12:50:¶

In rear/output/PXE/default/820_copy_to_net.sh
we also have a plain lftp ...put

I did not specifically aimed for PXE code, but in general adding mkdir -pf .. should not harm ...

V.

jsmeix commented at 2020-05-18 12:53:¶

By the way:
Sometimes we use mkdir $v -p sometimes only mkdir -p
I don't know when $v should be added or not.

gozora commented at 2020-05-18 13:09:¶

mkdir -fp foo/bar
mkdir: invalid option -- 'f'
Try 'mkdir --help' for more information.

This is not standard mkdir from bash, but rather mkdir from lftp as internal command.

excerpt from man lftp

...
mkdir [-p] [-f] dir(s)

Make remote directories. If -p is used, make all components of paths.  The -f option makes mkdir quiet and suppresses messages.
...

jsmeix commented at 2020-05-18 14:01:¶

@gozora
thank you for your explanations.
Now even I see that this mkdir is a special one :-)

gozora commented at 2020-05-18 14:19:¶

@jsmeix many thanks for your review!

V.

gozora commented at 2020-05-19 08:56:¶

@jsmeix will do!

Thanks for heads up!

V.

jsmeix commented at 2020-05-19 09:05:¶

A first test looks good to me:

# url_path "ftp://host_name/destination_directory"

/destination_directory

gozora commented at 2020-05-19 09:11:¶

Yep, looks good.

+++ url_path sftp://backup/mnt/rear/borg/iso/node1
+++ local url=sftp://backup/mnt/rear/borg/iso/node1
+++ local url_without_scheme=backup/mnt/rear/borg/iso/node1
+++ echo /mnt/rear/borg/iso/node1
++ local path=/mnt/rear/borg/iso/node1
++ lftp -c 'open sftp://backup/mnt/rear/borg/iso/node1; mkdir -fp /mnt/rear/borg/iso/node1'
cd: Access failed: No such file (/mnt/rear/borg/iso/node1)
++ lftp -c 'open sftp://backup/mnt/rear/borg/iso/node1;  mput /var/lib/rear/output/rear-node1.iso /tmp/rear.i6vmXjtPdArtp7S/tmp/VERSION /tmp/rear.i6vmXjtPdArtp7S/tmp/README /tmp/rear.i6vmXjtPdArtp7S/tmp/rear-node1.log'

V.

gozora commented at 2020-05-19 09:15:¶

And also when directory is already there:

+++ url_path sftp://backup/mnt/rear/borg/iso/node1
+++ local url=sftp://backup/mnt/rear/borg/iso/node1
+++ local url_without_scheme=backup/mnt/rear/borg/iso/node1
+++ echo /mnt/rear/borg/iso/node1
++ local path=/mnt/rear/borg/iso/node1
++ lftp -c 'open sftp://backup/mnt/rear/borg/iso/node1; mkdir -fp /mnt/rear/borg/iso/node1'
++ lftp -c 'open sftp://backup/mnt/rear/borg/iso/node1;  mput /var/lib/rear/output/rear-node1.iso /tmp/rear.pvWFPUSSvVmxuY0/tmp/VERSION /tmp/rear.pvWFPUSSvVmxuY0/tmp/README /tmp/rear.pvWFPUSSvVmxuY0/tmp/rear-node1.log'

gozora commented at 2020-05-19 09:29:¶

Ok, I just found out that $path is already defined and used in 950_copy_result_files.sh :-)

Will fix that!

V.

gozora commented at 2020-05-19 09:44:¶

Hope that 33bfbe17bec1efe44afe2a2235f864d39f8ce2a2 was the last one in this PR! :-)

V.

jsmeix commented at 2020-05-19 10:00:¶

I fear LFTP related issues may grow and grow
and this one is still a lot smaller than
https://github.com/rear/rear/pull/2384
15 commits there versus 5 commits here
;-)

gozora commented at 2020-05-19 10:09:¶

I fear LFTP related issues may grow and grow
and this one is still a lot smaller than

It depends on how many people will use it ...

I personally went this way, to completely eliminate NFS (running with BACKUP=BORG over ssh), because security people somehow dislike NFS ;-).

V.

gozora commented at 2020-05-19 14:32:¶

@gdha, @jsmeix many thanks for your review!

V.

jsmeix commented at 2020-05-19 14:46:¶

@gozora
thank you for your enhancement!

[Export of Github issue for rear/rear.]

#2401 PR merged: Create destination directory schema when using lftp¶