#2723 PR merged: fixed pxe file cp and permissions for sshfs target¶
Labels: bug, fixed / solved / done
DEvil0000 opened issue at 2021-12-06 13:52:¶
- Type: Bug Fix
- Impact: Normal
- How was this pull request tested?
OUTPUT=PXE
OUTPUT_PREFIX_PXE=$HOSTNAME
PXE_TFTP_PREFIX=$HOSTNAME.
PXE_TFTP_URL=sshfs://wingcon@172.200.200.1/tmp/test
PXE_CONFIG_URL=sshfs://wingcon@172.200.200.1/tmp/test/
PXE_RECOVER_MODE="manual"
PXE_CREATE_LINKS="IP"
BACKUP=BORG
USB_DEVICE=/dev/disk/by-label/REAR-000
USB_DEVICE_FILESYSTEM_LABEL=REAR-000
USB_DEVICE_PARTED_LABEL=gpt
USB_DEVICE_FILESYSTEM=ext4
USB_UEFI_PART_SIZE="512"
CLONE_USERS=(root)
CLONE_GROUPS=(root)
CLONE_ALL_USERS_GROUPS="false"
BORGBACKUP_REPO="/borg"
BORGBACKUP_UMASK="0002"
BORGBACKUP_ENC_TYPE="repokey"
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"
export TMPDIR="/wsp_var/tmp/"
USE_RESOLV_CONF="no"
USE_DHCLIENT=no
USE_STATIC_NETWORKING=1
USE_SERIAL_CONSOLE=y
COPY_KERNEL_PARAMETERS=( 'net.ifnames' 'biosdevname', 'console' )
REAR_INITRD_COMPRESSION="fast"
USING_UEFI_BOOTLOADER=1
USB_BOOTLOADER=grub
GRUB2_DEFAULT_BOOT="1"
MODULES=( 'no_modules' )
FIRMWARE_FILES=( 'no' )
EXCLUDE_RUNTIME_LOGFILE="yes"
-
Brief description of the changes in this pull request:
-
prevent cp error "failed to preserve ownership" for sshfs
-
prevent issue with write permissions trying to override image on later runs
Please note that atm PXE config is always written in syslinux style and never for grub.
jsmeix commented at 2021-12-07 11:47:¶
@rear/contributors
if someone of you is a PXE user or has good PXE knowledge
could you please also have a look here if it looks OK to you?
jsmeix commented at 2021-12-07 11:48:¶
@DEvil0000
thank you for your continuous fixes and improvements in various areas of
ReaR!
It is much appreciated.
jsmeix commented at 2021-12-09 09:50:¶
I will merge it as is today afternoon.
Flunkyball commented at 2021-12-27 13:59:¶
I just gave the fix a try however there is one issue occurring now:
The symlinks created on the sshfs share for either the IP or MAC of the rear client are now pointing to "/rear-client" instead of "rear-client". Hence the configuration is not found anymore.
jsmeix commented at 2022-01-03 13:48:¶
@Flunkyball
I am not a PXE user so I cannot actually see or reproduce how it fails
now.
Could you provide a more detailed description how which symlinks and
files
have been where before and where and how they are now as comparison?
In particular how symlinks and files are on the original system and what
the
cp commands result for them, cf. my findings about cp -L and
symlinks in
https://github.com/rear/rear/issues/2677#issuecomment-997859219
Ideally could you show us what code changes based on the current master
code
make it work again for you - i.e. not just reverting the changes of this
pull request
but that the intent behind this pull request is still provided and also
that the symlinks
are correct again.
Flunkyball commented at 2022-01-05 09:19:¶
@jsmeix
Sure, I just ran it again to show you the detailed output.
Used config:
OUTPUT=PXE
OUTPUT_PREFIX_PXE=$HOSTNAME
PXE_TFTP_IP=XXX.XXX.XXX.XXX
PXE_TFTP_PREFIX=$HOSTNAME.
PXE_RECOVER_MODE="manual"
PXE_CREATE_LINKS=MAC
PXE_REMOVE_OLD_LINKS=y
######### VARIANT 1: NFS FOR COPYING FILES
#PXE_TFTP_URL="nfs://$PXE_TFTP_IP/srv/pxe/tftp"
#PXE_CONFIG_URL="nfs://$PXE_TFTP_IP/srv/pxe/tftp/pxelinux.cfg"
######### VARIANT 2: SSHFS FOR COPYING FILES
PXE_TFTP_URL=sshfs://USERNAME@$PXE_TFTP_IP/tftp
PXE_CONFIG_URL=sshfs://USERNAME@$PXE_TFTP_IP/tftp/pxelinux.cfg
BACKUP=BAREOS
BAREOS_CLIENT=SOME-CIENT-fd
BAREOS_RESTORE_JOB=RestoreFiles-Full
BAREOS_FILESET=FS-Linux
Here is the comparison between variant 1 using NFS to copy the rear information working as expected vs. variant 2 using sshfs resulting in a wrong symlink:
VARIANT 1 (NFS):
user@pxe:/srv/tftp/pxelinux.cfg# ls -l | grep "HOSTNAME"
lrwxrwxrwx 1 root root 14 Jan 4 21:01 01-01-01-01-01-01-4a -> rear-HOSTNAME
lrwxrwxrwx 1 root root 14 Jan 4 21:01 01-01-01-01-01-01-8c -> rear-HOSTNAME
lrwxrwxrwx 1 root root 14 Jan 4 21:01 01-01-01-01-01-01-4c -> rear-HOSTNAME
-r--r--r-- 1 pxe Domain Computers 898 Jan 4 21:01 rear-HOSTNAME
VARIANT 2 (SSHFS):
user@pxe:/srv/tftp/pxelinux.cfg# ls -l | grep "HOSTNAME"
lrwxrwxrwx 1 pxe Domain Computers 15 Jan 5 10:05 01-01-01-01-01-01-4a -> /rear-HOSTNAME
lrwxrwxrwx 1 pxe Domain Computers 15 Jan 5 10:05 01-01-01-01-01-01-8c -> /rear-HOSTNAME
lrwxrwxrwx 1 pxe Domain Computers 15 Jan 5 10:05 01-01-01-01-01-01-4c -> /rear-HOSTNAME
-r--r--r-- 1 pxe Domain Computers 898 Jan 4 21:01 rear-HOSTNAME
As you can see using SSHFS as transfermethod produces a wrong symlink not corretly pointing to the "rear-HOSTNAME" folder within the same directory. Hence The pxe startup is not successful because the rear files are not found anymore.
jsmeix commented at 2022-01-05 13:24:¶
@Flunkyball
I cannot reproduce it on my openSUSE Leap 15.3 homeoffice laptop
where I use the '/nfs' directory locally as exported NFS share.
With etc/rear/local.conf for NFS
OUTPUT=PXE
OUTPUT_PREFIX_PXE=$HOSTNAME
PXE_TFTP_IP=127.0.0.1
PXE_TFTP_PREFIX=$HOSTNAME.
PXE_RECOVER_MODE="manual"
PXE_CREATE_LINKS=MAC
PXE_REMOVE_OLD_LINKS=y
PXE_TFTP_URL="nfs://$PXE_TFTP_IP/nfs/pxe/tftp"
PXE_CONFIG_URL="nfs://$PXE_TFTP_IP/nfs/pxe/tftp/pxelinux.cfg"
#PXE_TFTP_URL="sshfs://rear@$PXE_TFTP_IP/nfs/pxe/tftp"
#PXE_CONFIG_URL="sshfs://rear@$PXE_TFTP_IP/nfs/pxe/tftp/pxelinux.cfg"
I get
4980737 4 drwxrwxrwx 3 root root 4096 Jan 5 14:10 .
4980738 4 drwxrwxrwx 4 root root 4096 Jan 5 14:12 ./tftp
4980739 4 drwxrwxrwx 2 root root 4096 Jan 5 14:12 ./tftp/pxelinux.cfg
4980744 4 -r--r--r-- 1 nobody nobody 933 Jan 5 14:12 ./tftp/pxelinux.cfg/rear-linux-h9wr
4980747 0 lrwxrwxrwx 1 nobody nobody 15 Jan 5 14:12 ./tftp/pxelinux.cfg/01-52-54-00-eb-ce-15 -> rear-linux-h9wr
4980745 0 lrwxrwxrwx 1 nobody nobody 15 Jan 5 14:12 ./tftp/pxelinux.cfg/01-c8-cb-b8-41-16-f6 -> rear-linux-h9wr
4980746 0 lrwxrwxrwx 1 nobody nobody 15 Jan 5 14:12 ./tftp/pxelinux.cfg/01-b6-70-fc-a3-41-08 -> rear-linux-h9wr
4980740 4 drwxr-xr-x 2 nobody nobody 4096 Jan 5 14:12 ./tftp/linux-h9wr
4980741 8852 -rw-r--r-- 1 nobody nobody 9062624 Jan 5 14:12 ./tftp/linux-h9wr/linux-h9wr.kernel
4980743 4 -rw-r--r-- 1 nobody nobody 267 Jan 5 14:12 ./tftp/linux-h9wr/linux-h9wr.message
4980742 67708 -rw-r--r-- 1 nobody nobody 69331494 Jan 5 14:12 ./tftp/linux-h9wr/linux-h9wr.initrd.cgz
With etc/rear/local.conf for sshfs
OUTPUT=PXE
OUTPUT_PREFIX_PXE=$HOSTNAME
PXE_TFTP_IP=127.0.0.1
PXE_TFTP_PREFIX=$HOSTNAME.
PXE_RECOVER_MODE="manual"
PXE_CREATE_LINKS=MAC
PXE_REMOVE_OLD_LINKS=y
#PXE_TFTP_URL="nfs://$PXE_TFTP_IP/nfs/pxe/tftp"
#PXE_CONFIG_URL="nfs://$PXE_TFTP_IP/nfs/pxe/tftp/pxelinux.cfg"
PXE_TFTP_URL="sshfs://rear@$PXE_TFTP_IP/nfs/pxe/tftp"
PXE_CONFIG_URL="sshfs://rear@$PXE_TFTP_IP/nfs/pxe/tftp/pxelinux.cfg"
I get
# find . -ls
5373953 4 drwxrwxrwx 3 root root 4096 Jan 5 14:02 .
5373954 4 drwxrwxrwx 4 root root 4096 Jan 5 14:08 ./tftp
5373955 4 drwxrwxrwx 2 root root 4096 Jan 5 14:08 ./tftp/pxelinux.cfg
5373960 4 -r--r--r-- 1 rear users 933 Jan 5 14:08 ./tftp/pxelinux.cfg/rear-linux-h9wr
5373963 0 lrwxrwxrwx 1 rear users 15 Jan 5 14:08 ./tftp/pxelinux.cfg/01-52-54-00-eb-ce-15 -> rear-linux-h9wr
5373961 0 lrwxrwxrwx 1 rear users 15 Jan 5 14:08 ./tftp/pxelinux.cfg/01-c8-cb-b8-41-16-f6 -> rear-linux-h9wr
5373962 0 lrwxrwxrwx 1 rear users 15 Jan 5 14:08 ./tftp/pxelinux.cfg/01-b6-70-fc-a3-41-08 -> rear-linux-h9wr
5373956 4 drwxr-xr-x 2 rear users 4096 Jan 5 14:08 ./tftp/linux-h9wr
5373957 8852 -rw-r--r-- 1 rear users 9062624 Jan 5 14:08 ./tftp/linux-h9wr/linux-h9wr.kernel
5373959 4 -rw-r--r-- 1 rear users 267 Jan 5 14:08 ./tftp/linux-h9wr/linux-h9wr.message
5373958 67708 -rw-r--r-- 1 rear users 69331809 Jan 5 14:08 ./tftp/linux-h9wr/linux-h9wr.initrd.cgz
jsmeix commented at 2022-01-05 13:28:¶
Furthermore the changes in this pull request here are only in
usr/share/rear/output/PXE/default/800_copy_to_tftp.sh
while the symlinks are created in
usr/share/rear/output/PXE/default/810_create_pxelinux_cfg.sh
Flunkyball commented at 2022-01-05 16:22:¶
@jsmeix
Interesting that you cannot reproduce it. I tried it on a second client.
The first was Arch Linux the second is a Debian client with exactly the
same result regarding the symlinks.
On both clients I did a complete fresh git clone and launched rear
mkrescue from there.
I checked "usr/share/rear/output/PXE/default/810_create_pxelinux_cfg.sh" and had the ln -sf command logged for each entry with the following result:
2022-01-05 17:01:29.963735941 Including output/PXE/default/800_copy_to_tftp.sh
2022-01-05 17:01:29.977561505 Mounting with 'sshfs pxe@pxe-ssh:/tftp /var/tmp/rear.EbKZvIBgoIzaN8M/tftpbootfs -o rw,noatime'
2022-01-05 17:02:14.040837389 Copied kernel+initrd 453M to sshfs://pxe@pxe-ssh/tftp/HOSTNAME
2022-01-05 17:02:14.066075187 Including output/PXE/default/810_create_pxelinux_cfg.sh
2022-01-05 17:02:14.091668069 Mounting with 'sshfs pxe@pxe-ssh:/tftp/pxelinux.cfg /var/tmp/rear.EbKZvIBgoIzaN8M/tftpbootfs -o rw,noatime'
2022-01-05 17:02:14.921060403 ln -sf rear-HOSTNAME 01-01-01-01-01-01-4a
2022-01-05 17:02:14.947981748 ln -sf rear-HOSTNAME 01-01-01-01-01-01-4c
2022-01-05 17:02:14.973756517 ln -sf rear-HOSTNAME 01-01-01-01-01-01-8c
2022-01-05 17:02:14.983073672 Created pxelinux config 'rear-HOSTNAME' and symlinks for MAC adresses in sshfs://pxe@pxe-ssh/tftp/pxelinux.cfg
As you can see the parameters which are used by the script for the symlink are absolutely correct.
Running the same again with NFS as mount method produces the same log regarding the "ln -sf" command. However the resulting symlink on the share is correct.
If I manually mount the sshfs share with "sshfs pxe@pxe-ssh:/tftp
/mnt/test"
and create the symlink with "ln -sf rear-HOSTNAME 01-01-01-01-01-01-4a"
I end up again with a target including "/"
Hence it seems to be an issue with sshfs mount. However not on the side if rear I guess ;-)
Flunkyball commented at 2022-01-06 13:52:¶
I can confirm now that everything regarding rear and especially the fix discussed here is working.
The problem I was experiencing is do to the fact that debian stretch uses an unbelievable old version of proftpd (1.3.5 from 2017) in its repository. This version in particular had issues with symlinks generated wrong when mounted via sshfs.
Compiling the latest release 1.3.7c from 2021 myself it is working now without any issues.
@jsmeix
Thx for your support and the help to trace the issue !
[Export of Github issue for rear/rear.]