#1333 Issue closed: Any way to speed up mkbackup to cifs share? (getting less than 900KBps w/gzip)

Labels: support / question, fixed / solved / done

jiphen opened issue at 2017-04-25 15:12:

Hi,
I've been able to use mkbackup to store our backups onto a data domain cifs share but its been painfully slow, typically around 700 KBps. These backups are around 200+GB and its taken over 4 days to archive these backups to the share. I have a couple SLES 12 SP1/2 machines that act the same.

I have added the REAR_INITRD_COMPRESSION=fast which did help a little bit, maybe ~100KBps but still no where near an acceptable timeframe.
I have also added the BACKUP_PROG_COMPRESS_OPTIONS=( -I 'gzip -1 -n -c' ) which didn't have any effect on the speed.

Any other thoughts on how to speed up backups to a cifs share with gzip?

suse1 slow
suse4 slow

  • rear version (/usr/sbin/rear -V):
    Relax-and-recover 2.00 / Git

  • OS version (cat /etc/rear/os.conf):
    OS_VENDOR=SUSE_LINUX
    OS_VERSION=12

  • rear configuration files (cat /etc/rear/local.conf):
    OUTPUT=ISO
    OUTPUT_PREFIX='rear-suse1'
    BACKUP=NETFS
    BACKUP_OPTIONS=cred=/etc/rear/login.cifs
    BACKUP_URL=cifs://IP-ADDR/DDServers/
    ISO_MKISOFS_BIN=/usr/bin/ebiso
    NETFS_KEEP_OLD_BACKUP_COPY=yes
    REAR_INITRD_COMPRESSION=fast
    REQUIRED_PROGS=( "${REQUIRED_PROGS[@]}" snapper chattr lsattr )
    COPY_AS_IS=( "${COPY_AS_IS[@]}" /usr/lib/snapper/installation-helper /etc/snapper/config-templates/default )
    BACKUP_PROG_INCLUDE=( '/var/tmp/' '/srv/' '/var/lib/pgsql/' '/var/spool/' '/var/lib/libvirt/images/' '/var/opt/' '/tmp/' '/var/lib/named/' '/var/log/' '/boot/grub2/i386/' '/var/lib/mariadb/' '/home/' '/var/lib/mailman/' '/opt/' '/usr/local/' '/boot/grub2/x86_64/' )

  • Are you using legacy BIOS or UEFI boot?
    UEFI

gozora commented at 2017-04-25 16:17:

Well CIFS ...

Doing some backups over CIFS gave me (from one virtual machine to another) ~3MB/s.
I really don't think that ReaR is troublemaker here.
Did you try just to transfer some large files to your server using mount.cifs or smbclient? How was the performance?

Maybe you have some curious antivirus/firewall/other type of security malware sofware running on your samba server?

Why are your OS backups over 200GB ?

V.

jsmeix commented at 2017-04-26 10:09:

@jiphen
in general when you output your (backup) data via network
(BACKUP=NETFS) ReaR cannot do anything regarding
data transmission speed in your particular network.

Regarding BACKUP_URL=cifs://

I am not at all a CIFS user but as far as I see
the code in ReaR related to using CIFS
is primarily the mount_url() function in
usr/share/rear/lib/global-functions.sh

mount_url() {
    ...
        (cifs)
            if [ x"$options" = x"$defaultoptions" ];then
                # defaultoptions contains noatime which is not valid for cifs (issue #752)
                mount_cmd="mount $v -o rw,guest //$(url_host $url)$(url_path $url) $mountpoint"
            else
                mount_cmd="mount $v -o $options //$(url_host $url)$(url_path $url) $mountpoint"
            fi
            ;;

Perhaps you could somehow tweak that via special mount options
to get a faster data transmission speed to your remote CIFS share?

Regarding "backups are around 200+GB":

In general regarding huge backups see
https://github.com/rear/rear/issues/1006#issuecomment-248862040

In particular regarding how to split huge backups
into a primary part that is needed to recreate
the basic system after a disaster happened and
other parts for application data and things like that see
https://github.com/rear/rear/blob/master/doc/user-guide/11-multiple-backups.adoc

In general regarding issues with the backup:

Relax-and-Recover is meant as disaster recovery tool
to recreate the basic system after a disaster happened, cf.
https://en.opensuse.org/SDB:Disaster_Recovery

Relax-and-Recover is neither a backup software nor a
backup management software and it is not meant to be one, cf.
"Relax-and-Recover versus backup and restore" in
https://en.opensuse.org/SDB:Disaster_Recovery

jsmeix commented at 2017-05-03 09:38:

I think the question is sufficiently answered
so that I close this issue.

@jiphen
you can still add comments even when the issue is closed.
If you find out how one could mount a remote CIFS share
with special mount options so that data transmission speed
is faster, please report it here so that we could enhance ReaR
to use those special mount options by default for CIFS.

[Export of Github issue for rear/rear.]