#1105 Issue `closed`: rear with a very big usb device¶

Labels: enhancement, support / question, fixed / solved / done

dwerner1 opened issue at 2016-12-05 14:57:¶

I'm trying to get rear working with the following setup:

rear version (/usr/sbin/rear -V): 1.19-1
OS version (cat /etc/rear/os.conf or lsb_release -a): Debian testing 8.1.0
rear configuration files (cat /etc/rear/site.conf):

BACKUP=NETFS
OUTPUT=ISO
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
ISO_PREFIX=”rear-$HOSTNAME”
BACKUP_PROG_EXCLUDE=( ‘/tmp/’ ‘/dev/shm/’ ‘/mnt/’ ‘/media/’ $VAR_DIR/output/* )
BACKUP_SELINUX_DISABLE=1
BACKUP_TYPE=incremental
FULLBACKUPDAY=Mon

Brief description of the issue

I have an external USB device set up as RAID1, 16TB of size. I have chosen btfrs and also ext4 as filesystem, but both render being not bootable. The backup process and every rear function work fine. I never saw that problem with smaller disks and ext3, what am I doing wrong?

Dirk

gozora commented at 2016-12-05 16:01:¶

Hi @dwerner1,

I somehow can't figure out what is not working for you.
Are you using legacy or EFI boot?
Are you trying to backup your OS to some external (16TB USB) disk, and then recover from it (like do USB boot and rear recover)?
If so, how did you chose "btrfs and also ext4" as filesystem for /dev/disk/by-label/REAR-000?
Because normally you should do something like rear --format <device_name> which will format your disk with mkfs.ext3 ...

There might be even limit with MBR not supporting more than 2TB partitions.

dwerner1 commented at 2016-12-05 16:16:¶

Hi gozora,
thanks for sharing this issue!
Yes, I'm planning to backup a complete linux machine with OS and data and I wanna be able to recover that from a USB DAS device that contains 4 x 8TB disks, running as RAID1. I chose btfrs and ext4 because with ext3 I couldn't handle this partition size > 2TB. My main question is actually - can I leave out the rear setup step
sudo rear -v format /dev/sdX
and do the formatting manually and then proceed with
sudo rear -v mkrescue ?

gozora commented at 2016-12-05 16:45:¶

Ufff, that is hard question. In theory it should work, but reading, I'd say you should follow guidelines.

If I could give you an advice,

try to split your backup tasks between OS and data backup. (with support for multiple backups commit by @jsmeix it should be quite easy, it is not fully documented though, but you can check some examples here
do not try to boot from DAS directly, but use it just for data storage. Create small bootable USB stick instead, where you can be sure that you will not hit some kind of limit. Something like:

OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP=NETFS
BACKUP_URL=file:///directory/path/

I've never tried something like this, but there is a good chance that it will work.

V.

dwerner1 commented at 2016-12-05 16:52:¶

Wow, that's a really cool idea to split tasks and to put the boot related stuff on a separate small usb device. I will test that tomorrow. Again, many thanks for your input!!

dwerner1 commented at 2016-12-06 18:51:¶

I have tried another scenario today cause I was curious if that might work out:

I formatted an external 320GB USB hdd by doing

sudo rear -v format /dev/sdc1

That created an 320GB ext3 partition. Then I shrinked this partition to 100GB and created a btrfs partition in the freed space, labeled REAR-DATA . I adjusted the file /etc/rear/site.conf accordingly to

OUTPUT=USB USB_DEVICE=/dev/disk/by-label/REAR-000 BACKUP=NETFS BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA ISO_PREFIX=”rear-$HOSTNAME” BACKUP_PROG_EXCLUDE=( ‘/tmp/*’ ‘/dev/shm/*’ ‘/mnt/*’ ‘/media/*’ $VAR_DIR/output/\* ) BACKUP_SELINUX_DISABLE=1 BACKUP_TYPE=incremental FULLBACKUPDAY=Tue

That boots and offers the recovery menu! If I create the ext3 and btrfs partition manually, it does not work and says Missing operating system when trying to boot.

The idea came up cause I'd prefer to have everything disaster recovery specific on one device. Unfortunately my initial problem is not solved by that - I could never do that with the 16TB device, cause rear will understandably refuse to format that completely with an ext3 fs

dwerner1 commented at 2016-12-06 18:53:¶

Here's the file /etc/rear/site.conf again, hopefully in a better formatting

OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-DATA
ISO_PREFIX=”rear-$HOSTNAME”
BACKUP_PROG_EXCLUDE=( ‘/tmp/*’ ‘/dev/shm/*’ ‘/mnt/*’ ‘/media/*’ $VAR_DIR/output/\* )
BACKUP_SELINUX_DISABLE=1
BACKUP_TYPE=incremental
FULLBACKUPDAY=Tue

jsmeix commented at 2016-12-07 09:20:¶

I have no experience with such huge disks.

@dwerner1

Out of curiosity:
Why do you use btrfs at all?
I.e. what specific btrfs feature do you need?
In https://en.wikipedia.org/wiki/Ext4 I read

Red Hat recommends using XFS instead of ext4
for volumes larger than 100 TB.

and in general for "data" we (i.e. SUSE) also recommend XFS
(by default in SLE12 we use btrfs only for the basic system stuff)
so that I wonder why you don't use ext4 (should be o.k. up to 16TB)
or in general XFS for any "data" partitions?

FYI:
The ReaR script
usr/share/rear/format/USB/default/300_format_usb_disk.sh
does the formatting of the USB device
which you can adapt and enhance as you need it
for your particular case, cf. the
"Disaster recovery with Relax-and-Recover (ReaR)"
section and its sub-sections in
https://en.opensuse.org/SDB:Disaster_Recovery

gozora commented at 2016-12-07 09:53:¶

I've managed to make following configuration work:

BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=file:///backup/data/

EXCLUDE_MD=( $(grep -o -E '^md[0-9]+' /proc/mdstat) ) # exclude all md devices
COPY_AS_IS=( ${COPY_AS_IS[@]} /sbin/sysctl /etc/sysctl.conf /sbin/vconfig /sbin/if* /etc/sysconfig/network )
GRUB_RESCUE=n
ONLY_INCLUDE_VG=( "centos" )
EXCLUDE_BACKUP=( ${EXCLUDE_BACKUP[@]} fs:/crash fs:/usr/sap fs:/oracle )
BACKUP_PROG_EXCLUDE=( ${BACKUP_PROG_EXCLUDE[@]} '/mnt/*' )

This however requires modification of usr/share/rear/format/USB/default/300_format_usb_disk.sh as mentioned by @jsmeix.
e.g.

- echo "Yes" | parted -s $RAW_USB_DEVICE mkpart primary 0 100% >&2
+ echo "Yes" | parted -s $RAW_USB_DEVICE mkpart primary 0 25% >&2

rear format /dev/sdb
load whatever filesystem on free 75% of USB drive.
mount it to /backup/data/
rear mkbackup

This should create bootable USB disk on partition 1 and store data on partition 2

I was not able to test restore, as I don't have any easy way how to boot from USB. Maybe you will need to manually mount /dev/sdb2 to /backup/data once ReaR rescue/recovery system is loaded.

V.

gozora commented at 2016-12-07 10:14:¶

Ok, I've found a way (again couple of minutes after my last comment, this is my curse) how to test this, setup.
And I can confirm that it works just fine for me ...

jsmeix commented at 2016-12-07 11:02:¶

I think I could add some variables so that the user can specify
to some basic extent how to partition and format a USB disk.

gozora commented at 2016-12-07 11:35:¶

@jsmeix

I think I could add some variables so that the user can specify
to some basic extent how to partition and format a USB disk.

When I did implementation for USB / EFI, I maybe wrote something that might be useful.
see

dwerner1 commented at 2016-12-07 11:55:¶

@gozora
I can confirm that the above setup works like a charm! Great!
There was no specific plan behind using btfrs, I tried various file systems including xfs, ext4 and btfrs . I'll choose XFS for the data partition

jsmeix commented at 2016-12-07 16:31:¶

Have a look at my tentative attempt in
https://github.com/rear/rear/pull/1112
to support partitioning and formatting a huge medium
via the new variables USB_DEVICE_FILESYSTEM
and USB_DEVICE_FILESYSTEM_PERCENTAGE
for details see conf/default.conf

dwerner1 commented at 2016-12-07 17:32:¶

The script /usr/share/rear/format/USB/default/300_format_usb_disk.sh creates an msdos partition table by default, I have changed line 7

`- echo "Yes" | parted -s $RAW_USB_DEVICE mklabel msdos >&2

echo "Yes" | parted -s $RAW_USB_DEVICE mklabel gpt >&2`

to be able to create an xfs partition of 14TB afterwards

dwerner1 commented at 2016-12-07 17:33:¶

- echo "Yes" | parted -s $RAW_USB_DEVICE mklabel msdos >&2
+ echo "Yes" | parted -s $RAW_USB_DEVICE mklabel gpt >&2

jsmeix commented at 2016-12-08 09:39:¶

@dwerner1
many thanks for your valuable feedback.

I have seen that too but it didn't trigger the right things in my mind.
Only in case of U(EFI) it creates a GPT.

I had never worked before with the 'format' workflow scripts
but from my first glance they look somewhat "weird" to me.
I think a general overhaul of the 'format' workflow could be
a good idea but currently I do not understand it sufficiently
to do that so that for now I work around with new variables
so that experienced users can manually specify what to get.

In https://en.wikipedia.org/wiki/Master_boot_record
I read right now that

The organization of the partition table in the MBR
limits the maximum addressable storage space
of a disk to 2 TiB

Therefore I need one more new variable
USB_DEVICE_PARTED_LABEL
that is by default 'msdos' but can be set to 'gpt'
by the user when the medium is bigger than 2TB.

jsmeix commented at 2016-12-08 10:41:¶

@gozora @dwerner1 @gdha
does anybody have an idea what the reason behind is why in
usr/share/rear/format/USB/default/300_format_usb_disk.sh
there is

echo "Yes" | parted -s ...

regardless that "man parted" reads:

       -s, --script
              never prompts for user intervention

jsmeix commented at 2016-12-08 14:49:¶

With
https://github.com/rear/rear/pull/1112
merged there should be now support for partitioning
and formatting even huge devices via the 'format' workflow.
Accordingly I think this issue is fixed.

[Export of Github issue for rear/rear.]

#1105 Issue closed: rear with a very big usb device¶