#1439 Issue `closed`: Unable to boot from 1Tb external USB drive¶

Labels: support / question, fixed / solved / done

deonlinux opened issue at 2017-08-04 10:15:¶

Relax-and-Recover (ReaR) Issue Template¶

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

rear version (/usr/sbin/rear -V):Relax-and-Recover 2.2 / Git
OS version (cat /etc/rear/os.conf or lsb_release -a):
LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.6 (Final)
Release: 6.6
Codename: Final
rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):

[root@localhost rear-new]# cat etc/rear/local.conf 
# Default is to create Relax-and-Recover rescue media as ISO image
# set OUTPUT to change that
# set BACKUP to activate an automated (backup and) restore of your data
# Possible configuration values can be found in /usr/share/rear/conf/default.conf
#
# This file (local.conf) is intended for manual configuration. For configuration
# through packages and other automated means we recommend creating a new
# file named site.conf next to this file and to leave the local.conf as it is. 
# Our packages will never ship with a site.conf.
#
### write the rescue initramfs to USB and update the USB bootloader
OUTPUT=USB
#OUTPUT_URL=usb:///dev/sdb
#OUTPUT_URL=usb:///dev/disk/by-label/REAR-000

#OUTPUT=ISO
#OUTPUT_URL=file:///var/lib/rear/output/

### create a backup using the internal NETFS method, using 'tar'
BACKUP=NETFS

### write both rescue image and backup to the device labeled REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

Are you using legacy BIOS or UEFI boot? Legacy
Brief description of the issue:
Using external Manhattan SATA 3.5 " enclosure, USB3 interface connected to Intel Server USB2 interface. Intel S1200BC server motherboard. Unable to boot from external after creating Recovery & Backup.
Note: Smaller Sata disk (250Gb tried) works fine. Suspect Grub limitation (large number of cyl)? log has no errors.
Yet Centos can create bootdisk successfully on 1TB. Creates 2 partitions though (LVM).
Work-around, if any:Use CDROM rescue media and backup to USB device. inconvenient but usable.
Included files from /tmp folder during backup creation for 1TB disk:

fdisk_output.txt
parted.txt
partitions.txt
partitions-data.txt
partitions_unsorted.txt

gozora commented at 2017-08-04 10:29:¶

Hi @deonlinux,

Before we go any further; can you please try with configuration similar to following:

# USB testing
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

V.

gozora commented at 2017-08-04 10:31:¶

@deonlinux
Did you format your disk with rear format .... command ?

V.

deonlinux commented at 2017-08-04 11:14:¶

Yes, drive was formatted with rear format cmd
I have also used the suggestions specified previously (USB_DEVICE=/dev/disk/by-label/REAR-000 etc)
Sadly still does not work.

gozora commented at 2017-08-04 12:24:¶

You might have really hit some limits, so I can only guess from now on.
ReaR options I'd experiment with would be following:

USB_DEVICE_PARTED_LABEL=gpt #default is msdos

or maybe try to create smaller partition on your disk using:

USB_DEVICE_FILESYSTEM_PERCENTAGE=20 #default is 100

Full list can be find in default.conf

V.

deonlinux commented at 2017-08-04 13:49:¶

The suggestion of specifying gpt worked!!!
Percentage did not.
Great thanks gozora!
We can close this now I guess

schlomo commented at 2017-08-04 13:50:¶

If this is really a size limit then we should warn the user that it won't work.

Maybe we can default to GPT if this is an EFI system?

gozora commented at 2017-08-04 14:02:¶

@deonlinux, you are always welcome.

Hi @schlomo we are already doing this. Excerpt from default.conf comment:

# When UEFI is used the format workflow will create a GUID partition table in any case.

@jsmeix if I did not mention it earlier, thanks for your great comments!!!

V.

schlomo commented at 2017-08-04 14:32:¶

@gozora I am not sure about that. As far as I can understand 300_format_usb_disk.sh there is no auto detect mechanism for USB_DEVICE_PARTED_LABEL. If EFI is set then it forces EFI mode.

I think that the code actually has two paths:

EFI (set by command line) creates an EFI System Partition and of course uses gpt
Otherwise it uses gpt or msdos to create a BIOS boot medium

So to make @deonlinux happy we probably should have two types of auto detect here:

automatically use EFI mode for EFI systems (currently requires manual setting)
automatically use gpt, either on modern Linux distros or at least for large media.

jsmeix commented at 2017-08-04 14:40:¶

@gozora
the exact condition is in format/USB/default/300_format_usb_disk.sh

if is_true "$EFI" ; then

so that it depends on when "$EFI" has a 'true' value.
It has a 'true' value at least when the user specifies it
when calling 'rear format' e.g. as in

rear -v format -- --efi --force /dev/sdX

Note the mandatory '--' before the format workflow specific
options ('-v' is the generic rear option).
Without that the format workflow specific options won't work.
For details read the getopt documentation.

jsmeix commented at 2017-08-04 14:42:¶

In general regarding using modern things
more or less automatically, cf.
https://github.com/rear/rear/issues/1390

deonlinux commented at 2017-08-04 14:43:¶

My apologies - used the wrong disk enclosure (250Gb).
The 1TB drive still does not boot even after the gpt setting :-(
Again sorry for giving false hope.
What else can we try?

schlomo commented at 2017-08-04 14:53:¶

@deonlinux I think you should try to narrow down the issue that you see.

Can you boot the 1TB device on any other system? (try the mdos, gpt and EFI if you have any)
Can you create a bootable disk without ReaR and without using LVM, preferably with a similar disk layout at ReaR chooses?
Does your BIOS offer different ways of accessing the disk (USB hard disk, USB floppy), does any of that work?
Would the ReaR boot disk boot over another connection like SATA? That will also give you a much higher performance.
Is there a BIOS update available? Does it help?
Do you have an USB2 harddisk enclosure to try? Google seems to show a lot of hits on "USB3 boot issues"

Just some ideas of what I would be trying. BTW, restoring 1TB over USB2 will take a while...

If this is urgent and you want somebody to replicate your environment or work with your hands-on then please consider commercial support

deonlinux commented at 2017-08-07 13:28:¶

Hi Schlomo. I did some further tests:

Using an enclosure from another manufacturer (Coltech, USB3) I get the same results with the 1TB drive. Tried gpt as well, same result.
I am trying to locate a USB2 enclosure but it seems they are quite scarce here now.
I tried installing Mint 8.1 on the drive. Mint complains with the following message: "The partition /dev/sda1 assigned to /boot starts at an offset of 3584 bytes from the minimum alignment on the disk which will lead to very poor performance." It suggested I go back and create a new one which will correct this. Unfortunately it does not and whatever I try I cannot move beyond this.
I tried with Ubuntu server 16.04. It does not even detect the USB drive as a target to install to. Perhaps it does not allow it, never tried Ubuntu in this fashion.
Gparted interestingly says the first 8 Mb is not allocated (sectors 0 to 16383).
Placing the 1TB drive in the system using the SATA interface the device boots.

So, unless I locate an USB2 enclosure I cannot conclude that it is USB3 related. Despite all the technical docs on these enclosures saying the will work with up to 3TB drives!
All is not lost. We can swop the drives for 500Gb ones and continue. Most sites data would fit on it.

I will report back if I have some more news.

schlomo commented at 2017-08-07 14:28:¶

If it works with a SATA connection then I would suspect the BIOS in your server. Can you use SATA instead of USB for your production use case? If the server has an eSATA connector then I would even recommend that (although you might have to invest some effort into ensuring that the recovery happens to the internal disk and not the external one) for performance reasons.

The alignment errors are likely the result of different tools (parted, sfdisk ...) used by different distros.

Finally, could it be that you happen to have a 4k drive?

deonlinux commented at 2017-08-09 06:19:¶

Yes indeed the Seagate Baracuda 1TB does have 4k sectors.
I will attempt to reformat the drive manually and see if that fixes it. We can use SATA directly but that would entail sending a techie out to the customer site opening the servers and involve downtime of at least 8 hours. Something I was hoping to avoid.
Today is a public holiday for us so will only test this on Thursday.

schlomo commented at 2017-08-09 07:43:¶

Wow, first time ever that I hear about someone with such a drive. Would be interesting to see if the problem goes away if you find a 1TB drive with 512b sectors.

If it is really the 4k issue then it might indeed be an issue at the BIOS level. Does that server also support UEFI? You might have more success using UEFI boot in such a case.

deonlinux commented at 2017-08-18 08:29:¶

I have finally managed to obtain an enclosure with a USB2 interface. .The issue persists so it is definitely related to the 4k issue. Tested the default and gpt. I am not that familiar with setting up boot structures on drives. My attempt to format the drive with gparted and then just copied the different files across did not work. Attempting to use syslinux to install to the disk fails - it says it needs a FAT partition. Perhaps there are other methods/cmds I can try? Trying to understand the rear scripts is mindblowing! Much more complex that I am used to!

deonlinux commented at 2017-08-18 08:54:¶

One more thing. The bios does support UEFI but does not offer it for the drive. I expect it does not detect the drive as being UEFI capable? The BIOS was updated but the last time Intel released a bios update for this motherboard was in 2011.