#1078 Issue closed: NTFS support (via BLOCKCLONE backup method)

Labels: enhancement, documentation, fixed / solved / done

danboid opened issue at 2016-11-22 12:45:

It would be very useful if rear supported the backup and recovery of ntfs / Windows OS partitions for people who multiboot Linux with Windows, like I do in work.

I'm only expecting rear backup and recovery to work from under Linux and I'd be happy enough if that's the only way it ever worked but now that Ubuntu officially runs under Windows (10) there is maybe scope in the future for having rear work under the Windows 10 bash shell? I'm not really bothered if that is not possible but I do feel the ability to backup and restore NTFS is a bit of a must. Windows support has been the first question I have been asked with 2 of the first 3 people I have told about rear.

I can't remember the details now but I know that if you install Windows and leave it to its own devices it creates at least one boot / recovery partition as well as the OS partition but if you manually install it it can live happily in one NTFS partition, or at least that was the case with Windows 7. I mention this as it could be we can only get this working if the user has a simple config like that ie no fancy Windows boot partitions etc.

I don't know what rear does when it backs up a multiboot Linux system (ie dual booting Debian and OpenSUSE for example) yet but if I was backing up a Linux/Windows dual-boot machine I'd be expecting two separate archives to be produced.

gozora commented at 2016-11-22 13:34:

Functionality you are describing here, could be done by simple mount of NTFS volume + backup its content (e.g. using tar). I'm not sure if ReaR is needed for such operation.

danboid commented at 2016-11-22 14:28:

The same could be said for Linux. I know how to manually back up and restore Linux and Windows but rear is supposed to make the process easier and that's why I'm interested. More people will be interested in rear if it handles more use cases, the major missing one being Windows.

gozora commented at 2016-11-22 15:44:

So what you are proposing is "just" a NTFS support in ReaR?

danboid commented at 2016-11-22 15:55:

Yes, or I'd like to know why it has been decided not to support NTFS / Windows?

As it stands, I can only recommend rear to people who only use Linux but I'd like to be able to recommend it to people who multiboot with Windows too. As it stands I'd have to tell them they have to stick with Clonezilla unless they want to backup Windows separately and at that point you may as well do both with Clonezilla or fsarchiver or whatever.

It's not as easy to use as a pre-configured rear setup but at least Clonezilla handles Windows as well as Linux.

jsmeix commented at 2016-11-22 16:00:

Let's keep
"having rear work under the Windows 10 bash shell"
out of this issue and focus on
"backup and restore NTFS".

In general regarding backup and restore see
and therein see in particular the initial description
that reads (excerpt):

By design ReaR operates on the filesystem level.
That means that anything not mounted
is out of scope for ReaR.

The design goal of ReaR was to take a
backup / layout snapshot of a running system

Regarding backup of files on a NTFS filesystem:

It should "just work" to mount the NTFS filesystem
somewhere and mention that "somewhere" in

I think the current missing part is the restore of
files on a NTFS filesystem during "rear recover".

To do that one needs to:

"Simply" create a NTFS filesystem under Linux and
mount it so that one can "just" write (i.e. restore) files there.

gozora commented at 2016-11-22 16:06:

Hello @jsmeix
Reading code in usr/share/rear/layout/save/GNU/Linux/230_filesystem_layout.sh:

# Comma separated list of filesystems that is used for "mount/findmnt -t <list,of,filesystems>" below:
# Read filesystem information from the system by default using the traditional mount command
# limited to only the supported filesystems which results output lines of the form
#   device mountpoint filesystem (list,of,options)
# for example
#   /dev/sda2 / btrfs (rw,relatime,space_cache)
#   /dev/sda2 /.snapshots btrfs (rw,relatime,space_cache)
#   /dev/sda2 /var/tmp btrfs (rw,relatime,space_cache)
read_filesystems_command="mount -t $supported_filesystems | cut -d ' ' -f 1,3,5,6"

Isn't this a showstopper for NTFS?

gozora commented at 2016-11-22 16:27:

I unfortunately don't know exact reasons behind not including NTFS, but I'm personally not brave enough to read NTFS partitions directly from Linux (mainly because I don't like messing up with anything related to Windows ;-) ).
I can however take this challenge and try to recover WIndows with ReaR , after all it might be fun ...

jsmeix commented at 2016-11-22 16:35:

I think the code in
is an intentional showstopper for NTFS filesystem recreation
because currently there is no such thing in ReaR.

But I guess what goes into the backup should be
independent of filesystem layout recreation code.

I.e. I think one can get any mounted (i.e. accessible) files
into the backup regardless whether or not ReaR could
recreate the matching filesystems.

But I never ever tested anything with NTFS
because I do not at all use that filesystem
(in particular I do not use Windows).

danboid commented at 2016-11-22 17:11:

Good to hear you guys aren't opposed to NTFS support although its clearly not something you have much personal interest in, not being Windows users.

It'll be interesting to hear how you get on gozora. As I said, I think you'll find it much easier to backup and restore Windows if you manually create an NTFS partition first and then install Windows into it as this circumvents the Windows boot partition. Ideally though ReaR would be able to handle backing up and restoring stock Windows (7/8/10) installs which involve more than one partition.

gozora commented at 2016-11-22 17:46:

@danboid Yeah, instant regret :-), my MSDN account looks to have some authentication trouble (most probably due lack of use), I'll need to run tests with Windows 7 ... Will keep you posted ...


danboid commented at 2016-11-22 18:37:

Thanks Vlad! Sorry to have to inflict a lesser OS on you! :)

So what does rear do when you are backing up a multiboot machine? Does it pack every distro/OS into one tarball or does each OS get archived separately? I ask because in a multiboot config I would like to have the option to only restore specific OSs/partitions instead of all or nothing. I'd expect auto recover to restore them all but hopefully manual recovery would let you choose.

gozora commented at 2016-11-22 19:57:

Lesser :-), I'd not say lesser :-), It is different and (for me) very hard to control.
But I'm certain it have some bright sites which I was unfortunately not able to discover yet.
Anyhow, I was able to recover my MSDN account so I can run tests with Windows 10 now (assuming I'll be able to install it)

gozora commented at 2016-11-22 20:02:

@danboid and to answer your question. I'd guess that all OSes will be packed to single archive.
Assuming that I'll be able to recover Windows with ReaR during my tests, and we have at least small chance that something like this is possible, we can start discussion what would be the best way to implement it. Until than it is just wase of time ;-)

jsmeix commented at 2016-11-23 08:43:

Current Relax-and-Recover does not support
more than one single operating system, namely
the currently running operating system, cf.
I never tried but I assume current Relax-and-Recover
does not support two Linux systems on one harddrive.

Current Relax-and-Recover does not support
more than one single backup method, cf.
and only one single backup archive type, cf.
with usually only one single backup archive (e.g. for 'tar'), cf.
where the latter depends on the backup method, cf.

I have no personal experience to what extent mkfs.ntfs
actually works to make a NTFS filesystem so that it
actually works with current Windows versions.

I have no personal experience to what extent usual
Linux backup programy like 'tar' actually work to
backup and restore files on a NTFS filesystem so that
after restore it actually works again to be used by
current Windows versions.

jsmeix commented at 2016-11-23 08:46:

I would like to get some comments from you
what you think in general about adding
NTFS support in ReaR.

gozora commented at 2016-11-23 09:00:

I had a small talk with my NT colleagues this morning, and so far it looks like there might be a problem with permissions and unmovable files. I'll however continue with test today afternoon. (at least for my personal curiosity).

danboid commented at 2016-11-23 09:13:

Thanks for highlighting all the obstacles between the current state of rear and what would be required to get individual OSs recovered jsmeix - seems there is quite a lot to do to get to that stage.

In the past I've successfully used Clonezilla, ntfsclone and fsarchiver to backup and restore NTFS partitions (mainly Windows 7 and XP) under Linux but I understand the need to reduce deps and I've not tried cloning Windows 10 yet.

gdha commented at 2016-11-23 09:30:

In the past (during my mkCDrec times) I first zero-ed the unused space in NTFS volumes and just used dd + compress to dump the NTFS volumes. That worked quite well, but I must admit I still believe ReaR i meant for Linux FS (to recover Linux servers).

gozora commented at 2016-11-23 09:33:

@danboid ntfsclone looks like way to go, thanks for the hint.
@gdha fully agree, Windows don't deserve to be restored!!! :-)

jsmeix commented at 2016-11-23 09:35:

I have a private secret ultimate plan
( who will attend FOSDEM 2017? ;-)
why I probably would like to implement support for
"Multiple Backup Methods" at some time in the future.

For a possible use-case for "Multiple Backup Methods" see

Now the comment of @danboid here
and the commenf of @gdha here
may show another possible use-case for "Multiple Backup Methods":

Use a generic backup method (e.g. 'tar') to backup and restore
files on usual Linux filesystems and use another specific backup
method to backup and restore a Windows NTFS filesystem.

jsmeix commented at 2016-11-23 09:41:

I also agree that Windows NTFS filesystems
should just be dumped ( into the dumping ground ;-)

danboid commented at 2016-11-23 09:41:


I forgot to mention partclone.ntfs. I think thats got more features than ntfsclone. IIRC partclone is what clonezilla uses. The advantage fsarchiver has over these is that it can restore images onto smaller target partitions than they were cloned from, provided there is enough space for the files.

gozora commented at 2016-11-23 09:49:

@danboid that one look interesting as well, but I'll try ntfsclone first, as it looks to be part of ntfs-3g package.

jsmeix commented at 2016-11-23 09:50:

@gdha regarding your
"zero-ed the unused space ... and just used dd + compress"

Perhaps this might be a generic way to backup a whole
btrfs filesystem that contains snapshot subvolumes
so that after restore the btrfs snapshots are still
real btrfs snapshots and not duplicated files, cf.
that reads (excerpt and bottom line):
"btrfs snapshot subvolumes cannot be backed up and restored
as usual with file-based backup software"

gozora commented at 2016-11-23 10:08:

@jsmeix but that would mean that we avoid user to restore a single file/directory from archive in simple way.

jsmeix commented at 2016-11-23 10:20:

One cannot have all:
Unusual filesystems plus all features of usual filesystems.

In general using 'dd' for disk partitions:
This could be a nice generic way how ReaR could
backup and restore any "foreign" filesystems
(even filesystems that cannot be mounted under Linux).

Of course on a multi-boot system "rear recover" can probably
not recover "foreign" bootloaders (except all systems are booted
by a Linux bootloader that is supported by ReaR - i.e. Grub).
But at least after "rear recover" the user would have all his files back.
Hopefully he can boot his "foreign" systems manually via
appropriate "foreign" external boot media.

danboid commented at 2016-11-24 11:35:

Sorry for being OT but I've been wondering about two things related to rear I hope one of you will kindly answer:

  • Can I use the same USB drive, once it has been formatted and used as a rear recovery drive for one host, to recover multiple different host or is it one recovery drive per host?

  • If I lose / accidentally format or whatever my recovery stick whilst I still have the backup on a network share but I don't have a working machine just like the one I'm attempting to recover currently running to do a rear mkrescue, is there a way to create a working recovery USB drive or ISO that would be capable of restoring such a rear backup using any old Linux box?

I'm asking these question here because I seem to be having trouble subscribing to your mailing list you don't seem to have an IRC channel.


jsmeix commented at 2016-11-24 13:19:

in any case use new separated GitHub issues
for new separated isuses and/or questions.
Otherwise things get messed up and finally
all may end in the middle of nowhere.

jsmeix commented at 2016-11-24 13:26:

I asked colleagues about their experience with
ntfsprogs (e.g. mkfs.ntfs) in particular
if it actually works reliably in practice.

I got only very little response.
I assume only a few use NTFS filesystems.

So far what I got as replies:

It works fine for me, and has done so for some time.

The only caveat I have for ntfs-3g is that it's
handling of 'unclean' NTFS partitions is
very conservative, probably with good

It will not allow RW mounting of an unclean
ntfs filesystem, which means you might
need to use a Windows machine to run
a filesystem check before you can use
it fully again with ntfs-3g.

So I wouldn't use ntfs as a filesystem
of choice, but it no longer gets in my way
when I need to use it.

plus a subsequent reply in particular regarding
handling of 'unclean' NTFS partitions:

This causes small problem with dualboot
machine from Win8+.
Default powerof  for this systems
are Hybrid Sleep and NTFS partitions
are everytime flagged as unclean + some
data can be wrongly journaled. 
So this feature needs to be disabled on
dualboot systems with shared NTFS partition.

danboid commented at 2016-11-24 14:32:

The unclean mounting through having hybrid sleep / Fast Startup enabled also applies to Windows 10 as well as W8. Fast Startup is enabled by default but its easy to disable:

  1. Open Control Panel in the small icons view and click on Power Options.
  2. Click on Choose what the power buttons do.
  3. Click on Change settings that are currently unavailable.
  4. Uncheck Turn on fast startup (recommended).

If you'd prefer to answer my questions via email I'd be happy with that. I think opening a ticket for a couple of small questions seems like overkill.

gozora commented at 2016-11-24 17:59:

Just a short ping that I'm still alive and working hard on this topic. Unfortunately I don't have so much spare time as I would like and things are progressing quite slow as Windows is not so happy with (unintentional) damages I'm causing. I never thought how fragile it is when handled from inside Linux.
I was however able to use ntfsclone to make working copy of my Windows 10.
The question if and how it should be implemented into ReaR is misery to me.

@danboid @jsmeix thanks for your inputs on NTFS!

@jsmeix do you want to share with me your private secret ultimate plan? Maybe that could put me to right direction


jsmeix commented at 2016-11-24 18:10:

I was wondering all the time why nobody asked me
what my "private secret ultimate plan" is regardless
that I tried so much to make it look irresistibly interesting ;-)

But today is a bit too late for such a great announcement
so that you must be patient until tomorrow (and sleep well ;-)

Hint: It is related to

jsmeix commented at 2016-11-24 18:17:

only an untested idea regarding experiments with NTFS:
Perhaps for the very first experiments with NTFS
you can leave out Windows and use it only on Linux,
i.e. create a NTFS with mkfs.ntfs, mount it, put some
files therein, and enhance ReaR to work with that
(first all only on Linux). When this works, one can
try out how far it also works with a NTFS filesystem
that was created and is used by Windows.

gozora commented at 2016-11-24 18:22:

@jsmeix Yep, that is stage 3 of my tests (stage 1 install windows :-) and stage 2 restore it without ReaR intervention just to get some first slaps from this technology).
First problem I need to solve is that ntfs does not identify as NTFS but as fuseblk (which I guess covers all possible fuse file systems)

schlomo commented at 2016-11-24 19:04:

@jsmeix my thoughts are the usual: Pull requests are welcome as long as they don't break ReaR for non-NTFS users. Changing the ReaR architecture to accommodate the new features will require a lost of discussions. The goal of ReaR is to be the best DR solution for Linux users, whatever their use case might be.

That aside, I would strongly recommend to integrate another backup tool like clonezilla into ReaR and to have that tool take care of all the non-Linux stuff. AFAIK that already even handles partition resizing and other useful things that you probably don't want to invent again.

danboid commented at 2016-11-25 08:10:

Thanks for the update Vlad - its good to hear you have cloned W10 OK with ntfsclone. That's a solid start but I'd like to remind you all that I would be hoping for the ability to restore partitions onto smaller ones that have enough space to hold the files which, last I checked, is a feature that fsarchiver allows that the other tools do not. IIRC fsarchiver is written by the same guy who wrote partclone, which is what CZ uses hence this is a limitation of CZ I'd like to not inherit.

If ntfsclone can do this now then great!

jsmeix commented at 2016-11-25 09:50:

Regarding my "private secret ultimate plan", see

jsmeix commented at 2016-11-25 10:04:

@schlomo regarding your

In general "don't break ReaR" is mandatory, cf.
"Maintain backward compatibility" in

"I would strongly recommend to integrate another backup tool
like clonezilla into ReaR and to have that tool take care
of all the non-Linux stuff":

I do very much appreciate this!

When ReaR should also support non-Linux stuff
via other backup tools, it means support for
"Multiple Backup Methods" is a precondition
so that users can use one backup tool for their
Linux stuff (e.g. 'tar') and additionally another
backup tool (e.g. Clonezilla or even dumb 'dd')
for their non-Linux stuff.

For my "private secret ultimate plan" I also need
support for "Multiple Backup Methods" so that
users can use one kind of "backup" tool like 'zypper'
to get RPM package installed and additionally another
backup tool like 'tar' to get afterwards config files
and/or whatever data files installed so that the
final result is a ready-to-use system.

Therefore I will try to implement support for
"Multiple Backup Methods" in ReaR.
But currently I have no good idea how I could cleanly
implement that without changing too much in ReaR.

gdha commented at 2016-11-25 10:17:

We may not forget ReaR is used by thousands of users in Linux production environments, which do not care about Windows (as these run on separate servers or VMs). Therefore, I strongly would avoid backup/restore of NTFS file systems as we cannot guarantee the data integrity at all (it would only be a best effort).
Believe me - I've seen it before with mkCDrec - and I would like to avoid that history repeats ifself.

jsmeix commented at 2016-11-25 10:29:

I fully agree that we must first and foremost care
about current ReaR users and their use cases.

On the other hand NTFS support might be also
of interest for business production environments
in particular for company laptops and workstations
with Linux/Windows dual-boot installations?
But I have zero personal experience with such use-cases.

gozora commented at 2016-11-25 11:48:

Hello guys,
I don't think that anybody here really wishes to break current ReaR code!
I however think that @jsmeix have a good idea with support of multiple backup methods. If this is done carefully, it would be good start for implementation of alien file systems like NTFS (or anything else Linux can handle). Of course it can go either way, but I guess it is worth to try it.So if you need some help on this topic, count me in!


Believe me - I've seen it before with mkCDrec - and I would like to avoid that history repeats ifself.

I'd really like to know more about this, maybe on FOSDEM?


jsmeix commented at 2016-11-25 13:39:

I also think it is worth to at least try to go this way in general.

If we find out one cannot guarantee data integrity for NTFS,
that would be only bad luck for one particular alien filesystem
but it would not make the whole idea behind invalid.

I look forward to seeing you at FOSDEM 2017!

gozora commented at 2016-11-25 15:19:

Hello @danboid,
I basically know what I've needed to know about restoring WIndows (NTFS) under Linux.
Currently it looks like (at least from my point of view) that we can't put something like this into ReaR without lots of changes which could potentially affect lots of users which are running highly production Linux environments (as mentioned by @gdha in comment with which I fully agree btw.).

"Multiple Backup Methods" mentioned by @jsmeix could be a way how NTFS could find its place in ReaR. As mentioned earlier, it can be a dead end, but it is worth a try.


gozora commented at 2016-11-25 16:04:

@schlomo I just had a quick look on clonezilla (I never used i before), and at first glace it looks like it can't make backups of running system. You need to boot clonezilla live medium in order to make magic happen.
So inmplementing clonezilla into ReaR would basically mean "just" putting all the clonezilla binaries into ReaR rescue/recover medium. Which can be easily done by user with help of arrays REQUIRED_PROGS or PROGS.

danboid commented at 2016-11-26 08:40:

I hope an easy-ish way to implement multiple backup methods is found then.

I get the impression this will be some way off being implemented, maybe a year or two, if it happens at all.

schlomo commented at 2016-11-26 16:38:

@gozora you precisely describe the situation with regard to ReaR and non-Linux stuff. Obviously the non-Linux stuff is not active while Linux runs. So using tools from clonezilla should work perfectly well.

You are right that clonezilla does not take backups of running systems which is exactly why ReaR (and previously mkCDrec) was invented. ReaR takes care of the running Linux and out-tasks everything else to clonezilla or another tool.

@gdha you are right with your experience. I would hope that ReaR's modular design is sufficiently robust to give existing users stable use while still allowing new use cases to be added.

gozora commented at 2016-11-27 09:53:

Hello @danboid
"easy-ish way" is very relative. We need to look here for a compromise (which is at least for me quite a challenge) between not breaking current ReaR mission and implementing something new. Once I have idea how to do this, coding would be piece of cake.
I'm not abandoning your idea and still continue working on it, because as I stated couple of times, it is worth trying.


gozora commented at 2016-12-01 20:48:

Hello @danboid,

I've tested fsarchiver as you suggested, but somehow I don't have a good feeling about making backups of NTFS on filesystem level. The thing is that I was able to restore my Windows 10 with fsarchiver, I've however stumbled across hundreds of messages like follows during backup:

[errno=31, Too many links]: oper_save.c#349,createar_item_winattr():winattr:lgetxattr(/Windows/WinSxS/x86_microsoft-windows-i..mentation-migration_31bf3856ad364e35_11.0.10240.16384_none_facb7050c43c1570/WininetPlugin.dll,system.ntfs_dos_name): returned negative attribute size

According to my quick research, this problem have something to do with hardlinks and old DOS names. The reason why you didn't had trouble so far was that you've backed up Win7.
I've found following thread describing all this in more depth.
A quote from mentioned discussion:

WinXP did not use hard links, so there should be no problem. 
Win7 has begun to use hard links gradually, 
and Win8 uses a lot of hard links.

and another even more relevant

DOS names are an obsolescent feature, 
only useful for running very old applications,
but the major issue is that fsarchiver does not 
restore files which have dos name errors.

So as a result I was able to restore my OS and it even booted, I was however missing ~800 files. This proves to me that fsarchiver should not be used for OS backup/restores and if ReaR have to be anyhow involved in NTFS backups/restores, I'd much rather avoid interaction with filesystem layer and use ntfsclone.

Any feedback is more then welcome ;-)


gdha commented at 2016-12-02 07:33:

@gozora I would avoid NTFS altogether ;=)

gozora commented at 2016-12-02 08:04:

Yes agree here @gdha, however ntfsclone looks to me just like a bit smarted brother of dd. And while NTFS will be handled on block level, we will not deal with NTFS :-)

gozora commented at 2016-12-02 08:13:

My idea to implement NTFS support would be just to introduce another backup method (e.g. NTFSCLONE), and use "Multiple backups" feature recently implemented by @jsmeix.
Doing NTFS support this way, should not have any bad effects on existing ReaR code.

jsmeix commented at 2016-12-02 08:57:

what about dumb 'dd' as a simple generic "backup" method
because I think that should work for any unmounted filesystem
(strictly speaking for any block device that is not in use)
in particular it should work for any alien filesystem
and probably even for so called "raw" devices.
Of course 'dd' is dumb and slow and whatever else,
but I think it would at least "simply work" where real
backup methods do not or cannot work.

gozora commented at 2016-12-02 09:04:

@jsmeix in general I have nothing against dd and it should work perfectly fine.
I don't care about slowness until tool is reliable. But that one thing that ntfsclone knows better is "where NTFS data ends" (that is mine: "however ntfsclone looks to me just like a bit smarted brother of dd"). Hence ntfsclone will not copy whole block device 1:1 but stops after there are no more data to backup.

gozora commented at 2016-12-02 09:08:

Hmm, thinking of it now, input parameters for dd and ntfsclone should be quite same. So maybe we could do something like BACKUP=BLOCK_CLONE with some configuration options for tool to be used, like:

BLOCK_CLONE_BIN=dd | ntfsclone

danboid commented at 2016-12-02 09:18:

Thanks for the update Vlad!

I don't have any comment regarding those errors you are getting from fsarchiver because I haven't tried it with Windows 8 or 10 yet. I'll try to have a go with it later.

Going off what you say it sounds that if you can reliably detect NTFS partitions then rear should use ntfsclone and maybe use dd as a NTFS fallback and for alien filesystems.

jsmeix commented at 2016-12-02 10:14:

I look forward to your pull request :-)

I think the user needs to specify the block device
he wants to BLOCK_CLONE, e.g. something like
the following:


BLOCK_CLONE_TOOL="dd ibs=2M obs=2M"


BLOCK_CLONE_TOOL="ntfsclone -f --ignore-fs-check"


BLOCK_CLONE_TOOL="dd ibs=8M obs=8M"

where e.g. on /dev/sda3 there is an alien filesystem,
on /dev/sdb2 there is NTFS, and /dev/sdc is a raw device.

And finally there is the "tiny complication" ;-)
that ReaR must do the partitioning, i.e.
ReaR would have to create at least the right
block devices (i.e. partitions) for the alien filesystems...

jsmeix commented at 2016-12-02 10:21:

Regarding partitioning for alien filesystems:

As an initial step let the user specify what to do
e.g. something like /etc/rear/sdb2.conf

BLOCK_CLONE_DEVICE_SETUP="parted -s /dev/sdb mklabel ... ; parted -s /dev/sdb mkpart ...  ; parted -s /dev/sdb set ... type ... ; ..."
BLOCK_CLONE_TOOL="ntfsclone -f --ignore-fs-check"

and later implement the automatism.

jsmeix commented at 2016-12-02 10:28:

I have the gut feeling that partitioning for alien filesystems
should not be done with the current ReaR layout code
but kept strictly separated from it.

I think during "rear -C base_system recover"
nothing about alien stuff should happen.

I think any alien stuff should happen strictly separated
from recreating the Linux base system.

Simply put:
I would prefer that any alien stuff can never ever cause issues
for recreating regular Linux systems.

gozora commented at 2016-12-02 11:08:

hello @jsmeix exactly this can't be done with parted.
WIndows used MBR code that is hard to recreate. To my opinion best way is just to copy MBR 1:1 like:
dd if=/dev/sdb of=/dev/sdc bs=512 count=1
This will copy bootloader code + create partitions.

gozora commented at 2016-12-02 11:09:

And of course this partition code will be part of BLOCK_CLONE so indeed that should not cause any trouble.

jsmeix commented at 2016-12-02 11:31:

I humbly apologize for my uninformed example
but I am just a poor boy who cannot afford to pay
for non free 'stuff' (I have another word in my mind ;-)
that does not provide what free men need ;-)

I look even more forward to your pull request!
This could be a very interesting template case for the future
how one could cleanly provide support even for problematic
alien stuff in ReaR.

gozora commented at 2016-12-04 16:08:

Well I'm poor as well, but some time ago I've exchanged my soul for MSDN membership :-).
This is what I have so far: https://github.com/gozora/rear/commit/1db133f0a74bf3094f5196fe874543c50d3bed5a
It is surprisingly small amount of code compared how much time it took me to make it work :-).

I want to use existing ReaR code used for NETFS backups (at least for mounting and unmounting) without creating even a minimal chance for breaking something that works. So far this is just quick and dirty as I needed to get mount functionality on board as quick as possible, so I'm fully aware that some protos might not even work (so far I've done tests only with nfs://) so I rely on comments from you guys about possible implications of such approach ;-)

I've used following MO for Ubuntu Mate / Windows 10 dual boot:

  • This is most simple scenario when every OS is located on single physical HDD

  • HDD with Windows 10 have 2 partitions (1. boot; 2. c:\)

  • 3 config files in /etc/rear:

root@mate:/etc/rear# cat base_os.conf

root@mate:/etc/rear# cat windows_boot.conf


root@mate:/etc/rear# cat windows_data.conf 



  1. Backup Ubuntu Mate
    rear -C base_os mkbackup

  2. Backup Windows 10 partitions
    rear -C windows_boot mkbackuponly
    rear -C windows_data mkbackuponly

Restore part (which does not exist currently) could look like:
rear -C base_os recover
rear -C windows_boot restoreonly
rear -C windows_data restoreonly

jsmeix commented at 2016-12-05 09:15:

I am deeply impressed!

Dear "MSDN member": Vade retro satana!
cf. https://en.wikipedia.org/wiki/Vade_retro_satana

I do appreciate it very much that you know about the
Windows environment because I know basically nothing
in this area.

This week I will commit enhancements of the logging
cf. https://github.com/rear/rear/issues/1096
that is needed to make output useful when running
multiple backups/restores in parallel and I will commit
support for running multiple "mkbackuponly/restoreonly"
in parallel, cf. https://github.com/rear/rear/issues/1088

jsmeix commented at 2016-12-05 09:25:

regarding "every OS is located on single physical HDD":

I think that explains the difference between my proposal
using parted to create a partition for Windows and your way
using 'dd' for the Windows MBR.

What I had in mind was a dual boot system on a single disk
(i.e. what I think most normal users have e.g. on their laptops)
where GRUB is used as bootloader for Linux and Windows.
In this case I think it should work to use parted to create
a partition for Windows and afterwards that partition
is filled up with a 'dd' image of Windows.

What I do not know if after such a recovery
GRUB can still boot Windows.

gozora commented at 2016-12-05 09:30:

This is something I must check, Windows have some limitations in regards of on which partition it is installed on ...
I have a foggy feeling for an example that you can't install windows on other that first partition.
In any case parted will still be useless.

gozora commented at 2016-12-05 09:33:

@danboid can you share with us what disk layout do you use?

danboid commented at 2016-12-05 10:11:

Hi Vlad

My work drive looks like this:

Disk /dev/sda: 238.5 GiB, 256060514304 bytes, 500118192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb73b7001

Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 718847 716800 350M 7 HPFS/NTFS/exFAT
/dev/sda2 718848 247420927 246702080 117.7G 7 HPFS/NTFS/exFAT
/dev/sda3 247420928 500118191 252697264 120.5G 83 Linux

sda1/2 = Windows 8
sda3 = Arch

IIRC you don't have to install Windows in the first partition but it does always replace GRUB or whatever bootloader you are using with the Windows bootloader at install time, hence why you should install Windows first then Linux or other operating systems, if you are multi-booting.

Are you saying your current alien fs rear implementation doesn't support Windows being installed on a separate drive? I'd agree that having both installed on the same HD is more common but having Windows and Linux installed on separate drives is also a common config.

gozora commented at 2016-12-05 11:16:

@danboid thanks for info, could you do one more thing for me?
dd if=/dev/sda of=/tmp/sda_mbr.img bs=446 count=1 and upload /tmp/sda_mbr.img somewhere?
I'd like to see bootstrap code area of yours.

Are you saying your current alien fs rear implementation doesn't support Windows being installed on a separate drive?

Fist, this is not "implementation", let's say it is just a "concept" :-)
Second, just opposite, I just tested situation when each OS was installed on separate disk. Having all this mess on one disk would be certainly more challenging. ;-)

danboid commented at 2016-12-05 11:45:

MBR attached


gozora commented at 2016-12-05 11:50:

Thanks @danboid,
BTW, check sizes ;-)

-rw-rw-r-- 1 sodoma sodoma 606 Dec  5 12:48 mbr.zip
-rw-r--r-- 1 sodoma sodoma 446 Dec  5 11:40 sda_mbr.img

danboid commented at 2016-12-05 11:56:

Yes, I noticed that. I was in the mood for some anti-compression ;)

jsmeix commented at 2016-12-08 09:59:

regarding your

I seem to be having trouble subscribing to your mailing list

perhaps that one is related or even the root cause?

gozora commented at 2016-12-09 19:23:

Hello all,
Just a short update on my "progress".
I've managed to restore that parody of OS, in spite of its vital resistance... (It looks to me that MS guys wants reinstall to be only option how to bring it back to life.)
The biggest problem tends to be partitioning.
Windows looks to need exactly same partitioning layout to be able to boot. ReaR however seems to create partitions on approximate basis. Small example:

Original OS

root@onepart:~# parted /dev/sda unit s print
Disk /dev/sda: 73400320s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start      End        Size       Type      File system     Flags
 1      2048s      1026047s   1024000s   primary   ntfs            boot
 2      1026048s   36386815s  35360768s  primary   ntfs
 3      36388862s  73398271s  37009410s  extended
 5      36388864s  65011711s  28622848s  logical   ext4
 6      65013760s  73398271s  8384512s   logical   linux-swap(v1)

After recovery

RESCUE onepart:~ # parted /dev/sda unit s print
Disk /dev/sda: 104857600s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start      End         Size       Type      File system     Flags
 1      4096s      1523740s    1519645s   primary   ntfs            boot
 2      1523744s   54000093s   52476350s  primary   ntfs
 3      54000096s  104857599s  50857504s  extended
 5      54000104s  96477195s   42477092s  logical   ext4
 6      96477200s  104857599s  8380400s   logical   linux-swap(v1)

I'll try to solve this problem without touching original partitioning code. So far only clumsy solutions are on my mind, hope it will get better.


gozora commented at 2016-12-09 20:55:

Hmm, I just found #492 talking something about shifting partitions when recovering to disks of different sizes. Exactly my case. I'll try to restore to same sized disk ...

gozora commented at 2016-12-10 10:53:

With same sized disks situation looks a bit better. However extended partition is not correctly created even for same sized disk. It appears that Linux simply doesn't care, but Windows (again) refused to boot.
Extended partition was create as follows by ReaR rescue/recovery media:

/dev/sda3       36388862 73400319 37011458 17.7G  f W95 Ext'd (LBA)

But it should look something like

/dev/sda3       36388862 73398271 37009410 17.7G  5 Extended

So two thing went wrong

  1. End of extended partition is shifted by 2048 sectors
  2. LBA flag was incorrectly set for extended partition

As this could affect ReaR regular (Linux restore) operations as well, I'll try to find code responsible for this and correct it. I somehow have the feeling that shift by 2048 will not by coincidence.

gozora commented at 2016-12-11 15:05:

Hello @danboid ,

Today I've managed to make successful restore in both discussed scenarios (having Windows & Linux on same and on separate disks).
I'd very much appreciate if you could give it a try. You can download code from my Github page:
git clone -b disk_clone https://github.com/gozora/rear.git
During testing, please bare in mind that code is certainly not perfect and it might happen that you will not be able to recover your OSes at all! So don't do tests on systems/data you will need ;-).
Here is my configuration that worked for me:

General settings for ReaR

root@onepart:/etc/rear# cat local.conf



Backup of LInux

root@onepart:/etc/rear# cat base_os.conf 
this_file_name=$( basename ${BASH_SOURCE[0]} )

Backup of Windows boot partition

root@onepart:/etc/rear# cat windows_boot.conf 


Backup of Windows OS partition

root@onepart:/etc/rear# cat windows_data.conf 


To backup

rear -C base_os mkbackup
rear -C windows_boot mkbackuponly
rear -C windows_data mkbackuponly

To restore

rear -C base_os recover
rear -C windows_boot restoreonly
rear -C windows_data restoreonly

jsmeix commented at 2016-12-12 09:19:

In general on "same hardware" replacement disks
(in particular same disk size, ideally same disk model
basically only the disk UUID is different)
ReaR should recreate identical partitioning.

Reasoning as far as I understand it:
In disklayout.conf ReaR stores the values in bytes.
During "rear recover" parted should be called with
that byte values so that one should get the partitions
recreated with the same byte value positions on the
replacement disk. This should result an indentically
recreated partitioning - as far as I can imagine.

Regarding "extended partition is not correctly created
even for same sized disk":
I never used ReaR with extended partitions.
Perhaps there is a bug in ReaR or in parted
or whatever else?
Regarding parted:
There is the alignment-type option for parted.
Perhaps it is parted's alignment that results that
recreated partitions can be a bit shifted?

gozora commented at 2016-12-12 09:53:

Hello @jsmeix

Honestly, I did not study layout code as there were no complaints about it and like said before, I don't want to break existing code.
To overcome this I've put following code into layout/prepare/GNU/Linux/100_include_partition_code.sh, which will (re)create partitioning on disk where Windows resides, 1:1 with original (only limitation here is that target disk must be same size or larger than original).
I know that in #1107 you've started campaign against use of sfdisk, but I think, due my recent experience, that once we use sfdisk to get data from msdos and sgdisk for gpt, all should be OK.


danboid commented at 2016-12-12 10:12:

Hi Vlad

Great news! I doubt I'll have time to test it today but I'll try it as soon as I get the chance.

I realise you say its very experimental so I just wanted to confirm:

  • In your testing, did you try totally removing all partitions (esp. the Windows ones) before attempting a recovery?

  • Should this work with both BIOS and EFI Windows installs? Maybe it makes no difference?

  • Would it be possible to backup and restore with one config file, one backup comand and one restore command? Seeing as the two Windows configs look very similar apart from the partition, it would be nice if this could be made less redundant. Maybe this is just purely prototypical and not what it'll work like when finished?

  • Which version of Windows have you been testing with? Windows 10 Home 64-bit?


jsmeix commented at 2016-12-12 10:14:

I have no problem with any kind of dirty hacks
that are needed to make things work for now
as log as it is well documented in the code
so that later it could be cleaned up if possible, cf.

As far as I know sfdisk does not work with GPT.
When you need sfdisk can you implement a test
that errors out if a GPT partitioning is there?
Something like

is_true "$BLOCKCLONE_STRICT_PARTITIONING" && [[ "$label" = "gpt" || "$label" == "gpt_sync_mbr" ]] && Error "BLOCKCLONE_STRICT_PARTITIONING does not work with GPT"

Such a test should happen during "rear mkbackup/mkrescue"
and not when it is too late for the user during "rear recover".

jsmeix commented at 2016-12-12 10:18:

baciacally I insist on keeping any alien stuff strictly
separated from recreating the Linux base system, cf.

gozora commented at 2016-12-12 11:12:

Hello @danboid,

In your testing, did you try totally removing all partitions (esp. the Windows ones) before attempting a recovery?


Should this work with both BIOS and EFI Windows installs? Maybe it makes no difference?

I was doing testing with legacy boot. ( I can add EFI code as well, should not be a problem )

Would it be possible to backup and restore with one config file, one backup comand and one restore command? Seeing as the two Windows configs look very similar apart from the partition, it would be nice if this could be made less redundant. Maybe this is just purely prototypical and not what it'll work like when finished?

You can always create your own custom shell script, which will do all work for you. And yes this is just a prototype, all fancy staff can be considered once we have basic concept working ;-)

Which version of Windows have you been testing with? Windows 10 Home 64-bit?

It was 64-bit Windows maybe Enterprise (but I'm not sure)


gozora commented at 2016-12-12 11:15:


As far as I know sfdisk does not work with GPT.
When you need sfdisk can you implement a test
that errors out if a GPT partitioning is there?
Something like

This is what I had on my mind, unfortunately once I've installed Windows it created msdos type of partitioning, so all current code is related to msdos.
I guess that Windows will use GPT once I switch for EFI, then I'll adapt code accordingly.
But first I need to know that it works for somebody else (except me) as well ;-)


danboid commented at 2016-12-14 11:05:

I'm just about to try Vlads NTFS code but it has made me wonder - is there a preferred distro for testing rear with? Preferred for me would mean its quick and easy to install both the OS and rear so ideally a bare bones distro that includes all the rear deps as standard.

gozora commented at 2016-12-14 11:12:

Hello @danboid,
I was testing with MATE, but it should work fine with any distro you like.

danboid commented at 2016-12-14 18:06:

Hi Vlad!

I'm most of the way to creating a suitable test env for this now, I think. I cloned a Win 10 VM, resized its OS partition and installed CentOS 7 (minimal) into a third (XFS) partition. CentOS seems like a good distro for testing rear because rear is already in the CentOS repos so I installed rear with yum to get the extra deps then removed it before cloning your repo and doing a make rpm and installing the rear rpm.

The only thing left to do now before I test (I think) is to put the four rear config files into place but I wanted to make sure there aren't any extra options I may need? My plan is to create a samba share on my VM host and have rear backup to/ restore from that. That should be possible, right?

I expect I'll be able to get away with using almost exactly the same config files as you used Vlad but I'm wondering why the following lines in local.conf are required?


I thought rear always backed up the MBR, or that was the impression I got seeing as I didn't use any of these options previously and GRUB recovered fine?

Are there any other options you might suggest I use given my test plan? I'm hoping I'll only have to do this once! :)

gozora commented at 2016-12-14 18:52:

Hello @danboid

My plan is to create a samba share on my VM host and have rear backup to/ restore from that. That should be possible, right?

Yes that should be fully OK, you just need to adapt following config options (in local.conf) to match your backup location.


I'm sure you can adapt them to suit your needs ;-)

I thought rear always backed up the MBR, or that was the impression I got seeing as I didn't use any of these options previously and GRUB recovered fine?

You are totally right here, ReaR does all that partitioning and savig stuff just right, for Windows however we need a bit more to make the magic happen.
You can forget about GRUB_RESCUE ( I should remove it long time ago ), but

    As Windows uses some strict boot code (see https://github.com/rear/rear/issues/1078#issuecomment-266099227), I was not able to make it boot with ReaR default partitioning code. Once you use this option, partitions will be created EXACTLY same as on original OS. This however have some limits. Biggest limit is that you must restore to same or greater disk than original. Restoring to smaller disk will simply fail with some error message (some off topic) that I've probably did not changed yet, sorry!

    Tells ReaR where to look for partitioning data. In other words you put here disk (without partition number) where is your WIndows located. If you have just one disk sharing WIndows and Linux it will be most probably /dev/sda.

The configuration is a bit clumsy but I'm working on it to make it better and maybe even simpler.

Thanks for your help with testing!


jsmeix commented at 2016-12-15 10:32:

ReaR does not backup the MBR.
During "rear mkbackup/mkrescue" it remembers
the partitioning in disklayout.conf and
during "rear recover" it recreates the partitioning
from scratch (via diskrestore.sh).
I.e. during "rear recover" one gets a new
from scratch written MBR or GPT via
"parted mklabel msdos" or "pated mklabel gpt".

For alien systems this recreating the partitioning
from scratch via parted does not always work,
cf. the above comments from @gozora and me.

danboid commented at 2016-12-15 13:12:

After jumping through hundreds of hoops, I've backed up my dual-boot Win10/CentOS 7 VM as you described and I'm currently cloning my test VM just in case recovery fails.

I presume my only way to recover is to use those three separate commands ie auto recovery currently won't auto restore all three partitions for me, yet? Manual recovery is the only option for recovering NTFS right now?

Should NTFS support get merged, would I be able to use auto recovery and have NTFS partitions auto-recovered?

Back soon with the results...

gozora commented at 2016-12-15 13:20:

Hello @danboid,

What is your idea, about how recovery should work?

danboid commented at 2016-12-15 13:26:

When I boot the recovery iso, I would expect auto-recovery to restore all partitions (on all drives) by default. Is this not the default behaviour?

gozora commented at 2016-12-15 13:32:

Honestly, I don't know if ReaR can do something like this. I saw some "automatic recover" option in ReaR rescue/recovery system but I've never used it.
I think it is a matter of taste, I for example don't think starting recovery automatically is a good idea and prefer more control over it.

I'm not saying that this is final stage how this patch will behave, but cosmetics can be done after we have something working ;-)

danboid commented at 2016-12-15 13:41:

I've just tried the automatic recovery option but, as I expected from your response, it didn't work:

ERROR: Backup archive 'backup.tar.gz' not found.

If you are recovering a multiboot system, recovering all partitions in auto-recovery mode seems like the only sensible default to me.

I'll try a manual recovery now...

gozora commented at 2016-12-15 13:55:

If you are recovering a multiboot system, recovering all partitions in auto-recovery mode seems like the only sensible default to me.

I would not agree here, Imagine that only your Linux installation is broken (Windows is running just file). In such case restoring Windows can be useless ...

danboid commented at 2016-12-15 14:02:

What should auto-recovery do then, if it doesn't restore all partitions?

IMO you should only need to do a manual recovery if you DON'T want to restore all partitions - but that's just me.

jsmeix commented at 2016-12-15 15:48:

FYI regarding "automated recovery" see

gozora commented at 2016-12-15 15:53:

@jsmeix will it work with "Multiple backup methods"?
Like I said before, I've never tried it out, but my idea is that "automatic recover" just runs rear recover after boot.
We would need rear -C base_os recover; rear -C windows_boot restoreonly; rear -C windows_data restoreonly instead.
Maybe another variable in default.conf (AUTOEXEC) would be handy ;-) ?


jsmeix commented at 2016-12-15 15:59:

I agree with you point of view in your
cf. what I wrote in

clean up the currently single-big-and-fat "recover" workflow
Currently "rear recover" is not prepared to skip only
certain parts...

will it work with "Multiple backup methods"?

As far as I understand it for NTFS backup and restore
using "Multiple backup methods" is mandatory because
the NTFS backup and restore is a separated
and totally different backup method compared to the
backup method that is used for the basic Linux system.

jsmeix commented at 2016-12-15 16:04:

@gozora I think I misunderstood your

Currently multiple backups only works as documented
in doc/user-guide/11-multiple-backups.adoc
i.e. nothing "automated" there - one must manually
specify the right rear commands.

For the future see
but that will not happen for ReaR 2.0 - but later...

gozora commented at 2016-12-15 16:18:

Good, because I've started to to think that error is in my receiver :-)
No rush though, important thing is that it is on your radar.

danboid commented at 2016-12-15 16:19:


Unfortunately, whilst both backup and recovery seemed to proceed without errors, recovery failed. Surprisingly, neither Windows nor CentOS would boot after recovery although at least GRUB seemed to get restored OK.

Whilst CentOS (7 minimal install iso) seems like a good choice for testing rear to me, its not perfect for testing NTFS support as you have to add the EPEL repo to get ntfs-3g and ntfsprogs and EPEL doesn't work until you disable SSL for its repo. Also, the COS installer didn't detect my Windows install so I had to manually add it to the GRUB config.

Another hiccup on my way to creating the backup was that I was unable to disable fast boot under Windows 10 when running it under QEMU/KVM, which prevented me cloning the Windows data partition straight away. Rather than forcing ntfsclone to ignore the unclean partition, I rebooted into Windows and then shut it down using the command:

shutdown /s /f /t 0

After booting back into CentOS, I was able to backup the Windows data partition without using the force switch.

My config local.conf looked like:




The other config files I kept identical to the Vlad's but I also left /etc/rear/os.conf in-place as it seemed harmless to me:


I will attach screenshots of the errors displayed when I try to boot Windows. Push Enter, F8 or F9 does nothing (in true Windows stylee).
screenshot at 2016-12-15
screenshot at 2016-12-15

danboid commented at 2016-12-15 16:26:

One other thing I should've mentioned - after backing up my test VM with rear and before recovery I booted systemrescuecd on my test VM and nuked its partition table using the o option in fdisk.

gozora commented at 2016-12-15 16:29:

Hello @danboid,
I know these screens!!! (maybe I should start to consider to switch to NT support :-) ). I've got exactly these screens when partitions had different parameters!

Could you please do following:

  • on your source OS
    Get partition parameters (and put them here) of your disk where Linux and Windows are installed like:
    parted /dev/sda unit s print

  • boot ReaR rescue/recovery system and restore Linux only with:
    rear -C base_os recover
    Once done trigger same command (and post result here)
    parted /dev/sda unit s print



gozora commented at 2016-12-15 16:29:

One other thing I should've mentioned - after backing up my test VM with rear and before recovery I booted systemrescuecd on my test VM and nuked its partition table using the o option in fdisk.

This should not be a problem

gozora commented at 2016-12-15 16:44:

One more question @danboid, your NTFS partition(s) were mounted or unmounted during backup?

danboid commented at 2016-12-15 16:46:

Hi Vlad

By source OS I presume you mean my pre-rear'd VM, right? (ie the clone of my test VM that still boots CentOS and Windows fine):

[root@localhost rear]# parted /dev/sda unit s print
Disk /dev/sda: 130023424s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start      End         Size       Type     File system  Flags
 1      2048s      1026047s    1024000s   primary  ntfs         boot
 2      1026048s   89001983s   87975936s  primary  ntfs
 3      89001984s  130023423s  41021440s  primary  xfs

I then booted the rear recovery iso on my test VM that doesn't boot and ran those two commands, the output of which you can see in the attached screenshot:

screenshot at 2016-12-15

danboid commented at 2016-12-15 16:50:

My NTFS partitions were unmounted during backup and Windows still fails to boot after re-recovering base_os.

gozora commented at 2016-12-15 16:53:

By source OS I presume you mean my pre-rear'd VM, right? (ie the clone of my test VM that still boots CentOS and Windows fine):


Partitioning looks good, despite my previous suspicion.
I assume that you've restored to same virtual HW as you've made backup on, is that correct?

danboid commented at 2016-12-15 16:56:

Yes, I restored on the same VM as I backed up on. The parted text output in my last but one post was took from the clone of my test VM, which still boots OK.

jsmeix commented at 2016-12-15 16:57:

only FYI regarding screenshots from a ReaR recovery system:
You may have a look at
therein the section about
"First steps with Relax-and-Recover"
and therein the part about
"run 'rear recover' from remote via ssh"
that reads (excerpts):

In /etc/rear/local.conf set


This way you can access a running ReaR recovery system
from remote via ssh and then it is usally easier to get
logs and things like that from the ReaR recovery system.

danboid commented at 2016-12-15 16:59:


You've prob noticed in my last attached screenshot that sda1 was aligned differently after the original recovery than when I restored base_os a second time. After restoring a second time, the sectors match those on my cloned VM but this didn't make it boot.

danboid commented at 2016-12-15 17:12:


Thanks, I'll be sure to add that into my config next time!

gozora commented at 2016-12-15 17:14:

@danboid this just hit me in the head, you wrote:

Unfortunately, whilst both backup and recovery seemed to proceed without errors, recovery failed. Surprisingly, neither Windows nor CentOS would boot after recovery.

I'm interested in unbootable Centos, do you know what was the problem there? Did you saw at least Grub2 menu?

gozora commented at 2016-12-15 17:25:

You've prob noticed in my last attached screenshot that sda1 was aligned differently after the original recovery than when I restored base_os a second time. After restoring a second time, the sectors match those on my cloned VM but this didn't make it boot.

I must be blind or something ..
Your OS where you've run rear -C base_os mkbackup have this partition layout:

Number  Start      End         Size       Type     File system  Flags
 1      2048s      1026047s    1024000s   primary  ntfs         boot
 2      1026048s   89001983s   87975936s  primary  ntfs
 3      89001984s  130023423s  41021440s  primary  xfs

and same numbers are shown on your screenshot after you've done rear -C base_os recover

danboid commented at 2016-12-15 17:37:

Yes, I see the GRUB2 menu fine but all 3 Centos options fail to boot. The CentOS progress bar reaches the top, turns white and then booting stops.

If I edit the GRUB entries for CentOS to remove rhgb quiet so that it shows the boot messages you can see it only gets as far as loading Network Manager before it gives up.

screenshot at 2016-12-15

danboid commented at 2016-12-15 17:40:


Look near the top of my last-but-one screenshot where it says "Old situation:" and you'll see sda1 was aligned differently after the first restore.

I'm off home now so any more questions will have to wait until tomorrow.

Thanks Vlad!

gozora commented at 2016-12-15 17:59:

Look near the top of my last-but-one screenshot where it says "Old situation:" and you'll see sda1 was aligned differently after the first restore.

That is fully OK, that is effect of BLOCKCLONE_STRICT_PARTITIONING. I was describing this earlier. ReaR does not create exactly same partitioning with original. And in order not to break current code, I just let let ReaR to run original partitioning code and once it is finished, I just rewrite it again so it matches original. (this might change in the future).

I'm a bit concerned with your last screenshot. In general Linux should be able to boot regardless if you try to restore windows or not. This might be some problem not related to NTFS topic. I'll try to reproduce it somehow. :-).

Just for summary, can you please describe exact commands you've used for backup and for restore + add content of files in /etc/rear directory?



gozora commented at 2016-12-15 18:26:

@danboid , one more idea. maybe you could do this test with some more "pop" distro ;-). Something like Ubuntu, Fedora, OpenSuSE ?

danboid commented at 2016-12-15 19:13:

I'll attach my exact config files tomorrow but I believe I've already given you enough info for you to recreate my rear config exactly - at least as far as the rear confg files are concerned.

I had the same thoughts as you wrt Linux not booting. Does rear do any kind of checksumming for creating the backup archives? I know clonezilla asks if you want to perform checks before and after you clone when you use its advanced mode. Maybe I can force rear to do checksums with an additional option? As I said, no errors were reported during the backup process for all three partitions, same goes for when I was restoring them.

CentOS, probably because its the most popular RHEL derivative, is THE most popular (Linux or Windows) server OS in the UK, by a long shot so I suspect its one of the most popular server OSs worldwide alongside Debian, RHEL and Ubuntu. CentOS pretty much is Linux as far as the datacentres in the UK are concerned. rear HAS to work with CentOS, if nothing else.

It would be interesting to try it with CentOS 6 too as that prob has more installations than C7 presently.

gozora commented at 2016-12-15 19:37:

I'll attach my exact config files tomorrow but I believe I've already given you enough info for you to recreate my rear config exactly - at least as far as the rear confg files are concerned.

Wrong, you never can have enough info ;-)

I'm not suspecting that you had corrupted archive. But that something else went wrong.
I have my dualboot Centos 7 up and running now, and I'll try to reproduce soon.

ReaR is working just fine with Centos, I think that you can try it even with Centos 5 ;-)

danboid commented at 2016-12-15 20:18:

Some other things I should mention:

I installed CentOS 7 minimal x86-64 (64-bit) to a single XFS partition with no swap and no boot partition. The installer does nag at you for not creating a swap partition but I've been swapless for years now and happy! I realise this isn't always an option such as people who use suspend to disk - they need swap.

I used gparted on systemrescuecd to shrink my Windows data partition to make way for the CentOS partition. Prob doesn't matter but...

I used the virtio network adapter under QEMU for my test VM. I gave it 2GB RAM but other than that, it is virt-manager defaults for the VM hardware.

My samba server was running on the same (Arch Linux) box as the VM but I still had to punch holes in the firewalld config for smb to work. Again, this is likely unrelated but worth noting if you wanted to replicate my testing method.

I made sure I installed ntfs-3g, ntfsprogs and all the usual samba packages under C7.

The KVM VMs are stored in qcow2 files.

I used the C7 installer defaults as far as filesystem options are concerned.

I can't think of anything else worth mentioning right now...

danboid commented at 2016-12-15 20:44:

I did a full yum update after installing C7, rebooted the VM then installed rear, added the EPEL repo and the aforemtioned extra deps of samba and ntfsprogs etc.

gozora commented at 2016-12-15 20:49:

I think I have all I needed. After installation of Centos7 and Windows10 I'm in same situation as you are.
Neither Linux nor Windows are booting.
For Linux I'm suspecting selinux, and Windows heaven knows ...

I'll update you once I know more.


danboid commented at 2016-12-16 00:27:

Good to hear its not just me then!

Thanks V!

jsmeix commented at 2016-12-16 08:36:

only a side note FYI:
If you ever test with openSUSE do yourself and me a favour and
do not use SUSE's complicated default btrfs structure for your
test systems - unless you like to test especially btrfs issues, cf.

danboid commented at 2016-12-16 09:14:


I presume you are mainly (only) a Suse user? Do you work for them? I've not tried (open)suse in years so its time I gave it another go really.

I decided some time ago to never use BTRFS. It's just a mess. I run FreeBSD 11 on my home server because FreeBSD has more mature ZFS support than Linux. I'd like to see a Linux distro that supports ZFS as well as FreeBSD does but licensing makes that tricky or impossible.

I prefer to use XFS under Linux but in some cases ext4 is the only option or a better fit. In kinda related news Wayland is finally coming of age but I just have no hope for BTRFS. It seems fundamentally flawed and poorly designed from everything I hear about it. Maybe bcachefs will bring a good, modern filesystem to Linux that distros can actually use?

jsmeix commented at 2016-12-16 10:40:

I work for SUSE, see

I cannot recommend what filesystem(s) to use
for production systems (I am not a filesystem expert).

I only recommend what to use for testing ReaR
to avoid issues with filesystems - unless the goal is
to verity that ReaR works with particular filesystems.

In contrast testing how ReaR works with the filesystems
that are used on production systems is mandatory before
ReaR is used for disaster recovery on those systems, cf.
"No disaster recovery without testing and continuous validation"
at https://en.opensuse.org/SDB:Disaster_Recovery

To not use btrfs on openSUSE in the YaST installer's
"Suggested Partitioning" dialog click on
"Edit Proposal Settings" and then select
"Ext4" as "File System for Root Partition".

gozora commented at 2016-12-19 17:34:

Looks that I was right about Linux part. (in https://github.com/rear/rear/issues/1078#issuecomment-267440144).
Once I've disabled SELinux and rerun the backup, Centos 7 booted just fine after recovery.
I'll try to have a look on it once I have a bit of spare time.
Now I need to figure out what is wrong with Windows part ...

gozora commented at 2016-12-19 18:36:

Hello @danboid,

Hope you still remember me ;-). I was off during weekend playing Kerbal Space Program, somehow missing mood to solve problems, but now I'm back :-).

In https://github.com/rear/rear/issues/1078#issuecomment-268026797 I've found reason behind unbootable Linux. I don't have patched it yet, but if you disabled SELinux and rerun backup, your Linux should boot just fine.
For a case you are not familiar with SELinux you need to edit /etc/sysconfig/selinux as follows:

- SELINUX=enforcing
+ SELINUX=disabled


rear -C base_os  mkbackup

Now for Windows part. Once you've finished with recovering your Windows data partition try to poke a bit your MBR like this AFTER last part of NTFS restore is done:

  • while in ReaR rescue/recovery system you should have file sda_mbr.img (or simmilar) in /var/lib/rear/layout/
  • execute dd if=/var/lib/rear/layout/sda_mbr.img of=/dev/sda bs=446 bs=1 and reboot
  • now Windows should boot just fine

Hope this time it will work for you just fine. I'll of course implement this (somehow) into ReaR so user will not need any "hacking" in future.

danboid commented at 2016-12-20 09:35:

Hi Vlad

No probs about the wait! I do realise you have a life outside of developing ReaR! I've not played Kerbal Space but one of my co-workers at my last job was a fan of that.

As for the CentOS boot prob with SELinux, it seemed obvious when you suggested it. I was surprised I must've never tried cloning a box that had SELinux enabled before as otherwise I would've disabled it, which is something I normally do on a COS/RHEL install I'm actually using.

More importantly for this ticket, I just ran:

dd if=/var/lib/rear/layout/sda_mbr.img of=/dev/sda bs=446 bs=1

On my rear test VM and as you say, Windows 10 boots fine now! Yippee!

Good work Mr G!

gozora commented at 2016-12-26 16:46:

Hello @danboid,

I've updated my ReaR fork, with patched code according latest findings we've made.
I've also made successful test with restoring Windows 7, so the basic idea looks to be working.
Now I'll take care about cloning of disks using dd. Hope it will be same fun ...

Happy holidays everybody!

danboid commented at 2016-12-28 10:21:

Thanks for the update Vlad!

Happy holidays to you too!

gozora commented at 2017-01-19 09:15:

Pull request https://github.com/rear/rear/pull/1172 was created with initial BLOCKCLONE code

danboid commented at 2017-01-19 09:41:

Thanks for the update Vlad!

Has anyone else tried the ntfs-cloning yet apart from me and you?

Do you plan to streamline or improve it any more or is it pretty much done now?

gozora commented at 2017-01-19 09:50:

Hello @danboid,

As far as I know you are that only privileged tester so far (apart from me ;-) ).

No it is not yet done, I'll try to add at least GPT/UEFI support. Except that, I think that anybody can write improvements to code so only time will show how all this would develop.


gozora commented at 2017-01-26 16:19:

Hello @danboid,

Documentation for BLOCKCLONE backup method was merged today to upstream code.
As only known user who actually tested BLOCKCLONE, I'd appreciate if you could have a look on documentation and tell me your thoughts on it.
I've tried to write is as easy as possible, but it is quite hard to keep a distance especially if you are documenting your own code.



jsmeix commented at 2017-01-26 16:24:

"BLOCKCLONE backup method"
"BLOCKCLONE documentation"
there is now NTFS support in ReaR
so that I consider this issue to be solved.

Again @gozora contributed a very valuable part to ReaR.
Many thanks for that!

now it is your turn to test if the
BLOCKCLONE backup method
also works for you.
In case of issues:
For each separated issue a separated GitHub issue
should be created to keep separated things separated.

jsmeix commented at 2017-01-26 16:34:

for using the BLOCKCLONE backup method
a prerequirement is understanding how
"Using Multiple Backups for Relax-and-Recover"
works, see

danboid commented at 2017-01-27 09:21:

Great to see NTFS support for rear! Thanks for all your hard work @gozora ! I will never actually use BLOCKCLONE mode for NTFS but I understand the need for a fallback.

I will try to give blockclone mode a test soon but I'm short of free time now until I get a new place to live sorted. I got told last weekend I have to move out of my house by the end of Feb.

[Export of Github issue for rear/rear.]