#1019 Issue closed: MANUAL_INCLUDE=YES behaves unexpectedly

Labels: bug, support / question, fixed / solved / done

John-Leone opened issue at 2016-09-30 23:23:

Relax-and-Recover (rear) Issue Template

Please fill in the following items before submitting a new issue:

  • rear version (/usr/sbin/rear -V):
    Relax-and-Recover 1.17.2 / Git

  • OS version (cat /etc/rear/os.conf or lsb_release -a):
    Red Hat Enterprise Linux Server release 6.7 (Santiago)

  • rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):

    OUTPUT=ISO
OUTPUT_URL=nfs://xxx.xxx.xxx.xxx/dbar
BACKUP=NETFS
BACKUP_URL=nfs://xxx.xxx.xxx.xxx/dbar
SSH_ROOT_PASSWORD="redhat"
BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/media' '/var/tmp' '/var/crash')
MANUAL_INCLUDE=YES
BACKUP_PROG_INCLUDE=("${BACKUP_PROG_INCLUDE[@]}"'/boot' '/' '/opt' '/midrange' '/tmp' '/var' )
NETFS_KEEP_OLD_BACKUP_COPY=1

  • Brief description of the issue
    Trying to configure MANUAL_INCLUDE mode in the local.conf file to specify only the file systems that we want backed up for a server. For the test we did not include /home but after we recover the server /home is there. See below, when running mkbackup we see a message "Excluding mountpoint /home (MANUAL_INCLUDE mode)"

Am I missing something on the local.conf file?

  • Log Output 
        [root@xsmithm1d ~]# rear -v mkbackup
    Relax-and-Recover 1.17.2 / Git
    Using log file: /var/log/rear/rear-xsmithm1d.log
    Creating disk layout
    Excluding mountpoint /home. (MANUAL_INCLUDE mode)
    Creating root filesystem layout
    Copying files and directories
    Copying binaries and libraries
    Copying kernel modules
    Creating initramfs
    Making ISO image
    Wrote ISO image: /var/lib/rear/output/rear-xsmithm1d.iso (70M)
    Copying resulting files to nfs location
    Encrypting disabled
    Creating tar archive '/tmp/rear.y56DIIOlJAyFmCI/outputfs/xsmithm1d/backup.tar.gz'
    Archived 2187 MiB [avg 8683 KiB/sec]OK
    Archived 2187 MiB in 259 seconds [avg 8650 KiB/sec]

jsmeix commented at 2016-10-04 10:11:

I never used MANUAL_INCLUDE=YES before
and for me it behaves in a totally unexpected way.

From my current point of view MANUAL_INCLUDE=YES
behaves totally wrong.

What indicates that the current MANUAL_INCLUDE=YES
implementation is at least incomplete is that the only script
that contains MANUAL_INCLUDE is
usr/share/rear/layout/save/default/31_include_exclude.sh
which is about "Include/Exclude components"
(i.e. things in var/lib/rear/layout/disklayout.conf)
but there is noting at all about MANUAL_INCLUDE in
scripts that deal with backup and restore.

My testing:

I have a SLES12-SP2-RC3 test system with those
partitions/mountpoints:

# findmnt -a -t ext4 -o TARGET,SOURCE
TARGET  SOURCE
/       /dev/sda2
|-/var  /dev/sda4
`-/home /dev/sda3

and this etc/rear/local.conf:

OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://10.160.4.244/nfs
SSH_ROOT_PASSWORD="rear"
USE_DHCLIENT="yes"
KEEP_BUILD_DIR=""
MANUAL_INCLUDE=YES
BACKUP_PROG_INCLUDE=( '/*' '/var/*' )

I did

# usr/sbin/rear -d -D mkbackup
Relax-and-Recover 1.18 / Git
Using log file: /root/rear/var/log/rear/rear-f157.log
mkdir: created directory '/root/rear/var/lib'
mkdir: created directory '/root/rear/var/lib/rear'
mkdir: created directory '/root/rear/var/lib/rear/output'
Creating disk layout
Excluding mountpoint /. (MANUAL_INCLUDE mode)
Excluding mountpoint /home. (MANUAL_INCLUDE mode)
Excluding mountpoint /var. (MANUAL_INCLUDE mode)
Creating root filesystem layout
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /root/rear/var/lib/rear/output/rear-f157.iso (150M)
Copying resulting files to nfs location
Encrypting disabled
Creating tar archive '/tmp/rear.3rKHiCiYpT4Gc5R/outputfs/f157/backup.tar.gz'
Archived 851 MiB [avg 6657 KiB/sec]OK
Archived 851 MiB in 132 seconds [avg 6607 KiB/sec]

Accordingly I get this var/lib/rear/layout/disklayout.conf

# Disk /dev/sda
# Format: disk   
#disk /dev/sda 21474836480 msdos
# Partitions on /dev/sda
# Format: part      /dev/
#part /dev/sda 1076887552 1048576 primary none /dev/sda1
#part /dev/sda 16105078784 1077936128 primary boot /dev/sda2
#part /dev/sda 2146435072 17183014912 primary none /dev/sda3
#part /dev/sda 2145386496 19329449984 primary none /dev/sda4
# Filesystems (only ext2,ext3,ext4,vfat,xfs,reiserfs,btrfs are supported).
# Format: fs    [uuid=] [label=

i.e. everything is disabled therein so that I would get
nothing recreated during "rear recover".

On the other hand in my backup.tar.gz
I have also all of /home/* and
the whole /var/* content even twice.

Summary from my current point of view:

Currently "MANUAL_INCLUDE=YES" is broken.

At least "MANUAL_INCLUDE=YES" behaves so unexpectedly
that currently I don't know how it is meant to be used correctly.

jsmeix commented at 2016-10-04 10:18:

Some forensics about MANUAL_INCLUDE:

"git log -p usr/share/rear/conf/default.conf" shows

commit eaa95532cf7c3f0f5cbf14c316dcd5b48a49c04e
Author: Robert Paschedag <robert.paschedag@web.de>
Date:   Thu Jul 23 14:08:22 2015 +0200
    entered info for MANUAL_INCLUDE mode in conf/default.conf

and
"git log -p usr/share/rear/layout/save/default/31_include_exclude.sh"
shows

commit 0359a0c1972d5dfd2f32f527deca07396df54f7b
Author: Robert Paschedag <robert.paschedag@web.de>
Date:   Tue Jun 9 23:17:00 2015 +0200
    Added possibility to only backup BACKUP_PROG_INCLUDE

I.e. MANUAL_INCLUDE was added by
https://github.com/rear/rear/commit/0359a0c1972d5dfd2f32f527deca07396df54f7b
and
https://github.com/rear/rear/commit/eaa95532cf7c3f0f5cbf14c316dcd5b48a49c04e

jsmeix commented at 2016-10-04 10:24:

@gdha @schlomo
what is your opinion about MANUAL_INCLUDE?

I do not undrerstand how that is meant to be used
and from my current point of view the current implementation
looks so broken that currently I would simply remove it
but that could of course cause regressions for
Robert Paschedag who is probably the only user of it.

jsmeix commented at 2016-10-04 10:32:

@John-Leone
you have a typo in your BACKUP_PROG_INCLUDE array:

BACKUP_PROG_INCLUDE=("${BACKUP_PROG_INCLUDE[@]}"'/boot' '/' ...

there is no separator (space) between
"${BACKUP_PROG_INCLUDE[@]}" and '/boot'
which results in general not what is intended:

# arr=( 'foo' 'bar' )
# echo ${arr[@]}
foo bar
# arr=( "${arr[@]}"'baz' 'qux' )
# echo ${arr[@]}
foo barbaz qux

gdha commented at 2016-10-04 11:23:

@jsmeix if MANUAL_INCLUDE is so broken we should or ask @rpasche to fix it, or disable it in the meantime (or remove it) when there is no decent response on our request.

jsmeix commented at 2016-10-04 11:36:

@gdha
many thanks for inluding @rpasche - I did not know
the GitHub user account of Robert Paschedag.

I assume for the 1.19 release we can leave it
as is because default.conf explicitly states
"Do NOT use MANUAL_INCLUDE mode."
so that no user should use it carelessly.

I set the milestone to "Rear future".

jsmeix commented at 2016-10-04 11:55:

For completeness:
Now even I found out how to get the GitHub user account:
One has to look at the GitHub commit URL, e.g.
https://github.com/rear/rear/commit/0359a0c1972d5dfd2f32f527deca07396df54f7b
and there the GitHub user account is shown.
But - as far as I know - one cannot get the
GitHub user account via "git log".

rpasche commented at 2016-10-05 06:50:

Puh.....let me search in my mind.

I think the main problem for us was, that rear tried to backup every filesystem it found. This is ok. But often on our servers, extra disks have been added, that should not be backuped by rear, but with another backup software (i.e. TSM or whatever). Failing to explictly exclude the "new" filesystems caused rear (and the final NFS destination) to blow up with this extra data. So we decided to set a "defined" number of filesystems, that need to be saved by rear. Everything else should be ignored. And the easiest way for me was to just set all other filesystems to be "done".
I'm sorry for this inconvenience and as soon as I have time, I'll try to reproduce this and also have a look at our backups (and restores). But until now, we had no problems with that.

jsmeix commented at 2016-10-05 07:43:

@rpasche
many thanks for your prompt reply!

Can you provide one of your etc/real/local.conf files
preferably together with a matching disklayout.conf file
so that I get a better idea how you use it.

When it works for you there is perhaps only some
misunderstanding how MANUAL_INCLUDE is meant to be used
and then it should be sufficient to enhance its documentation
in default.conf to make it more clear how one must use it.

rpasche commented at 2016-10-05 10:51:

Hi @jsmeix ,

here are some files

rear_local.conf https://gist.github.com/rpasche/c64b9a35bf052eedb27150097c7c2498
disklayout.conf https://gist.github.com/rpasche/66ac59e99a94222aba25cb0425afbe7d
backup-include https://gist.github.com/rpasche/c965842f8b5098ed95f597535dbbb938
backup-exclude https://gist.github.com/rpasche/93ee502aa47943185e8bc56f4706f549

rpasche commented at 2016-10-05 11:09:

here, /data2 has been excluded through "MANUAL_INCLUDE" mode.

jsmeix commented at 2016-10-05 12:44:

@rpasche
many thanks for the files.

On quick very first glance I noticed the duplicates
in your backup-include - perhaps a general improvement
makes sense to deduplicate entries in backup-include
(and backup-exclude) to avoid duplicated files in the
backup.tar.gz - but I need to test that...

John-Leone commented at 2016-10-18 17:31:

Hi @jsmeix
How did your additional testing go? Did MANUAL_INCLUDE=YES ever work for you?
Thanks,
John

jsmeix commented at 2016-10-19 08:04:

@John-Leone
I had no time to do anything more here.
For me MANUAL_INCLUDE=YES does not at all work, see
https://github.com/rear/rear/issues/1019#issuecomment-251348729

rpasche commented at 2016-10-19 13:47:

@jsmeix, @John-Leone

Just created another test-vm, made backup and recovery.

Added a second disk with LVM mounted as /testdata with a 5GB file, that should not be backed up by rear. For testing reasons, I removed all virtual disks and recreated new ones.
The recovery worked as expected. The only drawbackup here now is, because /testdata was excluded and was the only filesystem on that LVM (on the second disk), the LVM has not been re-created on that second disk and thus, systemd went into emergency after reboot because that LVM was missing. But commenting this line in /etc/fstab fixed that here.

If I would not have "wiped" the second disk (with /testdata), there would not have been an error at all.

See https://gist.github.com/rpasche/ffa1c797ab363b8d8f88682ce09bb356. I have attached several files.

rpasche commented at 2016-10-20 06:52:

By the way....this should also work, so that the LVM on second disk would have been recreated, without restoring the "contents" of /testdata

BACKUP_PROG_INCLUDE=( "/" "/boot" "/var" "/home" "/data" "/ora" "/tmp" "/testdata" )
BACKUP_PROG_EXCLUDE=( "/data/*" "/ora/*" "/tmp/*" "/testdata/*")

jsmeix commented at 2016-10-20 08:43:

@rpasche regarding your
https://github.com/rear/rear/issues/1019#issuecomment-254817544

I assume you used current GitHub rear master for your test
or current rear 1.19 release (and not an older rear version).

If yes, your test shows that there is still a vaild use-case
for MANUAL_INCLUDE=YES but what is missing is a
more explanatory description how it must be used correctly
so that it works as it is meant to work because
for me - as I try to use it - it does not work.

John-Leone commented at 2016-10-20 11:05:

I’m going try testing again.

FlorianHeigl commented at 2016-10-20 23:38:

@rpasche btw, i also always just need a 'backup those 7 critical filesystems' mode for datacenter use, for P2V I sometimes just want everything. but my default use case is the classical 'only backup vg00' of olden days.

rpasche commented at 2016-10-21 11:43:

@jsmeix ,

looks like "gist" is somehow down right now, so here is the restore log with v1.19 (freshly cloned about 30 minutes ago)

Same config, same disklayout. This time, I did NOT wipe the second disk....only the first disk and also removed the 5GB file, that I had created on the second disk to verify, that it has NOT been backed up by rear (and also not recovered). After restore and reboot, the system just came up as if nothing had happened.

So...I don't know, why this MANUAL_INCLUDE mode is "sooo inpredicteable"?

I tell it in BACKUP_INCLUDE, what filesystems should be saved and from these filesystems, I also want to ignore some content. That's why for example /data is set in BACKUP_INCLUDE and /data/* (see the asterix) is set in BACKUP_EXCLUDE. So rear restores the /data "filesystem", but not the contents on it.

2016-10-21 13:19:51 Relax-and-Recover 1.19 / Git
2016-10-21 13:19:51 Command line options: /bin/rear -v recover
2016-10-21 13:19:51 Using log file: /var/log/rear/rear-rear-restore.log
2016-10-21 13:19:51 Including /etc/rear/os.conf
2016-10-21 13:19:51 Including conf/Linux-i386.conf
2016-10-21 13:19:51 Including conf/GNU/Linux.conf
2016-10-21 13:19:51 Including /etc/rear/local.conf
2016-10-21 13:19:51 Including /etc/rear/rescue.conf
2016-10-21 13:19:51 Running 'init' stage
2016-10-21 13:19:51 Including init/default/01_set_drlm_env.sh
2016-10-21 13:19:51 Including init/default/03_update_recovery_system.sh
2016-10-21 13:19:51 Including init/default/05_check_rear-recover_mode.sh
2016-10-21 13:19:51 Finished running 'init' stage in 0 seconds
2016-10-21 13:19:51 Using build area '/tmp/rear.iQ5TRwTH4iBLiud'
mkdir: created directory '/tmp/rear.iQ5TRwTH4iBLiud/rootfs'
mkdir: created directory '/tmp/rear.iQ5TRwTH4iBLiud/tmp'
2016-10-21 13:19:51 Running recover workflow
2016-10-21 13:19:51 Running 'setup' stage
2016-10-21 13:19:51 Including setup/default/01_pre_recovery_script.sh
2016-10-21 13:19:51 Finished running 'setup' stage in 0 seconds
2016-10-21 13:19:51 Running 'verify' stage
2016-10-21 13:19:51 Including verify/default/02_cciss_scsi_engage.sh
2016-10-21 13:19:51 Including verify/default/02_translate_url.sh
2016-10-21 13:19:51 Using NETFS_URL is deprecated. Use BACKUP_URL instead.
2016-10-21 13:19:51 Including verify/default/03_translate_tape.sh
2016-10-21 13:19:51 Including verify/default/04_validate_variables.sh
2016-10-21 13:19:51 Including verify/NETFS/default/05_check_NETFS_requirements.sh
2016-10-21 13:19:51 Skipping 'ping' test for host 'NFSSERVER' in BACKUP_URL 'nfs://NFSSERVER/data/rear/images'
2016-10-21 13:19:51 Including verify/default/05_create_mappings_dir.sh
2016-10-21 13:19:51 Including verify/GNU/Linux/05_sane_recovery_check.sh
2016-10-21 13:19:51 Including verify/NETFS/default/05_start_required_nfs_daemons.sh
2016-10-21 13:19:51 Starting required daemons for NFS: RPC portmapper (portmap or rpcbind) and rpc.statd if available.
2016-10-21 13:19:51 Started RPC portmapper 'rpcbind'.
2016-10-21 13:19:51 RPC portmapper 'rpcbind' available.
2016-10-21 13:19:51 Started rpc.statd.
2016-10-21 13:19:51 RPC status rpc.statd available.
rpc.idmapd: libnfsidmap: requested translation method, 'nsswitch', is not available

rpc.idmapd: Unable to create name to user id mappings.
2016-10-21 13:19:51 Starting rpc.idmapd failed.
mount: mount point /var/lib/nfs/rpc_pipefs does not exist
2016-10-21 13:19:51 Including verify/NETFS/default/06_mount_NETFS_path.sh
mkdir: created directory '/tmp/rear.iQ5TRwTH4iBLiud/outputfs'
2016-10-21 13:19:51 Mounting with 'mount -v -t nfs -o rw,noatime NFSSERVER:/data/rear/images /tmp/rear.iQ5TRwTH4iBLiud/outputfs'
mount.nfs: timeout set for Fri Oct 21 13:21:56 2016
mount.nfs: trying text-based options 'vers=4,addr=SERVER_IP,clientaddr=CLIENT_IP'
2016-10-21 13:19:56 Including verify/NETFS/default/07_set_backup_archive.sh
2016-10-21 13:19:56 Including verify/NETFS/default/09_set_readonly_options.sh
2016-10-21 13:19:56 Including verify/GNU/Linux/23_storage_and_network_modules.sh
2016-10-21 13:19:56 Including verify/GNU/Linux/26_recovery_storage_drivers.sh
2016-10-21 13:19:56 Including verify/NETFS/default/55_check_backup_archive.sh
2016-10-21 13:19:56 Calculating backup archive size
2016-10-21 13:19:56 Backup archive size is 468M /tmp/rear.iQ5TRwTH4iBLiud/outputfs/rear-restore/backup.tar.gz (compressed)
2016-10-21 13:19:56 Including verify/NETFS/default/60_check_encryption_key.sh
2016-10-21 13:19:56 Including verify/NETFS/default/98_umount_NETFS_dir.sh
2016-10-21 13:19:56 Unmounting '/tmp/rear.iQ5TRwTH4iBLiud/outputfs'
umount: /tmp/rear.iQ5TRwTH4iBLiud/outputfs (NFSSERVER:/data/rear/images) unmounted
rmdir: removing directory, '/tmp/rear.iQ5TRwTH4iBLiud/outputfs'
2016-10-21 13:19:56 Finished running 'verify' stage in 5 seconds
2016-10-21 13:19:56 Running 'layout/prepare' stage
2016-10-21 13:19:56 Including layout/prepare/default/01_prepare_files.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/10_include_partition_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/11_include_lvm_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/12_include_raid_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/13_include_filesystem_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/13_include_mount_filesystem_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/13_include_mount_subvolumes_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/14_include_swap_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/15_include_drbd_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/16_include_luks_code.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/17_include_hpraid_code.sh
2016-10-21 13:19:56 Including layout/prepare/default/20_recreate_hpraid.sh
2016-10-21 13:19:56 Including layout/prepare/GNU/Linux/21_load_multipath.sh
2016-10-21 13:19:56 Including layout/prepare/default/25_compare_disks.sh
2016-10-21 13:19:56 Comparing disks.
2016-10-21 13:19:56 Looking for sda...
2016-10-21 13:19:56 Device sda exists.
2016-10-21 13:19:56 Size of device sda matches.
2016-10-21 13:19:56 Disk configuration is identical, proceeding with restore.
2016-10-21 13:19:56 Including layout/prepare/default/30_map_disks.sh
2016-10-21 13:19:56 Including layout/prepare/default/31_remove_exclusions.sh
2016-10-21 13:19:56 Including layout/prepare/default/32_apply_mappings.sh
2016-10-21 13:19:56 Including layout/prepare/default/40_autoresize_disks.sh
2016-10-21 13:19:56 Including layout/prepare/default/50_confirm_layout.sh
2016-10-21 13:19:56 Including layout/prepare/default/51_list_dependencies.sh
2016-10-21 13:19:56 Including layout/prepare/default/52_exclude_components.sh
2016-10-21 13:19:56 Including layout/prepare/default/54_generate_device_code.sh
2016-10-21 13:19:56 Begin create_fs( fs:/ )
2016-10-21 13:19:56 Begin mount_fs( fs:/ )
2016-10-21 13:19:56 End mount_fs( fs:/ )
2016-10-21 13:19:56 End create_fs( fs:/ )
2016-10-21 13:19:56 Begin create_fs( fs:/data )
2016-10-21 13:19:56 Begin mount_fs( fs:/data )
2016-10-21 13:19:56 End mount_fs( fs:/data )
2016-10-21 13:19:56 End create_fs( fs:/data )
2016-10-21 13:19:56 Begin create_fs( fs:/home )
2016-10-21 13:19:56 Begin mount_fs( fs:/home )
2016-10-21 13:19:56 End mount_fs( fs:/home )
2016-10-21 13:19:56 End create_fs( fs:/home )
2016-10-21 13:19:56 Begin create_fs( fs:/tmp )
2016-10-21 13:19:56 Begin mount_fs( fs:/tmp )
2016-10-21 13:19:56 End mount_fs( fs:/tmp )
2016-10-21 13:19:56 End create_fs( fs:/tmp )
2016-10-21 13:19:56 Begin create_fs( fs:/var )
2016-10-21 13:19:56 Begin mount_fs( fs:/var )
2016-10-21 13:19:56 End mount_fs( fs:/var )
2016-10-21 13:19:56 End create_fs( fs:/var )
2016-10-21 13:19:56 Begin create_fs( fs:/boot )
2016-10-21 13:19:56 Begin mount_fs( fs:/boot )
2016-10-21 13:19:56 End mount_fs( fs:/boot )
2016-10-21 13:19:56 End create_fs( fs:/boot )
2016-10-21 13:19:56 Including layout/prepare/default/55_finalize_script.sh
2016-10-21 13:19:56 Including layout/prepare/default/60_show_unprocessed.sh
2016-10-21 13:19:56 Including layout/prepare/default/61_exclude_from_restore.sh
2016-10-21 13:19:56 Finished running 'layout/prepare' stage in 0 seconds
2016-10-21 13:19:56 Running 'layout/recreate' stage
2016-10-21 13:19:56 Including layout/recreate/default/10_ask_confirmation.sh
2016-10-21 13:19:56 Including layout/recreate/default/20_run_script.sh
2016-10-21 13:19:56 Start system layout restoration.
+++ create_component /dev/sda disk
+++ local device=/dev/sda
+++ local type=disk
+++ local touchfile=disk--dev-sda
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/disk--dev-sda ']'
+++ return 0
+++ Log 'Stop mdadm'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:19:56 Stop mdadm'
2016-10-21 13:19:56 Stop mdadm
+++ grep -q md /proc/mdstat
+++ Log 'Erasing MBR of disk /dev/sda'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:19:56 Erasing MBR of disk /dev/sda'
2016-10-21 13:19:56 Erasing MBR of disk /dev/sda
+++ dd if=/dev/zero of=/dev/sda bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.0230462 s, 22.2 kB/s
+++ sync
+++ LogPrint 'Creating partitions for disk /dev/sda (msdos)'
+++ Log 'Creating partitions for disk /dev/sda (msdos)'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:19:56 Creating partitions for disk /dev/sda (msdos)'
2016-10-21 13:19:56 Creating partitions for disk /dev/sda (msdos)
+++ Print 'Creating partitions for disk /dev/sda (msdos)'
+++ test -n 1
+++ echo -e 'Creating partitions for disk /dev/sda (msdos)'
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ parted -s /dev/sda mklabel msdos
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ parted -s /dev/sda mkpart '"primary"' 1048576B 1000341503B
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ parted -s /dev/sda set 1 boot on
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ parted -s /dev/sda mkpart '"primary"' 1000341504B 32211206143B
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ parted -s /dev/sda set 2 lvm on
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ sleep 1
+++ partprobe -s /dev/sda
/dev/sda: msdos partitions 1 2
+++ my_udevtrigger
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm trigger
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ component_created /dev/sda disk
+++ local device=/dev/sda
+++ local type=disk
+++ local touchfile=disk--dev-sda
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/disk--dev-sda
+++ create_component /dev/sda1 part
+++ local device=/dev/sda1
+++ local type=part
+++ local touchfile=part--dev-sda1
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/part--dev-sda1 ']'
+++ return 0
+++ component_created /dev/sda1 part
+++ local device=/dev/sda1
+++ local type=part
+++ local touchfile=part--dev-sda1
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/part--dev-sda1
+++ create_component /dev/sda2 part
+++ local device=/dev/sda2
+++ local type=part
+++ local touchfile=part--dev-sda2
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/part--dev-sda2 ']'
+++ return 0
+++ component_created /dev/sda2 part
+++ local device=/dev/sda2
+++ local type=part
+++ local touchfile=part--dev-sda2
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/part--dev-sda2
+++ create_component pv:/dev/sda2 lvmdev
+++ local device=pv:/dev/sda2
+++ local type=lvmdev
+++ local touchfile=lvmdev-pv:-dev-sda2
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmdev-pv:-dev-sda2 ']'
+++ return 0
+++ LogPrint 'Creating LVM PV /dev/sda2'
+++ Log 'Creating LVM PV /dev/sda2'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:19:58 Creating LVM PV /dev/sda2'
2016-10-21 13:19:58 Creating LVM PV /dev/sda2
+++ Print 'Creating LVM PV /dev/sda2'
+++ test -n 1
+++ echo -e 'Creating LVM PV /dev/sda2'
+++ lvm vgchange -a n system
  Volume group "system" not found
  Skipping volume group system
+++ true
+++ lvm pvcreate -ff --yes -v --uuid TRZ029-3c4D-QH9j-MTPn-Y3Wr-C1Yu-EtZvwE --restorefile /var/lib/rear/layout/lvm/system.cfg /dev/sda2
    DEGRADED MODE. Incomplete RAID LVs will be processed.
  Couldn't find device with uuid TRZ029-3c4D-QH9j-MTPn-Y3Wr-C1Yu-EtZvwE.
    Wiping cache of LVM-capable devices
    Set up physical volume for "/dev/sda2" with 60958720 available sectors
    Writing physical volume data to disk "/dev/sda2"
  Physical volume "/dev/sda2" successfully created
+++ component_created pv:/dev/sda2 lvmdev
+++ local device=pv:/dev/sda2
+++ local type=lvmdev
+++ local touchfile=lvmdev-pv:-dev-sda2
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmdev-pv:-dev-sda2
+++ create_component /dev/system lvmgrp
+++ local device=/dev/system
+++ local type=lvmgrp
+++ local touchfile=lvmgrp--dev-system
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmgrp--dev-system ']'
+++ return 0
+++ LogPrint 'Restoring LVM VG system'
+++ Log 'Restoring LVM VG system'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:19:58 Restoring LVM VG system'
2016-10-21 13:19:58 Restoring LVM VG system
+++ Print 'Restoring LVM VG system'
+++ test -n 1
+++ echo -e 'Restoring LVM VG system'
+++ '[' -e /dev/system ']'
+++ lvm vgcfgrestore -f /var/lib/rear/layout/lvm/system.cfg system
  Restored volume group system
+++ lvm vgchange --available y system
  6 logical volume(s) in volume group "system" now active
+++ LogPrint 'Sleeping 3 seconds to let udev or systemd-udevd create their devices...'
+++ Log 'Sleeping 3 seconds to let udev or systemd-udevd create their devices...'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:19:58 Sleeping 3 seconds to let udev or systemd-udevd create their devices...'
2016-10-21 13:19:58 Sleeping 3 seconds to let udev or systemd-udevd create their devices...
+++ Print 'Sleeping 3 seconds to let udev or systemd-udevd create their devices...'
+++ test -n 1
+++ echo -e 'Sleeping 3 seconds to let udev or systemd-udevd create their devices...'
+++ sleep 3
+++ component_created /dev/system lvmgrp
+++ local device=/dev/system
+++ local type=lvmgrp
+++ local touchfile=lvmgrp--dev-system
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmgrp--dev-system
+++ create_component /dev/mapper/system-lv_swap lvmvol
+++ local device=/dev/mapper/system-lv_swap
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_swap
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_swap ']'
+++ return 0
+++ component_created /dev/mapper/system-lv_swap lvmvol
+++ local device=/dev/mapper/system-lv_swap
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_swap
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_swap
+++ create_component /dev/mapper/system-lv_root lvmvol
+++ local device=/dev/mapper/system-lv_root
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_root
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_root ']'
+++ return 0
+++ component_created /dev/mapper/system-lv_root lvmvol
+++ local device=/dev/mapper/system-lv_root
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_root
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_root
+++ create_component /dev/mapper/system-lv_var lvmvol
+++ local device=/dev/mapper/system-lv_var
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_var
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_var ']'
+++ return 0
+++ component_created /dev/mapper/system-lv_var lvmvol
+++ local device=/dev/mapper/system-lv_var
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_var
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_var
+++ create_component /dev/mapper/system-lv_tmp lvmvol
+++ local device=/dev/mapper/system-lv_tmp
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_tmp
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_tmp ']'
+++ return 0
+++ component_created /dev/mapper/system-lv_tmp lvmvol
+++ local device=/dev/mapper/system-lv_tmp
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_tmp
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_tmp
+++ create_component /dev/mapper/system-lv_home lvmvol
+++ local device=/dev/mapper/system-lv_home
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_home
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_home ']'
+++ return 0
+++ component_created /dev/mapper/system-lv_home lvmvol
+++ local device=/dev/mapper/system-lv_home
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_home
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_home
+++ create_component /dev/mapper/system-lv_data lvmvol
+++ local device=/dev/mapper/system-lv_data
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_data
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_data ']'
+++ return 0
+++ component_created /dev/mapper/system-lv_data lvmvol
+++ local device=/dev/mapper/system-lv_data
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-system-lv_data
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/lvmvol--dev-mapper-system-lv_data
+++ create_component fs:/ fs
+++ local device=fs:/
+++ local type=fs
+++ local touchfile=fs-fs:-
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:- ']'
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ LogPrint 'Creating filesystem of type ext3 with mount point / on /dev/mapper/system-lv_root.'
+++ Log 'Creating filesystem of type ext3 with mount point / on /dev/mapper/system-lv_root.'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:01 Creating filesystem of type ext3 with mount point / on /dev/mapper/system-lv_root.'
2016-10-21 13:20:01 Creating filesystem of type ext3 with mount point / on /dev/mapper/system-lv_root.
+++ Print 'Creating filesystem of type ext3 with mount point / on /dev/mapper/system-lv_root.'
+++ test -n 1
+++ echo -e 'Creating filesystem of type ext3 with mount point / on /dev/mapper/system-lv_root.'
+++ wipefs -a /dev/mapper/system-lv_root
+++ mkfs -t ext3 -b 4096 -i 16371 -U 633fbdee-3312-499f-a9d3-0392eff11c78 /dev/mapper/system-lv_root
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 2684928 4k blocks and 673056 inodes
Filesystem UUID: 633fbdee-3312-499f-a9d3-0392eff11c78
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208

Allocating group tables:  0/82     done                            
Writing inode tables:  0/82     done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information:  0/82     done

+++ tune2fs -m 4 -c -1 -i 0d -o user_xattr,acl /dev/mapper/system-lv_root
tune2fs 1.42.12 (29-Aug-2014)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 4% (107397 blocks)
+++ LogPrint 'Mounting filesystem /'
+++ Log 'Mounting filesystem /'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:03 Mounting filesystem /'
2016-10-21 13:20:03 Mounting filesystem /
+++ Print 'Mounting filesystem /'
+++ test -n 1
+++ echo -e 'Mounting filesystem /'
+++ mkdir -p /mnt/local/
+++ mount -o rw,relatime,errors=remount-ro,data=ordered /dev/mapper/system-lv_root /mnt/local/
+++ component_created fs:/ fs
+++ local device=fs:/
+++ local type=fs
+++ local touchfile=fs-fs:-
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-
+++ create_component fs:/data fs
+++ local device=fs:/data
+++ local type=fs
+++ local touchfile=fs-fs:-data
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-data ']'
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ LogPrint 'Creating filesystem of type ext3 with mount point /data on /dev/mapper/system-lv_data.'
+++ Log 'Creating filesystem of type ext3 with mount point /data on /dev/mapper/system-lv_data.'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:03 Creating filesystem of type ext3 with mount point /data on /dev/mapper/system-lv_data.'
2016-10-21 13:20:03 Creating filesystem of type ext3 with mount point /data on /dev/mapper/system-lv_data.
+++ Print 'Creating filesystem of type ext3 with mount point /data on /dev/mapper/system-lv_data.'
+++ test -n 1
+++ echo -e 'Creating filesystem of type ext3 with mount point /data on /dev/mapper/system-lv_data.'
+++ wipefs -a /dev/mapper/system-lv_data
+++ mkfs -t ext3 -b 4096 -i 16381 -U 9c3106ff-46c0-4b1e-9071-823f45012ab6 /dev/mapper/system-lv_data
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 2251776 4k blocks and 564144 inodes
Filesystem UUID: 9c3106ff-46c0-4b1e-9071-823f45012ab6
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables:  0/69     done                            
Writing inode tables:  0/69     done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information:  0/69 2/69     done

+++ tune2fs -m 4 -c -1 -i 0d -o user_xattr,acl /dev/mapper/system-lv_data
tune2fs 1.42.12 (29-Aug-2014)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 4% (90071 blocks)
+++ LogPrint 'Mounting filesystem /data'
+++ Log 'Mounting filesystem /data'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:05 Mounting filesystem /data'
2016-10-21 13:20:05 Mounting filesystem /data
+++ Print 'Mounting filesystem /data'
+++ test -n 1
+++ echo -e 'Mounting filesystem /data'
+++ mkdir -p /mnt/local/data
+++ mount -o rw,relatime,data=ordered /dev/mapper/system-lv_data /mnt/local/data
+++ component_created fs:/data fs
+++ local device=fs:/data
+++ local type=fs
+++ local touchfile=fs-fs:-data
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-data
+++ create_component fs:/home fs
+++ local device=fs:/home
+++ local type=fs
+++ local touchfile=fs-fs:-home
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-home ']'
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ LogPrint 'Creating filesystem of type ext3 with mount point /home on /dev/mapper/system-lv_home.'
+++ Log 'Creating filesystem of type ext3 with mount point /home on /dev/mapper/system-lv_home.'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:05 Creating filesystem of type ext3 with mount point /home on /dev/mapper/system-lv_home.'
2016-10-21 13:20:05 Creating filesystem of type ext3 with mount point /home on /dev/mapper/system-lv_home.
+++ Print 'Creating filesystem of type ext3 with mount point /home on /dev/mapper/system-lv_home.'
+++ test -n 1
+++ echo -e 'Creating filesystem of type ext3 with mount point /home on /dev/mapper/system-lv_home.'
+++ wipefs -a /dev/mapper/system-lv_home
+++ mkfs -t ext3 -b 4096 -i 16375 -U 3f296303-856d-475b-b149-8390a43bd9e7 /dev/mapper/system-lv_home
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 487424 4k blocks and 122160 inodes
Filesystem UUID: 3f296303-856d-475b-b149-8390a43bd9e7
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912

Allocating group tables:  0/15     done                            
Writing inode tables:  0/15     done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information:  0/15     done

+++ tune2fs -m 4 -c -1 -i 0d -o user_xattr,acl /dev/mapper/system-lv_home
tune2fs 1.42.12 (29-Aug-2014)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 4% (19496 blocks)
+++ LogPrint 'Mounting filesystem /home'
+++ Log 'Mounting filesystem /home'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:06 Mounting filesystem /home'
2016-10-21 13:20:06 Mounting filesystem /home
+++ Print 'Mounting filesystem /home'
+++ test -n 1
+++ echo -e 'Mounting filesystem /home'
+++ mkdir -p /mnt/local/home
+++ mount -o rw,relatime,data=ordered /dev/mapper/system-lv_home /mnt/local/home
+++ component_created fs:/home fs
+++ local device=fs:/home
+++ local type=fs
+++ local touchfile=fs-fs:-home
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-home
+++ create_component fs:/tmp fs
+++ local device=fs:/tmp
+++ local type=fs
+++ local touchfile=fs-fs:-tmp
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-tmp ']'
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ LogPrint 'Creating filesystem of type ext3 with mount point /tmp on /dev/mapper/system-lv_tmp.'
+++ Log 'Creating filesystem of type ext3 with mount point /tmp on /dev/mapper/system-lv_tmp.'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:06 Creating filesystem of type ext3 with mount point /tmp on /dev/mapper/system-lv_tmp.'
2016-10-21 13:20:06 Creating filesystem of type ext3 with mount point /tmp on /dev/mapper/system-lv_tmp.
+++ Print 'Creating filesystem of type ext3 with mount point /tmp on /dev/mapper/system-lv_tmp.'
+++ test -n 1
+++ echo -e 'Creating filesystem of type ext3 with mount point /tmp on /dev/mapper/system-lv_tmp.'
+++ wipefs -a /dev/mapper/system-lv_tmp
+++ mkfs -t ext3 -b 4096 -i 16375 -U 33bf2d6c-7e2c-4934-9709-ba16abb403c4 /dev/mapper/system-lv_tmp
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 487424 4k blocks and 122160 inodes
Filesystem UUID: 33bf2d6c-7e2c-4934-9709-ba16abb403c4
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912

Allocating group tables:  0/15     done                            
Writing inode tables:  0/15     done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information:  0/15     done

+++ tune2fs -m 4 -c -1 -i 0d -o user_xattr,acl /dev/mapper/system-lv_tmp
tune2fs 1.42.12 (29-Aug-2014)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 4% (19496 blocks)
+++ LogPrint 'Mounting filesystem /tmp'
+++ Log 'Mounting filesystem /tmp'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:06 Mounting filesystem /tmp'
2016-10-21 13:20:06 Mounting filesystem /tmp
+++ Print 'Mounting filesystem /tmp'
+++ test -n 1
+++ echo -e 'Mounting filesystem /tmp'
+++ mkdir -p /mnt/local/tmp
+++ mount -o rw,relatime,data=ordered /dev/mapper/system-lv_tmp /mnt/local/tmp
+++ component_created fs:/tmp fs
+++ local device=fs:/tmp
+++ local type=fs
+++ local touchfile=fs-fs:-tmp
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-tmp
+++ create_component fs:/var fs
+++ local device=fs:/var
+++ local type=fs
+++ local touchfile=fs-fs:-var
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-var ']'
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ LogPrint 'Creating filesystem of type ext3 with mount point /var on /dev/mapper/system-lv_var.'
+++ Log 'Creating filesystem of type ext3 with mount point /var on /dev/mapper/system-lv_var.'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:06 Creating filesystem of type ext3 with mount point /var on /dev/mapper/system-lv_var.'
2016-10-21 13:20:06 Creating filesystem of type ext3 with mount point /var on /dev/mapper/system-lv_var.
+++ Print 'Creating filesystem of type ext3 with mount point /var on /dev/mapper/system-lv_var.'
+++ test -n 1
+++ echo -e 'Creating filesystem of type ext3 with mount point /var on /dev/mapper/system-lv_var.'
+++ wipefs -a /dev/mapper/system-lv_var
+++ mkfs -t ext3 -b 4096 -i 16363 -U 8537f481-b277-4c26-b2f5-e06178544868 /dev/mapper/system-lv_var
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 732160 4k blocks and 183632 inodes
Filesystem UUID: 8537f481-b277-4c26-b2f5-e06178544868
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912

Allocating group tables:  0/23     done                            
Writing inode tables:  0/2317/23     done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information:  0/23     done

+++ tune2fs -m 5 -c -1 -i 0d -o user_xattr,acl /dev/mapper/system-lv_var
tune2fs 1.42.12 (29-Aug-2014)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 5% (36608 blocks)
+++ LogPrint 'Mounting filesystem /var'
+++ Log 'Mounting filesystem /var'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:07 Mounting filesystem /var'
2016-10-21 13:20:07 Mounting filesystem /var
+++ Print 'Mounting filesystem /var'
+++ test -n 1
+++ echo -e 'Mounting filesystem /var'
+++ mkdir -p /mnt/local/var
+++ mount -o rw,relatime,data=ordered /dev/mapper/system-lv_var /mnt/local/var
+++ component_created fs:/var fs
+++ local device=fs:/var
+++ local type=fs
+++ local touchfile=fs-fs:-var
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-var
+++ create_component fs:/boot fs
+++ local device=fs:/boot
+++ local type=fs
+++ local touchfile=fs-fs:-boot
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-boot ']'
+++ return 0
+++ my_udevsettle
+++ has_binary udevadm
+++ for bin in '$@'
+++ type udevadm
+++ return 0
+++ udevadm settle
+++ LogPrint 'Creating filesystem of type ext3 with mount point /boot on /dev/sda1.'
+++ Log 'Creating filesystem of type ext3 with mount point /boot on /dev/sda1.'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:07 Creating filesystem of type ext3 with mount point /boot on /dev/sda1.'
2016-10-21 13:20:07 Creating filesystem of type ext3 with mount point /boot on /dev/sda1.
+++ Print 'Creating filesystem of type ext3 with mount point /boot on /dev/sda1.'
+++ test -n 1
+++ echo -e 'Creating filesystem of type ext3 with mount point /boot on /dev/sda1.'
+++ wipefs -a /dev/sda1
+++ mkfs -t ext3 -b 4096 -i 16366 -U bd12a504-305c-47ed-920b-cc269cf5d461 /dev/sda1
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 243968 4k blocks and 61184 inodes
Filesystem UUID: bd12a504-305c-47ed-920b-cc269cf5d461
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376

Allocating group tables: 0/8   done                            
Writing inode tables: 0/8   done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/8   done

+++ tune2fs -m 4 -c -1 -i 0d -o user_xattr,acl /dev/sda1
tune2fs 1.42.12 (29-Aug-2014)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 4% (9758 blocks)
+++ LogPrint 'Mounting filesystem /boot'
+++ Log 'Mounting filesystem /boot'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:08 Mounting filesystem /boot'
2016-10-21 13:20:08 Mounting filesystem /boot
+++ Print 'Mounting filesystem /boot'
+++ test -n 1
+++ echo -e 'Mounting filesystem /boot'
+++ mkdir -p /mnt/local/boot
+++ mount -o rw,relatime,data=ordered /dev/sda1 /mnt/local/boot
+++ component_created fs:/boot fs
+++ local device=fs:/boot
+++ local type=fs
+++ local touchfile=fs-fs:-boot
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/fs-fs:-boot
+++ create_component swap:/dev/mapper/system-lv_swap swap
+++ local device=swap:/dev/mapper/system-lv_swap
+++ local type=swap
+++ local touchfile=swap-swap:-dev-mapper-system-lv_swap
+++ '[' -e /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/swap-swap:-dev-mapper-system-lv_swap ']'
+++ return 0
+++ LogPrint 'Creating swap on /dev/mapper/system-lv_swap'
+++ Log 'Creating swap on /dev/mapper/system-lv_swap'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2016-10-21 13:20:08 Creating swap on /dev/mapper/system-lv_swap'
2016-10-21 13:20:08 Creating swap on /dev/mapper/system-lv_swap
+++ Print 'Creating swap on /dev/mapper/system-lv_swap'
+++ test -n 1
+++ echo -e 'Creating swap on /dev/mapper/system-lv_swap'
+++ mkswap -U 0dd4d9f7-0d9f-43fc-a619-f1b61b698bee /dev/mapper/system-lv_swap
Setting up swapspace version 1, size = 3903484 KiB
no label, UUID=0dd4d9f7-0d9f-43fc-a619-f1b61b698bee
+++ component_created swap:/dev/mapper/system-lv_swap swap
+++ local device=swap:/dev/mapper/system-lv_swap
+++ local type=swap
+++ local touchfile=swap-swap:-dev-mapper-system-lv_swap
+++ touch /tmp/rear.iQ5TRwTH4iBLiud/tmp/touch/swap-swap:-dev-mapper-system-lv_swap
+++ set +x
2016-10-21 13:20:08 Disk layout created.
2016-10-21 13:20:08 Including layout/recreate/default/25_verify_mount.sh
2016-10-21 13:20:08 Finished running 'layout/recreate' stage in 12 seconds
2016-10-21 13:20:08 Running 'restore' stage
2016-10-21 13:20:08 Including restore/NETFS/default/10_mount_NETFS_path.sh
mkdir: created directory '/tmp/rear.iQ5TRwTH4iBLiud/outputfs'
2016-10-21 13:20:08 Mounting with 'mount -v -t nfs -o ro NFSSERVER:/data/rear/images /tmp/rear.iQ5TRwTH4iBLiud/outputfs'
mount.nfs: timeout set for Fri Oct 21 13:22:08 2016
mount.nfs: trying text-based options 'vers=4,addr=SERVER_IP,clientaddr=CLIENT_IP'
2016-10-21 13:20:08 Including restore/NETFS/default/38_prepare_multiple_isos.sh
2016-10-21 13:20:08 Including restore/NETFS/default/40_restore_backup.sh
2016-10-21 13:20:08 Decrypting disabled
2016-10-21 13:20:08 Restoring tar archive '/tmp/rear.iQ5TRwTH4iBLiud/outputfs/rear-restore/backup.tar.gz'
2016-10-21 13:20:08 dd if=/tmp/rear.iQ5TRwTH4iBLiud/outputfs/rear-restore/backup.tar.gz | cat | tar --block-number --totals --verbose --anchored --gzip -C /mnt/local/ -x -f -
956877+1 records in
956877+1 records out
489921112 bytes (490 MB) copied, 13.6799 s, 35.8 MB/s
Total bytes read: 1124730880 (1.1GiB, 79MiB/s)
2016-10-21 13:20:23 Restored 1029 MiB in 14 seconds [avg 75312 KiB/sec]
2016-10-21 13:20:23 Including restore/NETFS/default/50_selinux_autorelabel.sh
2016-10-21 13:20:23 Including restore/default/50_selinux_autorelabel.sh
2016-10-21 13:20:23 Created /.autorelabel file : after reboot SELinux will relabel all files
2016-10-21 13:20:23 Including restore/NETFS/Linux-i386/51_selinux_fixfiles_exclude_dirs.sh
2016-10-21 13:20:23 Including restore/NETFS/default/51_set_capabilities.sh
2016-10-21 13:20:23 Including restore/default/90_create_missing_directories.sh
2016-10-21 13:20:23 Restore the Mountpoints (with permissions) from /var/lib/rear/recovery/mountpoint_permissions
2016-10-21 13:20:23 Including restore/NETFS/default/98_umount_NETFS_dir.sh
2016-10-21 13:20:23 Unmounting '/tmp/rear.iQ5TRwTH4iBLiud/outputfs'
umount: /tmp/rear.iQ5TRwTH4iBLiud/outputfs (NFSSERVER:/data/rear/images) unmounted
rmdir: removing directory, '/tmp/rear.iQ5TRwTH4iBLiud/outputfs'
2016-10-21 13:20:23 Including restore/default/99_move_away_restored_files.sh
2016-10-21 13:20:23 Finished running 'restore' stage in 15 seconds
2016-10-21 13:20:23 Running 'finalize' stage
2016-10-21 13:20:23 Including finalize/default/01_prepare_checks.sh
2016-10-21 13:20:23 Including finalize/default/10_populate_dev.sh
2016-10-21 13:20:23 Including finalize/GNU/Linux/15_migrate_disk_devices_layout.sh
2016-10-21 13:20:23 Including finalize/GNU/Linux/15_migrate_lun_wwid.sh
2016-10-21 13:20:23 Including finalize/GNU/Linux/15_migrate_uuid_tags.sh
2016-10-21 13:20:23 Including finalize/GNU/Linux/16_rename_diskbyid.sh
2016-10-21 13:20:23 Including finalize/Debian/i386/17_rebuild_initramfs.sh
2016-10-21 13:20:23 Including finalize/Linux-i386/21_install_grub.sh
/sbin/grub-probe
2016-10-21 13:20:23 Skip installing GRUB Legacy boot loader because GRUB 2 is installed (grub-probe or grub2-probe exist).
2016-10-21 13:20:23 Including finalize/Linux-i386/22_install_elilo.sh
2016-10-21 13:20:23 Including finalize/Linux-i386/22_install_grub2.sh
2016-10-21 13:20:23 Installing GRUB2 boot loader
Installing for i386-pc platform.
grub-install: warning: cannot open directory `/usr/share/locale': No such file or directory.
Installation finished. No error reported.
2016-10-21 13:20:24 Including finalize/Linux-i386/23_run_efibootmgr.sh
2016-10-21 13:20:24 Including finalize/GNU/Linux/30_create_mac_mapping.sh
2016-10-21 13:20:24 Including finalize/GNU/Linux/41_migrate_udev_rules.sh
2016-10-21 13:20:24 Including finalize/GNU/Linux/42_migrate_network_configuration_files.sh
2016-10-21 13:20:24 Including finalize/default/88_check_for_mount_by_id.sh
2016-10-21 13:20:24 Including finalize/default/89_finish_checks.sh
2016-10-21 13:20:24 Including finalize/default/90_remount_sync.sh
2016-10-21 13:20:24 Finished running 'finalize' stage in 1 seconds
2016-10-21 13:20:24 Running 'wrapup' stage
2016-10-21 13:20:24 Including wrapup/default/50_post_recovery_script.sh
2016-10-21 13:20:24 Including wrapup/default/98_good_bye.sh
2016-10-21 13:20:24 Including wrapup/default/99_copy_logfile.sh
2016-10-21 13:20:24 Finished running 'wrapup' stage in 0 seconds
2016-10-21 13:20:24 Finished running recover workflow
2016-10-21 13:20:24 Running exit tasks.

Now what next?

Regards,

rpasche commented at 2016-10-21 13:34:

Hi @jsmeix ,

maybe you can give me a hint. It's of course possible, that I was to blind to setup a configuration for our needs without developing the MANUAL_INCLUDE mode.

How do you create a config and define only a "standard" set of filesystems that you want to backup and ignore everything else that could be added in future, without the need to edit the configuration?

If I'm able to setup such a configuration, MANUAL_INCLUDE could be thrown away... ;-)

Meanwhile, started digging into the code again...
Regards,

rpasche commented at 2016-10-21 14:01:

I just also found the error, why the filesystems are listed twice for backup. It is in

https://github.com/rear/rear/blob/master/usr/share/rear/backup/NETFS/default/40_create_include_exclude_files.sh

Because everything from BACKUP_PROG_INCLUDE is added to backup-include.txt and all mountpoints NOT in EXCLUDE_MOUNTPOINTS. This is - truely - an error, when running in MANUAL_INCLUDE mode. The mounpoints in mountpoint_device should NOT be added to the backup-include.txt list if in MANUAL_INCLUDE mode as BACKUP_PROG_INCLUDE is used.

But I have to test this...

Regards,

jsmeix commented at 2016-11-08 09:05:

@rpasche
many thanks for working on the MANUAL_INCLUDE mode!

@John-Leone
if you like to test if current ReaR GitHub master code
(that contains https://github.com/rear/rear/pull/1060)
now works o.k. for you in MANUAL_INCLUDE mode:

How to test the currently newest rear GitHub master code:

Basically "git clone" it into a directory and
then run rear from within that directory like:

# git clone https://github.com/rear/rear.git
# cd rear
# vi etc/rear/local.conf
# usr/sbin/rear -d -D mkbackup

John-Leone commented at 2016-11-10 21:34:

Hello Johannes,
The MANUAL_INCLUDE=YES did not work as expected, the recovery restored all the file systems.
I was expecting /home and /u01 not to be recovered.

Please check below; is there something wrong with my configuration files?
For testing I used a Cisco C240 physical server running RHEL 6.8

[root@xsmithm1d rear]# cat local.conf

John added the lines below.

OUTPUT=ISO
OUTPUT_URL=sshfs://root@xxx.xx.xx.xx/dbar/
BACKUP=NETFS
BACKUP_URL=sshfs://root@xxx.xx.xx.xx/dbar/
SSH_ROOT_PASSWORD="redhat"
BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/media' '/var/tmp' '/var/crash')
MANUAL_INCLUDE=YES
BACKUP_PROG_INCLUDE=("${BACKUP_PROG_INCLUDE[@]}" '/boot' '/' '/opt' '/midrange' '/tmp' '/var' )
NETFS_KEEP_OLD_BACKUP_COPY=

I did…..

[root@xladmm1p ~]# ssh -A xsmithm1d "/root/rear-master/usr/sbin/rear -d -D mkbackup"
Relax-and-Recover 1.19 / Git
Using log file: /root/rear-master/var/log/rear/rear-xsmithm1d.log
insmod /lib/modules/2.6.32-642.3.1.el6.x86_64/kernel/fs/fuse/fuse.ko
Creating disk layout
Excluding mountpoint /home. (MANUAL_INCLUDE mode)
Excluding mountpoint /u01. (MANUAL_INCLUDE mode)
Creating root filesystem layout
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /root/rear-master/var/lib/rear/output/rear-xsmithm1d.iso (89M)
Copying resulting files to sshfs location
Encrypting disabled
Creating tar archive '/tmp/rear.oqfXxaujNSgKPLj/outputfs/xsmithm1d/backup.tar.gz'
Preparing archive operation OK
Archived 1337 MiB in 212 seconds [avg 6460 KiB/sec]
You should also rm -Rf /tmp/rear.oqfXxaujNSgKPLj

Result……..

[root@xsmithm1d rear]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rootvg-root
7.8G 2.2G 5.6G 28% /
tmpfs 16G 0 16G 0% /dev/shm
/dev/sda1 488M 111M 378M 23% /boot
/dev/mapper/rootvg-home
5.8G 13M 5.8G 1% /home
/dev/mapper/rootvg-midrange
4.8G 212M 4.6G 5% /midrange
/dev/mapper/rootvg-opt
7.8G 543M 7.3G 7% /opt
/dev/mapper/rootvg-tmp
7.8G 797M 7.0G 11% /tmp
/dev/mapper/rootvg-var
9.8G 1.1G 8.2G 12% /var
/dev/mapper/rootvg-u01
20G 44M 19G 1% /u01

Also…..
The recovery image configured the network as rename4 and I needed activate the rename4 interface to start the recovery, network works fine with Rear Version1.17.2

RESCUE xsmithm1d:~ # ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether 90:e2:ba:84:d9:d0 brd ff:ff:ff:ff:ff:ff
3: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 90:e2:ba:84:d9:d1 brd ff:ff:ff:ff:ff:ff
4: rename4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether f4:4e:05:9f:44:9e brd ff:ff:ff:ff:ff:ff
inet xxx.xxx.xxx.xx/24 scope global rename4
inet6 fe80::f64e:5ff:fe9f:449e/64 scope link
valid_lft forever preferred_lft forever
5: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether f4:4e:05:9f:44:9f brd ff:ff:ff:ff:ff:ff

jsmeix commented at 2016-11-11 08:46:

@John-Leone FYI:
currently I do not work on this issue,
perhaps at a later time - as time permits - but no guarantee.

John-Leone commented at 2016-11-11 12:30:

Thanks for the update.

rpasche commented at 2016-11-11 15:27:

@John-Leone
Can you provide the backup.log and recovery.log and also disklayout.conf and disktodo.conf

rpasche commented at 2016-11-11 17:39:

Can you also show output of pvs, lvs and vgs? Do you actually have contents restored to /u01 or /home? Looks pretty small.

John-Leone commented at 2016-11-14 20:54:

I have uploaded the files as requested, please see rear.tar.gz

https://github.com/rear/rear/files/590448/rear.tar.gz

John-Leone commented at 2016-11-14 20:58:

Below is the output from lvs, pvs and vgs.
For testing purposes I created /u01 and put very small files on the filesytem, then I expected /u01 and /home to be excluded during the recovery.

[root@xsmithm1d recover]# lvs
  LV       VG     Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home     rootvg -wi-ao----  6.00g
  midrange rootvg -wi-ao----  5.00g
  opt      rootvg -wi-ao----  8.00g
  origroot rootvg -wi-a----- 11.00g
  origvar  rootvg -wi-a----- 13.00g
  root     rootvg -wi-ao----  8.00g
  swap     rootvg -wi-ao----  8.00g
  tmp      rootvg -wi-ao----  8.00g
  u01      rootvg -wi-ao---- 20.00g
  var      rootvg -wi-ao---- 10.00g

[root@xsmithm1d recover]# pvs
  PV         VG     Fmt  Attr PSize PFree
  /dev/sda2  rootvg lvm2 a--u 1.09t 1018.22g

[root@xsmithm1d recover]# vgs
  VG     #PV #LV #SN Attr   VSize VFree
  rootvg   1  10   0 wz--n- 1.09t 1018.22g

rpasche commented at 2016-11-15 21:22:

@John-Leone

So after looking into the files, I have some questions regarding your restore tests.

Is this a test machine and did you "wipe" the disks before you tried to recover? To me, it looks like the neither /u01 nor /home have been backuped nor restored.

grep -E '/(home|u01)' backup.log
block 226236: /u01/
block 1653949: /usr/lib64/python2.6/site-packages/matplotlib/mpl-data/images/home.png
block 1653976: /usr/lib64/python2.6/site-packages/matplotlib/mpl-data/images/home.svg
block 1654060: /usr/lib64/python2.6/site-packages/matplotlib/mpl-data/images/home.ppm
block 1654118: /usr/lib64/python2.6/site-packages/matplotlib/mpl-data/images/home.xpm
block 3592346: /usr/share/doc/libxslt-1.1.26/EXSLT/html/home.png
block 3594513: /usr/share/doc/libxslt-1.1.26/html/home.png
block 3647508: /usr/share/doc/valgrind-3.8.1/html/images/home.png
block 3966491: /etc/selinux/targeted/modules/active/homedir_template
block 4222545: /home/

The "directories" have been backuped...but that's normal as the directories itself are stored on /.
I also see, that the filesystems of /home and /u01 have not been restored. What I see, is that the LVMs have been restored (the complete VG with vgcfgrestore). Because of that, the "original" fs of /home and /u01 have been found again. That's why the files on /home and /u01 are present "again".

It would be cool, if you could try your backup and recover again. But this time, wipe the whole disk before you try to recover, if this is possible for you.

jsmeix commented at 2016-11-16 09:39:

A side note FYI regarding "wipe the disk":
We have a pending issue for that, see
https://github.com/rear/rear/issues/799

John-Leone commented at 2016-11-16 22:14:

I tested again, this is what I did:

  1. First "rear -v mkbackup"
  2. next, deleted everything from /home
  3. then, un-mounted /u01 & " lvremove /dev/mapper/rootvg-u01"

This is the local.conf file

John added the lines below.

OUTPUT=ISO
OUTPUT_URL=sshfs://root@xxx.xx.xx.xx/dbar/
BACKUP=NETFS
BACKUP_URL=sshfs://root@xxx.xx.xx.xx/dbar/
SSH_ROOT_PASSWORD="redhat"
BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/media' )
MANUAL_INCLUDE=YES
BACKUP_PROG_INCLUDE=("${BACKUP_PROG_INCLUDE[@]}" '/boot' '/' '/opt' '/midrange' '/tmp' '/var' )
NETFS_KEEP_OLD_BACKUP_COPY=

After the system was recovered /home was mounted but still empty.
and /u01 was recovered and mounted.

Since "BACKUP_PROG_INCLUDE" statement did not include /u01 & /home I didn't expect /u01 to be restored and /home was restored but none of the files were recovered.

rpasche commented at 2016-11-17 08:03:

Hi @John-Leone ,

the only thing you can blame MANUAL_INCLUDE mode right now, as ít - currently - does NOT "remove" a logical volume of the filesystem, that had been ignored "after" the whole volume group has been restored.

After the system was recovered /home was mounted but still empty.

Well....this is what you wanted, hey? At least, that nothing has been restored, as nothing has been backuped.

and /u01 was recovered and mounted.

Just to explain again, what is happening.

The whole volume group gets restored within recovery (with vgcfgrestore, "including" the logical volumes for /home and /u01 in your case.
So your systems knows, that there are 2 logical volumes with /home and /u01 again. And because you did not overwrite a range of sectores with zeros (or random crap) where /home and /u01 started, your filesystem drivers will find a valid filesystem ready to mount again. And because nothing has been backuped from /home (because it was excluded), nothing has been restored...hence, it's empty.

The same with /u01. Just removing the logical volume with lvremove won't destroy the filesystem in it. You just removed some metadata about the name, size, startsector, endsector and some other stuff from LVM. But we restored this information with vgcfgrestore so everything is found again on /u01.

Currently, my first opinion would be, to remove the logical volumes after vgcfgrestore has restored the information within recovery to fix this. I'll have to check this.

To "quickly" fix this iin your case, you could just "lvremove" /home and /u01 after restore has finished and your setup should look as expected.

rpasche commented at 2016-11-17 11:18:

Hmm...looks like in MIGRATION_MODE the parts of the LVM get recreated step by step. Still looking....

John-Leone commented at 2016-11-30 18:50:

@rpasche
Are you still working on MANUAL_INCLUDE mode?

rpasche commented at 2016-12-01 08:38:

Yes....somehow....but I'm currently not sure, if the current situation is a "real" problem.

I don't think, that it is. The main feature of ReaR is to recover the system to a state that it was before a crash. And this is what it did. It did not touch the filesystems /home or /u01, To be more precise...it did NOT RECREATE the filesystems on those 2 volumes. For me it sounds that you like your system to look different after a recover as you don't want that /home or /u01 be recovered.

Just another point....if the logical volumes of /home and /u01 would not be recovered, all your data that you store in those 2 directories (after you took the server back into production after the recover) will consume space on the / filesystem.

Regards...

jsmeix commented at 2016-12-01 10:11:

To only specify what is in the backup independent
form what components (e.g. partitions, filesystems, ...)
will be recreated (or excluded from being recreated)
the following could be of interest:

While implementing
https://github.com/rear/rear/issues/1088
I needed functionality to specify exactly without ReaR's implicit
automagic what I want to have in the backup and what not
(but it must work in a backward compatible way so that
I cannot simply remove ReaR's implicit automagic here).

Therefore via
https://github.com/rear/rear/pull/1091
I invented BACKUP_ONLY_INCLUDE
so that I do not have to use MANUAL_INCLUDE=YES
and - by the way - I invented also its logical counterpart
BACKUP_ONLY_EXCLUDE which could be useful to get things
in the backup that are not intended to be restored (i.e. have
the content of EXCLUDE_MOUNTPOINTS in the backup).

I think in general a backup should be complete
(i.e. basically it should contain all regular files - except tmp files)
regardless what exactly is actually needed to be restored,
see also EXCLUDE_RESTORE versus
BACKUP_RESTORE_MOVE_AWAY_FILES
in default.conf.

This means in current GitHub 'master' code
there are now the new boolean variables
BACKUP_ONLY_INCLUDE and
BACKUP_ONLY_EXCLUDE used in
backup/NETFS/default/400_create_include_exclude_files.sh

I.e. currently it only works for those backupo methods that use
backup/NETFS/default/400_create_include_exclude_files.sh
which are NETFS, RSYNC, and BORG because
backup/BORG/default/400_create_include_exclude_files.sh
and
backup/RSYNC/default/400_create_include_exclude_files.sh
are symbolic links to
backup/NETFS/default/400_create_include_exclude_files.sh

Cf. the section about
"Relax-and-Recover versus backup and restore" in
https://en.opensuse.org/SDB:Disaster_Recovery
that reads in particular (excerpt):

It is very different to what extent support for
each individual backup tool is implemented
in ReaR because support for each individual
backup tool is implemented separated
from each other.

How it works:

When BACKUP_ONLY_INCLUDE is set to a 'true' value
only what is specified in BACKUP_PROG_INCLUDE
will be in the backup but not implicitly also all local filesystems
as defined in mountpoint_device.

When BACKUP_ONLY_EXCLUDE is set to a 'true' value
only what is specified in BACKUP_PROG_EXCLUDE
will be excluded from the backup but not implicitly also
all excluded mountpoints in EXCLUDE_MOUNTPOINTS.

rpasche commented at 2016-12-01 10:44:

@jsmeix

Wow! You have been busy. I have to test this new stuff at the other customer when I'm there the next time.

gdha commented at 2017-01-07 11:17:

@John-Leone @jsmeix @rpasche It would be nice to get rid of MANUAL_INCLUDE variable when the variables of @jsmeix BACKUP_PROG_INCLUDE and BACKUP_PROG_EXCLUDE are proven to work as designed. Sure, more testing is required, but at least we have a goal for ReaR v2.1, right?

rpasche commented at 2017-01-07 11:43:

Am 7. Januar 2017 12:17:15 MEZ schrieb gdha notifications@github.com:

@John-Leone @jsmeix @rpasche It would be nice to get rid of
MANUAL_INCLUDE variable when the variables of @jsmeix
BACKUP_PROG_INCLUDE and BACKUP_PROG_EXCLUDE are proven to work as
designed. Sure, more testing is required, but at least we have a goal
for ReaR v2.1, right?

True but unfortunately I had no time yet to test the new version 🙁

John-Leone commented at 2017-01-08 15:22:

@John-Leone @jsmeix @rpasche @gdha
Thanks for the update.

John-Leone commented at 2017-01-11 21:41:

@jsmeix @rpasche @gdha
I was wondering what is the ETA for ReaR v2.1?

jsmeix commented at 2017-01-12 08:32:

see
https://github.com/rear/rear/milestones

John-Leone commented at 2017-02-03 17:48:

@jsmeix @gdha @rpasche I think the intent for MANUAL_INCLUDE was to define only the file systems you need backed-up then the others would be ignored so you don't consume all the DASD on the NFS share. BACKUP_PROG_EXCLUDE works fine but since each server is different we need to add which items need to be excluded.

gdha commented at 2017-04-25 18:23:

@rpasche @jsmeix @John-Leone I think we can close this issue, no?

jsmeix commented at 2017-04-26 10:32:

I close it because with BACKUP_ONLY_INCLUDE
and BACKUP_ONLY_EXCLUDE there is an
alternative to MANUAL_INCLUDE so that now
users can use what fits best for them.

[Export of Github issue for rear/rear.]