#733 Issue closed: REAR not backing up all LVM volumes¶
Labels: enhancement, support / question,
won't fix / can't fix / obsolete
razorfish-sl opened issue at 2015-12-07 22:22:¶
It seems that lvm volumes that contain block images do not restore correctly.
multiple LVM volumes over:
sda, 20gb
sdb , 350gb block image
Installed rear 1.17.2
setup NAS via NFS
dropped down to:
Init1
reconfigured missing network interface for ethernet
rear -v mkbackup
Total backup gz only 20gb
moved to secondary system, booted from iso
rear restore
sda restored
sdb MISSING....
gdha commented at 2015-12-08 13:17:¶
@razorfish-sl could you paste the content of
/var/lib/rear/layout/disklayout.conf file?
razorfish-sl commented at 2015-12-09 09:30:¶
gdha commented at 2015-12-09 09:40:¶
@razorfish-sl According the disklayout no Lvol from /dev/cinder-volumes was mounted at the time rear was running. Can you confirm this? If that is the case then this is how rear has been designed. Only deal with mounted FS, Lvols (of active Vgs).
razorfish-sl commented at 2015-12-10 13:07:¶
[root@core ~]# cd /dev/cinder-volumes
[root@core cinder-volumes]# ls -la
total 0
drwxr-xr-x. 2 root root 60 Dec 7 12:49 .
drwxr-xr-x. 21 root root 3340 Dec 9 01:30 ..
lrwxrwxrwx. 1 root root 7 Dec 7 12:49
volume-7bf54f60-5e6e-421b-851c-849b3c2908c2 -> ../dm-2
[root@core cinder-volumes]#
but it is 'not' formally mounted to a mount point as in 'mount /dev/mapper/centos-spare /spare'
it cannot be because the partitions are for use only by the internal virtualized systems, they are block devices only, their internal structure might not even be readable by the host os.
You 'could' virtualize those block devices as 'files' in the host , but then you would have potential issues with fragmentation.
gdha commented at 2015-12-11 09:14:¶
@razorfish-sl So, to me, rear does what it needs to do - archiving the
root disk; from your explanation it wouldn't make sense anyway to
archive the cinder-volumes.
I don't know what your expectation are, but I have no clue how you need
to backup the cinder-volumes without risking the backup is worthless?
razorfish-sl commented at 2015-12-13 07:04:¶
how can the backup be worthless if I issued an init 1?
cinder storage is fixed block storage for the instances, backing it up
is no different than doing a "dd' to a zip/tar file
gdha commented at 2015-12-24 10:50:¶
@razorfish-sl Ok understood - just mount it for the sake of rear before running mkbackup then it will be taken into account.
razorfish-sl commented at 2015-12-25 03:43:¶
That may be problematic, since you would have to know & mount each block
volume listed.
and since it is a 'cloud' you cannot really know what is or is not there
until backup time.
Which would make it a manual process, uinless there is a "mount with
'*' functionality".
gdha commented at 2016-02-10 15:59:¶
@razorfish-sl is this feature request? Be able to include block volumes which are not mounted? Are how do you see it?
razorfish-sl commented at 2016-02-11 23:11:¶
Hi,
Yes it would be useful in a cloud setup.
Don't get me wrong I love 'REAR', but missing this sort of thing without
warning is a major problem...
With this "openstack" setup,some volumes are not 'physically' mounted to
prevent the OS fighting over them.
The device is accessed directly from within open-stack , even though it
can be mounted externally by the OS and contains a valid filesystem,
This device is accessed from within a VM with a different user setup, if
you pull it into a normal linux setup with the users not present, it
causes all sorts of 'rights' issues if you try to do an internal file
backup.
so it makes far more sense to do a "block" backup and not care about the
internal content.
schlomo commented at 2016-02-12 07:02:¶
@razorfish-sl this is a very interesting use case you are bringing up here.
Please keep in mind that ReaR was designed for a different purpose: Bare metal disaster recovery of a single server using an external backup solution.
The internal NFS-based backup was added much later as a tool for quick testing and it is much less powerul as a backup solution than real backup software.
By design ReaR operates on the filesystem level. That means that anything not mounted is out of scope for ReaR.
The design goal of ReaR was to take a backup / layout snapshot of a
running system so we never even considered thinking about doing an
init 1 for ReaR.
ReaR also has a very clear separation between structural information (e.g. LVM configurations) and file contents. The nice thing about structural information is that it does not change so fast while the system runs so that is perfectly OK to record it without any kind of quiescing. Filesystems and block blobs are the opposite: If you want to get a consistent backup then you must think about quiescing or you must have a system where it does not matter if the first 100 files/blocks are recorded 2 hourse before the last 100 files/blocks.
With the system area of a server this is for most cases an acceptable risk. For a database filesystem this is certain death. @gdha was alluding to this issue when he wondered about the backup beeing worthless.
We have already had other users who wanted to recovery Oracle data partitions etc. Or who wanted to include shared storage devices that where not mounted etc.
That said, we will be most happy to get a pull request for new functionality and of course also help you develop it.
Maybe backup concepts for cinder are yet to be developed.
jsmeix commented at 2016-02-12 10:00:¶
@schlomo
many thanks for your explanatory information about
fundamental design ideas behind rear.
That helps so much!
I think this is not yet sufficiently explained in the rear
documentation. I mean it is not yet explicitly spelled out
in the current rear documentation.
I think it would help if 09-design-concepts.adoc
was updated and enhanced.
jsmeix commented at 2016-02-12 10:28:¶
I think https://github.com/rear/rear/issues/733#issuecomment-183205395 explains sufficiently why this particular issue currently contradicts the basic design of rear.
Therefore I close this particular issue for now as "won't fix" which
actually means "cannot fix" because it would require major design
changes in rear.
For the future I filed https://github.com/rear/rear/issues/769 "Brainstorming about fundamental enhancements for the backup".
[Export of Github issue for rear/rear.]