#581 Issue closed: Cannot create ISO image >4GB on RHEL6¶
Labels: documentation, support / question
wwoo1 opened issue at 2015-04-24 21:21:¶
Hello... I could not backup to ISO images >4GB. I set ISO_MAX_SIZE=10000 and that did not help either. Thanks.
[root@teak rear]# rear -v mkbackup
Relax-and-Recover 1.17.0 / Git
Using log file: /var/log/rear/rear-teak.log
Creating disk layout
Creating root filesystem layout
TIP: To login as root via ssh you need to set up /root/.ssh/authorized_keys or SSH_ROOT_PASSWORD in your configuration file
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Encrypting disabled
Creating tar archive '/backups/rear/rear.RPwWr8Nk1dERlAI/tmp/isofs/backup/backup.tar.gz'
Archived 5533 MiB [avg 6112 KiB/sec]OK
Archived 5533 MiB in 928 seconds [avg 6106 KiB/sec]
Preparing 2 ISO images ...
Making additionnal ISO image : rear-teak_01.iso
Wrote ISO image: /backups/rear/rear-teak_01.iso (1.2G)
Making ISO image
ERROR: Could not create ISO image (with /usr/bin/mkisofs)
Aborting due to an error, check /var/log/rear/rear-teak.log for details
Terminated
[root@teak rear]#
From /var/log/rear/rear-teak.log
2015-04-24 12:34:54 Wrote ISO image: /backups/rear/rear-teak_01.iso (1.2G)
2015-04-24 12:34:54 Including output/ISO/Linux-i386/82_create_iso_image.sh
2015-04-24 12:34:54 Starting '/usr/bin/mkisofs'
2015-04-24 12:34:54 Making ISO image
genisoimage 1.1.9 (Linux)
Scanning .
Scanning ./backup
File ./backup/backup.tar.gz.00 is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.
2015-04-24 12:34:54 ERROR: Could not create ISO image (with /usr/bin/mkisofs)
=== Stack trace ===
Trace 0: /usr/sbin/rear:251 main
Trace 1: /usr/share/rear/lib/mkbackup-workflow.sh:22 WORKFLOW_mkbackup
Trace 2: /usr/share/rear/lib/framework-functions.sh:70 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:31 Source
Trace 4: /usr/share/rear/output/ISO/Linux-i386/82_create_iso_image.sh:17 source
Trace 5: /usr/share/rear/lib/_input-output-functions.sh:132 StopIfError
Message: Could not create ISO image (with /usr/bin/mkisofs)
===================
2015-04-24 12:34:54 Running exit tasks.
rmdir: removing directory, `/backups/rear/rear.RPwWr8Nk1dERlAI/outputfs'
2015-04-24 12:34:54 Finished in 965 seconds
2015-04-24 12:34:54 Removing build area /backups/rear/rear.RPwWr8Nk1dERlAI
rmdir: removing directory, `/backups/rear/rear.RPwWr8Nk1dERlAI/outputfs'
rmdir: failed to remove `/backups/rear/rear.RPwWr8Nk1dERlAI/outputfs': No such file or directory
rmdir: removing directory, `/backups/rear/rear.RPwWr8Nk1dERlAI'
2015-04-24 12:34:55 End of program reached
gdha commented at 2015-04-26 16:33:¶
See
http://stackoverflow.com/questions/20209263/genisoimage-script-cannot-burn-bigger-than-4gb-but-less-than-4-5gb-file
It was never the purpose of rear to create big ISO images, but with
the above link you could tweak rear to make your problem go away. Keep
in touch on your progress...
wwoo1 commented at 2015-04-27 19:41:¶
I added -allow-limited-size to $ISO_MKISOFS_BIN in
/usr/share/rear/output/ISO/Linux-i386/81_prepare_multiple_iso.sh and
the image size was able to go past 4GB, but it looks like 8.6GB is the
limit. I worked around it by tar'ing the large directories and manually
restoring them after recovery. Thanks for your help.
multipathmaster commented at 2015-09-01 13:48:¶
So yes, having -allow-limited-size to $ISO_MKISOFS_BIN will enable
the ISO image to become larger than 4.7 GIGs. However the real issue is
that in RHEL6/CENTOS6 mkisofs is actually a soft link back to
/etc/alternatives that points to genisoimage. The issue is that
genisomage is not infact mkisofs, with the REAL version of mkisofs being
a subset of a tools package called CDRTOOLS. Which is open source,
however because of certain limitations of working for a large
corporation, we do not have the ability to install (./configure make
make install) for the real binary version of mkisofs, so instead, you
can always add a new variable within the
/usr/share/rear/backup/NETFS/default/50* called SPLIT_COMMAND2 which
references the actual split differently than the normal $SPLIT_COMMAND
that is called under the tar function sequence of that script.
so inside 50_make_backup.sh you can put this at the top, or inside the
set_tar_features
SPLIT_COMMAND2="split --bytes=${BKUP_MAX_SIZE} - ${backuparchive}."
Or name it whatever you wish...
${BKUP_MAX_SIZE} is a variable that you set in /etc/rear/*.conf
(whichever conf file you wish to use, os.conf/site.conf/default.conf,
whatever you currently are running with rear).
cat /etc/rear/site.conf | grep BKUP_MAX_SIZE
BKUP_MAX_SIZE=3500MB
Now under the tar section of ${BACKUP_PROG} section of 50_make_backup.sh
# tar compatible programs here
(tar)
set_tar_features
Log $BACKUP_PROG $TAR_OPTIONS --sparse --block-number --totals --verbose \
--no-wildcards-match-slash --one-file-system \
--ignore-failed-read $BACKUP_PROG_OPTIONS \
$BACKUP_PROG_X_OPTIONS \
${BACKUP_PROG_BLOCKS:+-b $BACKUP_PROG_BLOCKS} $BACKUP_PROG_COMPRESS_OPTIONS \
-X $TMP_DIR/backup-exclude.txt -C / -c -f - \
$(cat $TMP_DIR/backup-include.txt) $LOGFILE \| $BACKUP_PROG_CRYPT_OPTIONS BACKUP_PROG_CRYPT_KEY \| $SPLIT_COMMAND2
$BACKUP_PROG $TAR_OPTIONS --sparse --block-number --totals --verbose \
--no-wildcards-match-slash --one-file-system \
--ignore-failed-read $BACKUP_PROG_OPTIONS \
$BACKUP_PROG_X_OPTIONS \
${BACKUP_PROG_BLOCKS:+-b $BACKUP_PROG_BLOCKS} $BACKUP_PROG_COMPRESS_OPTIONS \
-X $TMP_DIR/backup-exclude.txt -C / -c -f - \
$(cat $TMP_DIR/backup-include.txt) $LOGFILE | $BACKUP_PROG_CRYPT_OPTIONS $BACKUP_PROG_CRYPT_KEY | $SPLIT_COMMAND2
see how now I am calling on $SPLIT_COMMAND2 at the tail end of these 2 lines?
What does this do for RHEL6?
Your backup.tar.gz will go into your tmp directory (you can define this
as well, or leave it default if you think you have enough room), and it
will only allow the backup.tar.gz do grow to 3.5 GIGS, and then it
splits it off and creates a new one, and you wind up with a naming
convention such as this --> backup.tar.gz.aa
backup.tar.gz.ab
backup.tar.gz.ac
backup.tar.gz.ad
so on and so forth. This will keep the backup.tar.gz from exceeding the 4GiB hard limit that is set by the false mkisofs that is really a pointer back to genisoimage. Then once the ISO is created, it can grow to as big as it needs, while copying the backup.tar.gz files within the ISO.
Once it is time for recovery, the recovery section utilizes a for i in
ls ${BUILDDIR}/path/to/backups/*.tar.gz which finds the path to those
backup.tar.gz and because of globbing within a native shell it should
extract them in the same order as it had created them. Although some
testing needs to be done.
This simple step will allow much larger than 4 gig ISO images, but keeps the actual NETFS tar.gz files smaller than 4GIB so that you avoid that pesky error about the file size being too large.
gdha commented at 2016-02-19 09:33:¶
Added a note to the FAQ of rear (on the web pages)
Br0adax3 commented at 2016-10-05 23:00:¶
Uhm. I have run into this problem with an installation on RHEL7.2, using the version that comes with the OS (1.17.2). I have modified the scripts as suggested here, and still got the error. The problem file in my case is ./isolinux/initrd.cgz, which has grown to some 5.8G. Any suggestions?
Thanks.
jsmeix commented at 2016-10-06 09:15:¶
I have a general question:
Currently I cannot imagine a good reason why
one would like to have the backup.tar.gz within the ISO?
"BACKUP_URL=iso://" is mentioned in the documentation
but without a reason when this could be useful or to what
extent this is meant to be used.
What is the advantage to mix up the backup.tar.gz
with the rear recovery system ISO image?
Why not keep separated things separated?
Furthermore in general regarding huge backup.tar.gz see
https://github.com/rear/rear/issues/1006#issuecomment-248862040
Finally in general regarding issues with the backup:
Relax-and-Recover (rear) is neither a backup software nor a
backup management software and it is not meant to be one, cf.
"Relax-and-Recover (rear) versus backup and restore" in
https://en.opensuse.org/SDB:Disaster_Recovery
In particular in this case here rear only calls an external tool
that makes an ISO image in the usual way as it is normally
intended for rear i.e. an ISO image that contains only
the rear recovery system (up to about 500 MB)
or at most an ISO image that contains the rear recovery
system plus a backup of the files of the basic operating
system (up to very few GB - usually less than 4 GB)
but not to also a backup of huge application files
like databases and things like that.
[Export of Github issue for rear/rear.]