#581 Issue closed: Cannot create ISO image >4GB on RHEL6

Labels: documentation, support / question

wwoo1 opened issue at 2015-04-24 21:21:

Hello... I could not backup to ISO images >4GB. I set ISO_MAX_SIZE=10000 and that did not help either. Thanks.

[root@teak rear]# rear -v mkbackup
Relax-and-Recover 1.17.0 / Git
Using log file: /var/log/rear/rear-teak.log
Creating disk layout
Creating root filesystem layout
TIP: To login as root via ssh you need to set up /root/.ssh/authorized_keys or SSH_ROOT_PASSWORD in your configuration file
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Encrypting disabled
Creating tar archive '/backups/rear/rear.RPwWr8Nk1dERlAI/tmp/isofs/backup/backup.tar.gz'
Archived 5533 MiB [avg 6112 KiB/sec]OK
Archived 5533 MiB in 928 seconds [avg 6106 KiB/sec]
Preparing 2 ISO images ...
Making additionnal ISO image : rear-teak_01.iso
Wrote ISO image: /backups/rear/rear-teak_01.iso (1.2G)
Making ISO image
ERROR: Could not create ISO image (with /usr/bin/mkisofs)
Aborting due to an error, check /var/log/rear/rear-teak.log for details
[root@teak rear]#

From /var/log/rear/rear-teak.log

2015-04-24 12:34:54 Wrote ISO image: /backups/rear/rear-teak_01.iso (1.2G)
2015-04-24 12:34:54 Including output/ISO/Linux-i386/82_create_iso_image.sh
2015-04-24 12:34:54 Starting '/usr/bin/mkisofs'
2015-04-24 12:34:54 Making ISO image
genisoimage 1.1.9 (Linux)
Scanning .
Scanning ./backup
File ./backup/backup.tar.gz.00 is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.
2015-04-24 12:34:54 ERROR: Could not create ISO image (with /usr/bin/mkisofs)
=== Stack trace ===
Trace 0: /usr/sbin/rear:251 main
Trace 1: /usr/share/rear/lib/mkbackup-workflow.sh:22 WORKFLOW_mkbackup
Trace 2: /usr/share/rear/lib/framework-functions.sh:70 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:31 Source
Trace 4: /usr/share/rear/output/ISO/Linux-i386/82_create_iso_image.sh:17 source
Trace 5: /usr/share/rear/lib/_input-output-functions.sh:132 StopIfError
Message: Could not create ISO image (with /usr/bin/mkisofs)
2015-04-24 12:34:54 Running exit tasks.
rmdir: removing directory, `/backups/rear/rear.RPwWr8Nk1dERlAI/outputfs'
2015-04-24 12:34:54 Finished in 965 seconds
2015-04-24 12:34:54 Removing build area /backups/rear/rear.RPwWr8Nk1dERlAI
rmdir: removing directory, `/backups/rear/rear.RPwWr8Nk1dERlAI/outputfs'
rmdir: failed to remove `/backups/rear/rear.RPwWr8Nk1dERlAI/outputfs': No such file or directory
rmdir: removing directory, `/backups/rear/rear.RPwWr8Nk1dERlAI'
2015-04-24 12:34:55 End of program reached

gdha commented at 2015-04-26 16:33:

See http://stackoverflow.com/questions/20209263/genisoimage-script-cannot-burn-bigger-than-4gb-but-less-than-4-5gb-file
It was never the purpose of rear to create big ISO images, but with the above link you could tweak rear to make your problem go away. Keep in touch on your progress...

wwoo1 commented at 2015-04-27 19:41:

I added -allow-limited-size to $ISO_MKISOFS_BIN in /usr/share/rear/output/ISO/Linux-i386/81_prepare_multiple_iso.sh and the image size was able to go past 4GB, but it looks like 8.6GB is the limit. I worked around it by tar'ing the large directories and manually restoring them after recovery. Thanks for your help.

multipathmaster commented at 2015-09-01 13:48:

So yes, having -allow-limited-size to $ISO_MKISOFS_BIN will enable the ISO image to become larger than 4.7 GIGs. However the real issue is that in RHEL6/CENTOS6 mkisofs is actually a soft link back to /etc/alternatives that points to genisoimage. The issue is that genisomage is not infact mkisofs, with the REAL version of mkisofs being a subset of a tools package called CDRTOOLS. Which is open source, however because of certain limitations of working for a large corporation, we do not have the ability to install (./configure make make install) for the real binary version of mkisofs, so instead, you can always add a new variable within the /usr/share/rear/backup/NETFS/default/50* called SPLIT_COMMAND2 which references the actual split differently than the normal $SPLIT_COMMAND that is called under the tar function sequence of that script.

so inside 50_make_backup.sh you can put this at the top, or inside the set_tar_features

SPLIT_COMMAND2="split --bytes=${BKUP_MAX_SIZE} - ${backuparchive}."

Or name it whatever you wish...

${BKUP_MAX_SIZE} is a variable that you set in /etc/rear/*.conf (whichever conf file you wish to use, os.conf/site.conf/default.conf, whatever you currently are running with rear).

cat /etc/rear/site.conf | grep BKUP_MAX_SIZE

Now under the tar section of ${BACKUP_PROG} section of 50_make_backup.sh

# tar compatible programs here
                Log $BACKUP_PROG $TAR_OPTIONS --sparse --block-number --totals --verbose \
                        --no-wildcards-match-slash --one-file-system \
                        --ignore-failed-read $BACKUP_PROG_OPTIONS \
                        $BACKUP_PROG_X_OPTIONS \
                        -X $TMP_DIR/backup-exclude.txt -C / -c -f - \
                        $(cat $TMP_DIR/backup-include.txt) $LOGFILE \| $BACKUP_PROG_CRYPT_OPTIONS BACKUP_PROG_CRYPT_KEY \| $SPLIT_COMMAND2
                $BACKUP_PROG $TAR_OPTIONS --sparse --block-number --totals --verbose \
                        --no-wildcards-match-slash --one-file-system \
                        --ignore-failed-read $BACKUP_PROG_OPTIONS \
                        $BACKUP_PROG_X_OPTIONS \
                        -X $TMP_DIR/backup-exclude.txt -C / -c -f - \
                        $(cat $TMP_DIR/backup-include.txt) $LOGFILE | $BACKUP_PROG_CRYPT_OPTIONS $BACKUP_PROG_CRYPT_KEY | $SPLIT_COMMAND2

see how now I am calling on $SPLIT_COMMAND2 at the tail end of these 2 lines?

What does this do for RHEL6?

Your backup.tar.gz will go into your tmp directory (you can define this as well, or leave it default if you think you have enough room), and it will only allow the backup.tar.gz do grow to 3.5 GIGS, and then it splits it off and creates a new one, and you wind up with a naming convention such as this --> backup.tar.gz.aa

so on and so forth. This will keep the backup.tar.gz from exceeding the 4GiB hard limit that is set by the false mkisofs that is really a pointer back to genisoimage. Then once the ISO is created, it can grow to as big as it needs, while copying the backup.tar.gz files within the ISO.

Once it is time for recovery, the recovery section utilizes a for i in ls ${BUILDDIR}/path/to/backups/*.tar.gz which finds the path to those backup.tar.gz and because of globbing within a native shell it should extract them in the same order as it had created them. Although some testing needs to be done.

This simple step will allow much larger than 4 gig ISO images, but keeps the actual NETFS tar.gz files smaller than 4GIB so that you avoid that pesky error about the file size being too large.

gdha commented at 2016-02-19 09:33:

Added a note to the FAQ of rear (on the web pages)

Br0adax3 commented at 2016-10-05 23:00:

Uhm. I have run into this problem with an installation on RHEL7.2, using the version that comes with the OS (1.17.2). I have modified the scripts as suggested here, and still got the error. The problem file in my case is ./isolinux/initrd.cgz, which has grown to some 5.8G. Any suggestions?


jsmeix commented at 2016-10-06 09:15:

I have a general question:

Currently I cannot imagine a good reason why
one would like to have the backup.tar.gz within the ISO?

"BACKUP_URL=iso://" is mentioned in the documentation
but without a reason when this could be useful or to what
extent this is meant to be used.

What is the advantage to mix up the backup.tar.gz
with the rear recovery system ISO image?

Why not keep separated things separated?

Furthermore in general regarding huge backup.tar.gz see

Finally in general regarding issues with the backup:

Relax-and-Recover (rear) is neither a backup software nor a
backup management software and it is not meant to be one, cf.
"Relax-and-Recover (rear) versus backup and restore" in

In particular in this case here rear only calls an external tool
that makes an ISO image in the usual way as it is normally
intended for rear i.e. an ISO image that contains only
the rear recovery system (up to about 500 MB)
or at most an ISO image that contains the rear recovery
system plus a backup of the files of the basic operating
system (up to very few GB - usually less than 4 GB)
but not to also a backup of huge application files
like databases and things like that.

[Export of Github issue for rear/rear.]