#2719 Issue closed: OUTPUT=ISO image cannot be dumped as is (with 'dd') onto a USB stick¶
Labels: support / question
bobOnGitHub opened issue at 2021-11-21 23:20:¶
TLDR:
After run mkrescue
[root@localhost output]# sfdisk -d /var/lib/rear/output/rear-localhost.iso
sfdisk: /var/lib/rear/output/rear-localhost.iso: does not contain a recognized partition table
[root@localhost output]#
-
ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.6 / 2020-06-17version installed from repo 21/11/21 (dnf install rear) -
OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
NAME="Rocky Linux"
VERSION="8.4 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.4"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.4 (Green Obsidian)"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:rocky:rocky:8.4:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky Linux"
ROCKY_SUPPORT_PRODUCT_VERSION="8"
- ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):
[root@localhost output]# cat /etc/rear/local.conf
# This file etc/rear/local.conf is intended for the user's
# manual configuration of Relax-and-Recover (ReaR).
# For configuration through packages and other automated means
# we recommend a separated file named site.conf next to this file
# and leave local.conf as is (ReaR upstream will never ship a site.conf).
# The default OUTPUT=ISO creates the ReaR rescue medium as ISO image.
# You need to specify your particular backup and restore method for your data
# as the default BACKUP=REQUESTRESTORE does not really do that (see "man rear").
# Configuration variables are documented in /usr/share/rear/conf/default.conf
# and the examples in /usr/share/rear/conf/examples/ can be used as templates.
# ReaR reads the configuration files via the bash builtin command 'source'
# so bash syntax like VARIABLE="value" (no spaces at '=') is mandatory.
# Because 'source' executes the content as bash scripts you can run commands
# within your configuration files, in particular commands to set different
# configuration values depending on certain conditions as you need like
# CONDITION_COMMAND && VARIABLE="special_value" || VARIABLE="usual_value"
# but that means CONDITION_COMMAND gets always executed when 'rear' is run
# so ensure nothing can go wrong if you run commands in configuration files.
#
OUTPUT=ISO
BACKUP=REQUESTRESTORE
[root@localhost output]#
-
Hardware vendor/product (PC or PowerNV BareMetal or ARM) or VM (KVM guest or PowerVM LPAR):
PC
Dell Precision T7810 -
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86 compatible -
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
System was created in an older machine and disk relocated.
System boot is Grub2
Hardware is Legacy/UEFI -
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
INTERNAL 1TB HDD SATA III -
Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT"):
System is /dev/sda
[root@localhost output]# lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT
NAME KNAME PKNAME TRAN TYPE FSTYPE SIZE MOUNTPOINT
/dev/loop0 /dev/loop0 loop squashfs 55.5M /var/lib/snapd/snap/core18/2246
/dev/loop1 /dev/loop1 loop squashfs 32.3M /var/lib/snapd/snap/snapd/13170
/dev/loop2 /dev/loop2 loop squashfs 217.4M /var/lib/snapd/snap/code/80
/dev/loop3 /dev/loop3 loop squashfs 4K /var/lib/snapd/snap/bare/5
/dev/loop4 /dev/loop4 loop squashfs 99.4M /var/lib/snapd/snap/core/11993
/dev/loop5 /dev/loop5 loop squashfs 32.5M /var/lib/snapd/snap/snapd/13640
/dev/loop6 /dev/loop6 loop squashfs 164.8M /var/lib/snapd/snap/gnome-3-28-1804/161
/dev/loop7 /dev/loop7 loop squashfs 65.2M /var/lib/snapd/snap/gtk-common-themes/1519
/dev/loop8 /dev/loop8 loop squashfs 216.5M /var/lib/snapd/snap/code/78
/dev/loop9 /dev/loop9 loop squashfs 99.3M /var/lib/snapd/snap/core/11743
/dev/loop10 /dev/loop10 loop squashfs 83.4M /var/lib/snapd/snap/beekeeper-studio/115
/dev/loop11 /dev/loop11 loop squashfs 83.4M /var/lib/snapd/snap/beekeeper-studio/113
/dev/loop12 /dev/loop12 loop squashfs 131.9M /var/lib/snapd/snap/labplot/39
/dev/loop13 /dev/loop13 loop squashfs 323.5M /var/lib/snapd/snap/kde-frameworks-5-qt-5-15-core20/14
/dev/loop14 /dev/loop14 loop squashfs 964.5M /var/lib/snapd/snap/android-studio/115
/dev/loop15 /dev/loop15 loop squashfs 61.9M /var/lib/snapd/snap/core20/1169
/dev/loop16 /dev/loop16 loop squashfs 55.4M /var/lib/snapd/snap/core18/2128
/dev/sda /dev/sda sata disk 931.5G
|-/dev/sda1 /dev/sda1 /dev/sda part xfs 1G /boot
`-/dev/sda2 /dev/sda2 /dev/sda part crypto_LUKS 930.5G
`-/dev/mapper/luks-bdf533b2-afca-4eda-8029-e0346ebaea99 /dev/dm-0 /dev/sda2 crypt LVM2_member 930.5G
|-/dev/mapper/rl-root /dev/dm-1 /dev/dm-0 lvm xfs 70G /
|-/dev/mapper/rl-swap /dev/dm-2 /dev/dm-0 lvm swap 7.8G [SWAP]
`-/dev/mapper/rl-home /dev/dm-3 /dev/dm-0 lvm xfs 852.8G /home
/dev/sdb /dev/sdb sata disk 931.5G
`-/dev/sdb1 /dev/sdb1 /dev/sdb part crypto_LUKS 931.5G
`-/dev/mapper/luks-79851d3a-c64a-4264-9f51-af166f098786 /dev/dm-5 /dev/sdb1 crypt xfs 931.5G
/dev/sdc /dev/sdc usb disk 931.5G
`-/dev/sdc1 /dev/sdc1 /dev/sdc part ntfs 931.5G /run/media/bob/Transend
/dev/sr0 /dev/sr0 sata rom 1024M
[root@localhost output]#
- Description of the issue (ideally so that others can reproduce
it):
attempt make the rescue image:
[root@localhost output]# rear mkrescue -v
Relax-and-Recover 2.6 / 2020-06-17
Running rear mkrescue (PID 257628)
Using log file: /var/log/rear/rear-localhost.log
Running workflow mkrescue on the normal/original system
Using autodetected kernel '/boot/vmlinuz-4.18.0-305.25.1.el8_4.x86_64' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda)
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct ...
Creating recovery system root filesystem skeleton layout
Skipping 'virbr0': not bound to any physical interface.
To log into the recovery system via ssh set up /root/.ssh/authorized_keys or specify SSH_ROOT_PASSWORD
Copying logfile /var/log/rear/rear-localhost.log into initramfs as '/tmp/rear-localhost-partial-2021-11-21T22:21:45+00:00.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/4.18.0-305.25.1.el8_4.x86_64 (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
Symlink '/usr/lib/modules/4.18.0-305.25.1.el8_4.x86_64/source' -> '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' via the 'COPY_AS_IS' configuration variable.
Symlink '/usr/lib/modules/4.18.0-305.25.1.el8_4.x86_64/build' -> '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' via the 'COPY_AS_IS' configuration variable.
Testing that the recovery system in /tmp/rear.Q8MwLoymjYrPSYt/rootfs contains a usable system
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (386620782 bytes) in 37 seconds
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-localhost.iso (381M)
Exiting rear mkrescue (PID 257628) and its descendant processes ...
Running exit tasks
[root@localhost output]#
Check the iso image :
[root@localhost output]# sfdisk -d /var/lib/rear/output/rear-localhost.iso
sfdisk: /var/lib/rear/output/rear-localhost.iso: does not contain a recognized partition table
[root@localhost output]#
-
Workaround, if any:
-
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
[root@localhost ~]# rear -D mkrescue
Relax-and-Recover 2.6 / 2020-06-17
Running rear mkrescue (PID 285319)
Using log file: /var/log/rear/rear-localhost.log
Running workflow mkrescue on the normal/original system
Using autodetected kernel '/boot/vmlinuz-4.18.0-305.25.1.el8_4.x86_64' as kernel in the recovery system
Creating disk layout
Overwriting existing disk layout file /var/lib/rear/layout/disklayout.conf
Using guessed bootloader 'GRUB' (found in first bytes on /dev/sda)
Verifying that the entries in /var/lib/rear/layout/disklayout.conf are correct ...
Creating recovery system root filesystem skeleton layout
Handling network interface 'enp0s25'
enp0s25 is a physical device
Handled network interface 'enp0s25'
Skipping 'virbr0': not bound to any physical interface.
To log into the recovery system via ssh set up /root/.ssh/authorized_keys or specify SSH_ROOT_PASSWORD
Copying logfile /var/log/rear/rear-localhost.log into initramfs as '/tmp/rear-localhost-partial-2021-11-21T22:58:42+00:00.log'
Copying files and directories
Copying binaries and libraries
Copying all kernel modules in /lib/modules/4.18.0-305.25.1.el8_4.x86_64 (MODULES contains 'all_modules')
Copying all files in /lib*/firmware/
Skip copying broken symlink '/etc/mtab' target '/proc/299036/mounts' on /proc/ /sys/ /dev/ or /run/
Symlink '/usr/lib/modules/4.18.0-305.25.1.el8_4.x86_64/source' -> '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' via the 'COPY_AS_IS' configuration variable.
Symlink '/usr/lib/modules/4.18.0-305.25.1.el8_4.x86_64/build' -> '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' refers to a non-existing directory on the recovery system.
It will not be copied by default. You can include '/usr/src/kernels/4.18.0-305.25.1.el8_4.x86_64' via the 'COPY_AS_IS' configuration variable.
Testing that the recovery system in /tmp/rear.6zrNXSX4YyShRRr/rootfs contains a usable system
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (386669275 bytes) in 36 seconds
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-localhost.iso (381M)
Exiting rear mkrescue (PID 285319) and its descendant processes ...
Running exit tasks
You should also rm -Rf --one-file-system /tmp/rear.6zrNXSX4YyShRRr
[root@localhost ~]#
/var/log/rear/rear-localhost.log
File attached....
pcahyna commented at 2021-11-22 11:35:¶
rear-localhost.iso: does not contain a recognized partition table
Why should the ISO contain a partition table? I believe that only on PowerPC the CD-ROMs are partitioned.
hpannenb commented at 2021-11-23 16:16:¶
@bobOnGitHub What would You like to achieve with Your mentioned ReaR config? What is the struggle You determine? It would be great You provide some more details on this issue.
bobOnGitHub commented at 2021-11-23 17:26:¶
Ok,
Why should the ISO contain a partition table? I believe that only on PowerPC the CD-ROMs are partitioned.
In recent years any bootable image tagged as iso I've downloaded
contained partitions and could simply be
dd'd to a usb stick - no flapping around with bootable-usb-maker tools,
endless commandline typing or burning cds/dvds.
Eg.
[bob@localhost OS]$ ls -lh CentOS-7-x86_64-DVD-1804.iso
-rwxrwxrwx. 1 bob bob 4.2G Aug 2 2018 CentOS-7-x86_64-DVD-1804.iso
[bob@localhost OS]$ sfdisk -d CentOS-7-x86_64-DVD-1804.iso
label: dos
label-id: 0x4c545524
device: CentOS-7-x86_64-DVD-1804.iso
unit: sectors
CentOS-7-x86_64-DVD-1804.iso1 : start= 0, size= 8730624, type=0, bootable
CentOS-7-x86_64-DVD-1804.iso2 : start= 2368, size= 17976, type=ef
[bob@localhost OS]$
[bob@localhost Downloads]$ sfdisk -d Rocky-8.4-x86_64-dvd1.iso
label: dos
label-id: 0x4a14a4ee
device: Rocky-8.4-x86_64-dvd1.iso
unit: sectors
Rocky-8.4-x86_64-dvd1.iso1 : start= 0, size= 19351552, type=0, bootable
Rocky-8.4-x86_64-dvd1.iso2 : start= 23488, size= 19576, type=ef
[bob@localhost Downloads]$ sfdisk -d kubuntu-20.04.3-desktop-amd64.iso
label: dos
label-id: 0x4b11d83a
device: kubuntu-20.04.3-desktop-amd64.iso
unit: sectors
kubuntu-20.04.3-desktop-amd64.iso1 : start= 0, size= 6258688, type=0, bootable
kubuntu-20.04.3-desktop-amd64.iso2 : start= 4567600, size= 8000, type=ef
[bob@localhost Downloads]$ sfdisk -d Rocky-8.5-x86_64-dvd1.iso
label: dos
label-id: 0x06729585
device: Rocky-8.5-x86_64-dvd1.iso
unit: sectors
Rocky-8.5-x86_64-dvd1.iso1 : start= 0, size= 20930560, type=0, bootable
Rocky-8.5-x86_64-dvd1.iso2 : start= 23944, size= 19616, type=ef
In fact, it is so long since I burned an iso image to a physical cd/dvd that I had forgotten all about traditional iso formats and growisofs. Who does that any more ? There is no reason to: keep an image like the ones above with/in the backup 'till you need it, extract it when/if you do, dd it to a usb stick or copy it to your vm storage and away you go. I don't believe I've bought DVD RWs in over 10 years.
In any case thanks for the heads up @pcahyna - that explains that problem and I can report the iso image does work when growisofs'd to a physical disk or loaded into a vm as a cdrom... though I haven't managed to recreate my disk/system yet.
I will close the issue - it was a user misunderstanding. You might want to ammend the documentation to point out the ISO is not dd-able to a usb as people might expect today - admittedly few Linux admins are likely to get caught out there but desktop end-users trying to escape Windows probably will.
@bobOnGitHub What would You like to achieve with Your mentioned ReaR config? What is the struggle You determine? It would be great You provide some more details on this issue.
Thanks @hpannenb, as you can see this particular issue is resolved/was a misunderstanding on my part. I'm not sure at this point ReaR is right for me - I'm still looking at it though I may have a simpler and more sustainable/maintainable solution for my own particular circumstances.
pcahyna commented at 2021-11-23 21:52:¶
In recent years any bootable image tagged as iso I've downloaded contained partitions and could simply be dd'd to a usb stick - no flapping around with bootable-usb-maker tools, endless commandline typing or burning cds/dvds.
Thanks for the explanation. There is a separate OUTPUT=RAWDISK method
that produces disk images to dd to USB flash drives and other disk-like
devices, although I have never used it. There is also OUTPUT=USB,
which saves the backup to a mounted USB drive and thus avoids producing
an image file.
As you can see, it is not a real bug, but to unify RAWDISK and ISO
methods in order to produce images that can be used both for regular
disks and DVDs is a valid feature to ask for (although I admit it would
not be on top of my personal priority list right now).
bobOnGitHub commented at 2021-11-23 22:54:¶
Cheers, I did try to dd the USB output to a usb stick which didn't work either - I can't recall exactly what I did but it wasn't to a mounted drive so that explains that (and in retrospect I should have picked that up from the man pages). Useful to know.
As to the RAWDISK option, it didn't work for me but could be any number of reasons.
I tried :
[root@localhost rear]# cat local.conf
OUTPUT=RAWDISK
BACKUP=REQUESTRESTORE
OUTPUT_URL=file:///var/lib/rear/output/
with rear -v mkrescue
note if you do not specify the OUTPUT_URL it does not default as per
the man pages. What
happens (using -v) is that you are told about the file but not where it
is. Checking the logs you find it was written
to a temp dir ... where it no longer exists. In any case there is no
feedback to the user that anything is amiss other than the absence of
the expected file so you might want to look at that.
Anyway, having specified the OUTPUT_URL and then followed the subsequent README advice to the letter and rebooting the PC, selecting the USB device (a usb stick) I got a "Selected boot device failed. Press any key to reboot the sytem" message. Who knows / might have been a fluke bad write etc. but it dawned on me about then how the ISO option was supposed to work and I went back to check that and got something running to see how the next stages worked..
(Also the resulting RAWDISK USB is labelled with a space in the name "RESCUE SYS" which is not ideal.)
Using the ISO/DVD option:
note that when it came to setting up LUKS it asked for a passphrase and
then appeared to hang - I killed it on a vm but later just happened to
leave it run on the PC thinking it'd still be blinking when I got back
(from getting a coffee) but it had finally moved on and was asking for
the passphrase again - don't think it was my disks as the vm and bare
metal were different disks and all the s.m.a.r.t. info says they're
good. It did set up the disk layout, though as I say, I haven't got a
system recreated yet - I won't bore you with the details at this point /
too many variables there to be useful or worth thinking about.
Anyway, I'm "researching" at the moment so I'm off on another tangent now as rsync looks a better option overall than Duplicity - which is what I was looking at.
jsmeix commented at 2021-11-24 07:14:¶
It seems in some cases it even works to some limited extent
to dump the OUTPUT=ISO image onto a (USB) disk, see
https://github.com/rear/rear/issues/2712
that reads (excerpts)
OUTPUT=ISO
...
PC ... x86_64 ... UEFI
...
* Description of the issue (ideally so that others can reproduce it):
ISO image is created. I write it a USB stick with dd if=<image_file> of=/dev/sdb
...
The image gets written to the USB drive and the USB drive is bootable,
but it just boots to a prompt. I can login as root without a password and then
manually start a restore but I've always seen a menu come up that shows
the backups and the option for automated (well sort of) restore.
hpannenb commented at 2021-11-24 08:47:¶
@bobOnGitHub Using ReaR's "default" configuration please bare in mind it just creates a bootable rescue image (only) from the existing system. So to recover a system entirely You need to configure a backup method (e.g. NETFS) as well.
Unfortunately there is no default configuration described yet but please consider taking a look at https://github.com/rear/rear/blob/master/doc/user-guide/04-scenarios.adoc .
I remember using ReaR for the first time was not really "out-of-the-box"-ish.
From the config I am sometimes using:
export TMPDIR="/var/tmp/"
BACKUP=NETFS
BACKUP_URL=iso://backup
OUTPUT=ISO
OUTPUT_URL=null
ISO_MAX_SIZE=4500
This will create a ISO file(s) (max. 4.5 GB) including the
(tar-ed)backup of the entire system including the bootable recovery
environment. The resulting file(s) can be found under
/var/lib/rear/output/.
jsmeix commented at 2021-11-24 09:35:¶
ReaR never was and is not "out-of-the-box"-ish and it is not meant to be
so
in foreseeable future with the manpower we currently have at ReaR
upstream and
according to what those who pay for ReaR (enterprise users) would like
to pay for.
See also the sections "Inappropriate expectations"
and "First steps with Relax-and-Recover" in
https://en.opensuse.org/SDB:Disaster_Recovery
gdha commented at 2021-11-24 12:44:¶
@bobOnGitHub Valid point and not complicated to implement according
https://wiki.syslinux.org/wiki/index.php?title=Isohybrid
and updating the script
usr/share/rear/output/ISO/Linux-i386/820_create_iso_image.sh - please
help us with improving our product by proving a pull request (PR). Of
course, make sure it works on all supported Linux Operating Systems.
Thanks
[Export of Github issue for rear/rear.]