#2648 Issue `closed`: OUTPUT=USB bug summary¶

Labels: enhancement, bug, cleanup, no-issue-activity

DEvil0000 opened issue at 2021-07-05 16:38:¶

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.6 / 2020-06-17
OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
Ubuntu 20.04.2, kernel 5.4.0-77-generic
Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
apu
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86_64
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
coreboot
Description of the issue (ideally so that others can reproduce it):
I did start to dig a bit deeper in output usb mode and found more then one bug with this method. To reduce effort of tracking I think a summary ticket does make sense. So this ticket is meant to provide this summary.

The only output=usb settings working fine for me are:

UEFI + GPT (if uefi is supported by the system)

General improvements possible for output=usb:

add grub bootloader option
reuse output=rawdisk code (when grub is not used this may make sense)
reuse output=iso code (grub can boot from iso)
to me it is not clear what should be done by format and what by mkrescue but I would consider moving the bootloader writing to format command
hybrid gpt/mbr could be used if wanted
hybrid efi/legacy could be used if wanted

bugs with the current version:

in case of EFI it always creates a gpt partition - msdos is just not handled (300_format_usb_disk.sh:34)
gpt partitions are not created correctly when NO EFI is used.
** the bootloader must be within a gpt partition with type ef02 BIOS boot partition (this partition must have bios_grub flag set)
** it would make perfect sense to create a separate boot partition containing the system image (like it is done for efi)
the flag "legacy_boot" must be configurable since some systems may want it and some don't like it (from my understanding it is normally not used but if it is used then for the /boot partition)
the extlinux/syslinux boot config is not correct for my APU board with coreboot. Not sure what is wrong there but since output=rawdisk creates a working one it must be something there. (no matter if msdos or gpt is used) #2644

jsmeix commented at 2021-07-06 12:27:¶

We had already
https://github.com/rear/rear/issues/764
Switch to grub2 (grub-mkrescue) for creating the recovery ISO image
but noone contributed the needed code to ReaR
so it got closed because of no-issue-activity.

@pcahyna
if time permits could you have a look here
and provide some general comment?
Thank you in advance!

jsmeix commented at 2021-07-27 09:18:¶

With also https://github.com/rear/rear/pull/2650
and https://github.com/rear/rear/pull/2661 merged
at least some of the issues mentioned here should be (hopefully) fixed.

I can neither test coreboot nor serial console
because I do not have matching hardware
but I will test how far OUTPUT=USB works for me
on my older x86_64 laptop with traditional BIOS.

jsmeix commented at 2021-07-27 09:49:¶

It still works for me with my setup as in
https://github.com/rear/rear/pull/2659#issuecomment-886692737
and now even booting the existing system from the built-in local harddisk
works via the GRUB2 menu entry Boot next disk
because now the created boot/grub2/grub.cfg contains

menuentry "Boot next disk" {
    insmod chain
    set root=(hd1)
    chainloader +1
}

jsmeix commented at 2021-07-27 10:05:¶

I think the function create_grub2_cfg() is still somewhat messy
because my created boot/grub2/grub.cfg contains

search --no-floppy --file /boot/efiboot.img --set
search --no-floppy --set=root --label MY-BOOT

I think it does not make sense to have several GRUB2 search commands
at the same place (I guess only the last one matters in this case)
in contrast to having several GRUB2 search commands
each one in its own separated menuentry as in
the "Multi-boot manual config" section of the "GNU GRUB Manual"
https://www.gnu.org/software/grub/manual/grub/grub.html#Multi_002dboot-manual-config

I think I have to clean up our various methods how to set up
the GRUB environment variable root via different variables
grub2_set_root and GRUB2_SET_USB_ROOT for the same thing
plus a hardcoded search --no-floppy --file /boot/efiboot.img --set

DEvil0000 commented at 2021-07-27 16:49:¶

To give an update on the current state from my point of view:
All bugs and shot commings should be fixed now. Maybe there is still one left around the serial configuration.

There are still some improvements possible without any priority to me:

reuse output=iso code and optionally store a iso in the boot partition instead of the initrd (grub can boot from iso)
hybrid gpt/mbr could be used (see gdisk)
hybrid efi/legacy could be used

I will re-test master as soon as possible what may be next week.
Thank you very much for your help with those changes

pcahyna commented at 2021-07-27 17:10:¶

* grub can boot from iso

I am interested how to do it - the only procedure I found is to let Grub use the kernel and initrd contained in the ISO ( like http://www.panticz.de/MultiBootUSB ) but not to chainload the bootloader found on the ISO.

jsmeix commented at 2021-07-28 06:39:¶

@pcahyna
thank you that you joined here because I need your GRUB2 knowledge:

Could you tell whether or not my assumption in
https://github.com/rear/rear/issues/2648#issuecomment-887383494
is true - i.e. whether or not it makes sense to have more than one
search command in the general section (i.e. outside of a menuentry)
in grub.cfg?

Could you also have a look at
https://github.com/rear/rear/pull/2662
there in particulat at my FIXME: comment
in usr/share/rear/lib/bootloader-functions.sh
about whether or not --set=esp makes sense?

jsmeix commented at 2021-07-28 06:48:¶

My goal behind adding GRUB2 support for OUTPUT=USB is
that ReaR uses only GRUB2 as recovery system bootloader
so that we can drop using syslinux
cf. https://github.com/rear/rear/issues/764

pcahyna commented at 2021-07-28 07:41:¶

@jsmeix sorry that I haven't replied earlier. I suppose your question is whether the second search will overwrite the result of the first search so the first one becomes useless? I don't know the answer off-hand, I will try to find out.

jsmeix commented at 2021-07-28 09:19:¶

@pcahyna no need to be sorry - we all have various day job things to do.

Yes, that is my question.
When my assumption is true I would like to consolidate our current
various methods how to set up the GRUB environment variable root
into one single method and use only that method everywhere
so that we get one single search command in our grub.cfg
that is the right one for the intended purpose of that particular grub.cfg

In particular the current hardcoded

search --no-floppy --file /boot/efiboot.img --set

causes pain in my eyes and even more in my brain
because that cannot be right for BIOS and
it is also not right for OUTPUT=USB because
as far as I see boot/efiboot.img is created by
output/ISO/Linux-i386/700_create_efibootimg.sh
but that is only run for OUTPUT=ISO and it actually creates it
as isofs/boot/efiboot.img only if USING_UEFI_BOOTLOADER is true.

DEvil0000 commented at 2021-07-28 09:59:¶

@pcahyna for grub boot from iso you basically configure a loop device as source for the kernel and initrd (including path within the ISO).
Maybe you can also chainload to it but I am not sure about that.

see:
https://wiki.ubuntuusers.de/GRUB_2/Skripte/
https://help.ubuntu.com/community/Grub2/ISOBoot
https://www.linux-community.de/blog/iso-images-mit-grub-2-booten/

edit:
the idea behind this is that the booting source would always be the same iso. No matter if you boot via ipmi or usb stick recover media.

pcahyna commented at 2021-07-28 13:05:¶

@DEvil0000 thanks, that's what I thought. I would then say that doing it this way is needlessly complicated (a change of naming convention for the kernel and initrd would break the process) and does not have big advantage compared to putting the kernel+initrd in the filesystem directly. (You would not be reusing the bootloader and its configuration files from the ISO, so there would not be much to reuse.)

jsmeix commented at 2021-07-28 13:47:¶

I have a generic question regarding "grub can boot from iso":

Does "grub can boot from iso" mean something different
than using GRUB2 as bootloader on an ISO as described
in the section "Making a GRUB bootable CD-ROM" in
https://www.gnu.org/software/grub/manual/grub/grub.html
?

pcahyna commented at 2021-07-28 16:36:¶

@jsmeix no, we mean chainloading a bootable ISO image (image as a file). The "Making a GRUB bootable CD-ROM" deals with creating an ISO image that will be then used to burn a CD-ROM and one will boot from the CD-ROM. We are instead interested in having only an ISO image file accessible to GRUB loaded from the harddisk (or USB disk) and booting off that image file, if possible as if BIOS had booted a CD-ROM created from this image.

DEvil0000 commented at 2021-07-28 17:14:¶

@jsmeix I think it gets more clear with an example grub menu entry:

menuentry "Ubuntu 20.04 ISO" {
   set isofile="/boot/ubuntu-20.04-desktop-amd64.iso"
   rmmod tpm
   loopback loop (hd0,5)$isofile
   linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile noprompt noeject
   initrd (loop)/casper/initrd
}

I think it may also be possible to chainload from the iso instead and reuse the bootloader from there with something like:
(not tested and did not find documentation about this)

menuentry "Ubuntu 20.04 ISO" {
   set isofile="/boot/ubuntu-20.04-desktop-amd64.iso"
   rmmod tpm
   loopback loop (hd0,5)$isofile
   chainloader (loop)
# or chainloader (loop)$efipath
}

So everything recovery system related would be one single iso file which is the same for ipmi inserted or on a usb drive.
I could also imagine to write a grub mod for wget of an iso so one would not need to mess with PXE boot - which is just not possible in our customer networks.
Maybe this explains my usecase better.

edit: there is a tftp module since PXE needs it - this is maybe also a option for getting a iso without a full PXE boot (no dhcp, no dns or such).
linux (tftp,a.b.c.d)/tftpboot/bzImage or similar may work

pcahyna commented at 2021-07-28 17:20:¶

I think it may also be possible to chainload from the iso instead and reuse the bootloader from there with something like:

I don't think it is possible. I searched and found docs to do it only for grub4dos (a separate GRUB fork) and Syslinux ( https://wiki.syslinux.org/wiki/index.php?title=Boot_an_Iso_image and https://wiki.syslinux.org/wiki/index.php?title=MEMDISK#ISO_images ) - but the goal was to get rid of Syslinux, not to add another dependency on it. (@lzaoral, this may be of interest for you for testing purposes though).

jsmeix commented at 2021-07-29 10:09:¶

Regarding more than one 'search' command in grub.cfg
https://github.com/rear/rear/issues/2648#issuecomment-887383494
I asked our SUSE GRUB maintainer and he replied (excerpts)

> Currently our generated grub.cfg contains (excerpt)
> --------------------------------------------------------------
> search --no-floppy --file /boot/efiboot.img --set
> search --no-floppy --set=root --label MY-BOOT
> --------------------------------------------------------------
>
> The first 'search' may not set 'root' because /boot/efiboot.img
>  may not exist.

Yes.

> The second 'search' may not set 'root' because nothing with
> filesystem label 'MY-BOOT' may exist.

Yes.

> I assume the second 'search' overwrites 'root' if it was set
> by the first 'search'.

Yes.

> I assume when /boot/efiboot.img exists but no 'MY-BOOT' exists
> then 'root' is still set to what the first 'search' had set.
> Or will 'root' be re-set to some default/fallback value when the
> second 'search' cannot find a filesystem with label 'MY-BOOT'?

No it won't be re-set if the second search fails, so the value is
retained. That basically will do the work if you want the last found
wins.

jsmeix commented at 2021-07-29 12:52:¶

With https://github.com/rear/rear/pull/2662 merged
we (hopefully) moved one more step forward with
GRUB2 as bootloader for OUTPUT=USB.

jsmeix commented at 2021-07-29 12:54:¶

@DEvil0000 @pcahyna
I am not in the office until next Tuesday so
I wish you a relaxed and recovering weekend!

github-actions commented at 2021-09-28 02:11:¶

Stale issue message

jsmeix commented at 2021-10-15 10:39:¶

This needs to be done before ReaR 2.7 release.
I think it is already mostly done.
I think only some config variable names need to be clarified.
See also https://github.com/rear/rear/issues/2666

github-actions commented at 2021-12-15 02:14:¶

Stale issue message

github-actions commented at 2022-02-21 02:09:¶

Stale issue message

github-actions commented at 2022-04-23 02:49:¶

Stale issue message

jsmeix commented at 2022-06-01 11:31:¶

I postponed https://github.com/rear/rear/issues/2666
to ReaR 2.8

Via https://github.com/rear/rear/pull/2815
I describe in default.conf the currently
implemented user config variable names
USB_BOOT_PART_SIZE and USB_DEVICE_BOOT_LABEL

I think the current user config variable names
USB_BOOT_PART_SIZE and USB_DEVICE_BOOT_LABEL
are OK - at least I don't know really better names.

This issue is no longer a blocker for ReaR 2.8
but https://github.com/rear/rear/pull/2815 is.

I postpone this issue to ReaR 2.8.