#3578 Issue open: Default boot entry is not preserved because grubenv is removed during recovery¶
Labels: enhancement, cleanup
svlv opened issue at 2026-03-05 15:44:¶
ReaR version¶
2.9
Describe the ReaR bug in detail¶
Hi!
I've experienced an issue with missing the saved boot entry after
recovery on Rocky 9, which was upgraded from version 9.6 to 9.7 . The
original system uses the latest installed kernel, but after recovery it
uses the the first boot entry from /boot/loader/entries/* in
alphabetical order.
Before recovery
- Current kernel version
$ uname -r
5.14.0-611.34.1.el9_7.x86_64
- Default boot entry - it is the latest kernel version on this system
$ sudo grubby --info=DEFAULT
index=2
kernel="/boot/vmlinuz-5.14.0-611.34.1.el9_7.x86_64"
args="rootdelay=300 console=ttyS0 earlyprintk=ttyS0 no_timer_check net.ifnames=0 crashkernel=1G-2G:192M,2G-64G:256M,64G-:512M"
root="UUID=8fac9486-4a35-45c8-9712-9c66ddc261eb"
initrd="/boot/initramfs-5.14.0-611.34.1.el9_7.x86_64.img"
title="Rocky Linux (5.14.0-611.34.1.el9_7.x86_64) 9.7 (Blue Onyx)"
id="5cf83cd1296b4ba88f9fc7e574b990f8-5.14.0-611.34.1.el9_7.x86_64"
saved_entryin/boot/grub2/grubenvcontains the latest kernel version. It is how it is saved and used by GRUB.
$ sudo cat /boot/grub2/grubenv
# GRUB Environment Block
# WARNING: Do not edit this file by tools other than grub-editenv!!!
boot_success=1
boot_indeterminate=0
saved_entry=5cf83cd1296b4ba88f9fc7e574b990f8-5.14.0-611.34.1.el9_7.x86_64
After recovery
- Current kernel version has been changed
$ uname -r
5.14.0-570.17.1.el9_6.x86_64
- The default boot entry has been changed. Now, it is just the first boot entry
$ sudo grubby --info=DEFAULT
index=0
kernel="/boot/vmlinuz-5.14.0-570.17.1.el9_6.x86_64"
args="rootdelay=300 console=ttyS0 earlyprintk=ttyS0 no_timer_check net.ifnames=0 crashkernel=1G-2G:192M,2G-64G:256M,64G-:512M"
root="UUID=8fac9486-4a35-45c8-9712-9c66ddc261eb"
initrd="/boot/initramfs-5.14.0-570.17.1.el9_6.x86_64.img"
title="Rocky Linux (5.14.0-570.17.1.el9_6.x86_64) 9.6 (Blue Onyx)"
id="26599199cb4a4d3db754be754593c5b0-5.14.0-570.17.1.el9_6.x86_64"
- The root cause -
saved_entryis no longer set in/boot/grub2/grubenv
$ sudo cat /boot/grub2/grubenv
# GRUB Environment Block
# WARNING: Do not edit this file by tools other than grub-editenv!!!
boot_indeterminate=2
boot_success=1
Platform¶
No response
OS version¶
Rocky Linux 9.7
Backup¶
No response
Storage layout¶
No response
What steps will reproduce the bug?¶
No response
Workaround, if any¶
Set BACKUP_RESTORE_MOVE_AWAY_FILES=( ) in local.conf
Additional information¶
Likely all systems using saved_entry in grubenv are affected.
However, systems that use grubenv from /boot/efi/EFI/*/grubenv are
safe.
svlv commented at 2026-03-05 15:58:¶
In https://github.com/rear/rear/issues/1828, @jsmeix explained the
reason why /boot/grub2/grubenv must be removed.
I suggest do not removing /boot/grub2/grubenv by default, but unset
all variables except for default and saved_entry.
E.g. grub2-editenv /boot/grub2/grubenv unset env_block - can be used
to unset variable and grub2-editenv list to list all variables.
I've been preparing the patch for that. But what do you think about that, is it worth creating a PR?
pcahyna commented at 2026-03-05 18:27:¶
@svlv thanks for looking into this. I also need a look at this, because we had more similar bug reports. The challenge has been, IIRC, how to regenerate GRUB configuration properly on different versions (at least RHEL versions). Will have a deeper look tomorrow.
svlv commented at 2026-03-06 07:17:¶
Why grub-mkconfig isn't run on EFI systems?
What I see, now grub-mkconfig is not run on EFI systems in
usr/share/rear/finalize/Linux-i386/660_install_grub2.sh because the
script finishes before that. There is no need to run grub-install on
EFI systems, but grub-mkconfig should be probably run there or not?
pcahyna commented at 2026-03-06 19:26:¶
@svlv my understanding is that indeed grub-mkconfig is entirely
independent of grub-install and therefore the decision of whether to
run does not depend on the firmware type at all. This is related to the
reason why I did not like GRUB2-BLS as a bootloader type: it expresses
the style of GRUB configuration, not installation (unlike other
bootloader types).
svlv commented at 2026-03-09 09:31:¶
@jsmeix
I've tried to reproduce the error: invalid environment block when
grubenv is preserved on SLES 15, but everything was OK. No issue after
recovery. The issue with grubenv is mentioned here -
https://github.com/rear/rear/issues/1828#issuecomment-399031356.
Is there an additional condition to reproduce it or it is not relevant any more?
My configuration:
OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL=nfs://192.168.0.3/backup
BACKUP_RESTORE_MOVE_AWAY_FILES=( )
Arch - x86_64, distro - SLES 15, Firmware - BIOS, VM is running in VMware
pcahyna commented at 2026-03-09 11:09:¶
@svlv sorry for the delay. I checked my notes and I would also like to
avoid removing grubenv, its removal causes other issues beyond what
you saw. Will provide further details.
jsmeix commented at 2026-03-09 12:35:¶
I think my reasoning in
https://github.com/rear/rear/issues/1828#issuecomment-399050741
is valid - if not please explain and correct me.
Therefore I don't like to keep a file which is meaningless
on a system which is (re)-created anew from scratch.
I think the right way is to configure the default boot entry
anew from scratch, e.g. by using grub-set-default as
https://www.gnu.org/software/grub/manual/grub/html_node/saved_005fentry.html
tells.
So it is a valid issue that
"Default boot entry is not preserved"
but I think the reason why
"because grubenv is removed during recovery"
is not right because I think grubenv is not the right way
how to recreate the default boot entry.
Think about a system which had two boot entries
"THIS" and "THAT" and "THAT" was the default boot entry
but during "rear recover" only "THIS" gets recreated
(for whatever reason).
Then saved_entry=THAT in grubenv is wrong
which could lead to whatever inexplicable or misleading
error messages when booting the recreated system,
e.g. of same kind as what I experienced
with error: invalid environment block. in
https://github.com/rear/rear/issues/1828
starting at
https://github.com/rear/rear/issues/1828#issuecomment-398717889
I think ReaR should be properly enhanced to
remember the default boot entry of the original system and
try to reinstall that on the recreated system if possible.
jsmeix commented at 2026-03-09 12:49:¶
@svlv
when a user knows that keeping grubenv in the recreated system
is OK in his specific case, then something like
BACKUP_RESTORE_MOVE_AWAY_FILES=( )
is the currently intended method.
I think this issue is no bug because
the code does what it is meant to do and
that is properly documented in default.conf
svlv commented at 2026-03-09 12:54:¶
@jsmeix I've tried to reproduce what you experienced on a fresh SLES 15 but without success.
Think about a system which had two boot entries
"THIS" and "THAT" and "THAT" was the default boot entry
but during "rear recover" only "THIS" gets recreated
(for whatever reason).
I’ve experienced the reverse use case: after removing grubenv, the
entry that became the default was wrong. It did not have the required
initrd file.
jsmeix commented at 2026-03-09 13:10:¶
@svlv
do I understand it correctly what you experienced is:
GRUB was reinstalled with an entry which did not boot
(because its initrd file was missing)?
This looks like a bug related to how ReaR reinstalls GRUB.
The actual root cause could be how ReaR recreates initrds
or whether or not all needed initrds were restored from
the backup or whatever - perhaps even a bug in GRUB?
svlv commented at 2026-03-09 13:14:¶
@jsmeix
when a user knows that keeping grubenv in the recreated system
is OK in his specific case, then something like
Basically, I am trying to understand when it is not OK: only in your
particular environment or it is something common . Unlikely you
experienced the issue with saved_entry. The problem could be with
env_block=512+1.
You decided to remove grubenv because you experienced the issue.
Perhaps without the issue, you hadn't touched it at all. Meaning, the
problem could be misinterpret.
jsmeix commented at 2026-03-09 13:31:¶
I set BACKUP_RESTORE_MOVE_AWAY_FILES in default.conf
because of my reasoning that by default grubenv should
be considered as outdated in the recreated system.
If my reasoning is wrong please explain why.
My main point of my reasoning is that
on a by "rear recover" recreated system
there is no such thing as a meaningful previous boot
so grubenv is meaningless on the recreated system.
In more precise wording:
on a by "rear recover" recreated system
there is no such thing as a meaningful previous boot
of that exact system
"The system" was booted before as the original system,
but ReaR does not make an identical copy of the system
(as 'dd' of the system's disk partitions could do it)
so ReaR's recreated system was never booted before.
svlv commented at 2026-03-09 13:55:¶
@jsmeix
do I understand it correctly what you experienced is:
GRUB was reinstalled with an entry which did not boot
(because its initrd file was missing)?
Yes, a dangling boot entry became the default one. Of course ReaR can
inspect entries from /boot/loader/entries and remove dangling ones.
But I think it is another issue. Howerver without it I probably even
didn't notice it.
I set BACKUP_RESTORE_MOVE_AWAY_FILES in default.conf
because of my reasoning that by default grubenv should
be considered as outdated in the recreated system.
I understand your point of view. I even partially agree with it because
grubenv, e.g., may contain boot_indeterminate, boot_success,
boot_once , etc and these variables must be reset during recovery.
However, my correct understanding that there was the error
invalid environment block, and this error was fixed by removing the
grubenv file. No other known reasons, am I right?
If my reasoning is wrong please explain why.
GRUB relies on grubenv. grubenv may contain essential information,
such as saved_entry, default or kernelopts. It can be considered
as part of grub.cfg. I guess it shouldn't be simply removed.
jsmeix commented at 2026-03-09 14:17:¶
I experienced that error: invalid environment block.
and this lead me to my understanding at that time
that grubenv as a whole should be considered
as outdated in the recreated system
in particular because its documentation reads
The GRUB environment block
It is often useful to be able
to remember a small amount of information
from one boot to the next.
which I understand as THE reason why grubenv exists.
So from my point of view grubenv has no reason to exist
when there was no previous boot.
That error: invalid environment block. is not the reason
why I set BACKUP_RESTORE_MOVE_AWAY_FILES in default.conf.
That error: invalid environment block. only triggered me
to investigate and find out what grubenv is meant to do.
What I found out what grubenv is meant to do is the reason
why I set BACKUP_RESTORE_MOVE_AWAY_FILES in default.conf.
Now @svlv tells that
(excerpts)
GRUB relies on grubenv ... it can be considered
as part of grub.cfg ... it shouldn't be removed
For me this is new information and it totally changes things.
When GRUB needs grubenv to work properly then grubenv
must of course not be removed.
But perhaps a possibly outdated grubenv from the
original system is also not a proper solution?
So a proper solution could be to also recreate
grubenv which matches the recreated system?
In general:
I am not at all a sufficient bootloader expert
to make educated statements what
an actually right solution is.
svlv commented at 2026-03-09 14:39:¶
By the way, based on the Environment block documentation on Btrfs it works differently:
On Btrfs filesystems, a reserved area in the filesystem header may be used to store the environment block. This static block avoids the problems of updating a normal file on a copy-on-write filesystem, where writing raw block is not stable and requires metadata update. The reserved area provides a fixed location that GRUB can update directly, allowing commands such as grub-reboot and ‘GRUB_SAVEDEFAULT’ to function correctly on Btrfs volumes.
jsmeix commented at 2026-03-09 14:48:¶
In general on Btrfs all is always different :-(
cf. in https://en.opensuse.org/SDB:Disaster_Recovery
the section
"All filesystems are equal, but some are more equal than others"
Sigh!
pcahyna commented at 2026-03-09 18:35:¶
I generally agree with
GRUB relies on grubenv. grubenv may contain essential information, such as saved_entry, default or kernelopts (emphasis mine). It can be considered as part of grub.cfg. I guess it shouldn't be simply removed.
RHEL 8 has a peculiar behavior when one sets the kernel command line
options using grubby: grubby sets the options in grubenv. This is
done so that grubby (or any other tool that needs to alter the kernel
command line) can avoid modifying the BLS files and they can be
considered static and part of the kernel packages. See:
# grub2-editenv list
saved_entry=43718aa8f32748d1bb4f80afb9a1b0ad-4.18.0-553.107.1.el8_10.x86_64
kernelopts=root=UUID=384a38ac-d639-47a4-9224-ec792d528fb7 ro crashkernel=auto net.ifnames=0 rhgb console=tty0 console=ttyS0,115200n8
boot_success=0
# fips-mode-setup --enable
(busy doing various stuff to enable FIPS, among them changing kernel command line ...)
FIPS mode will be enabled.
Please reboot the system for the setting to take effect.
# grub2-editenv list
saved_entry=43718aa8f32748d1bb4f80afb9a1b0ad-4.18.0-553.107.1.el8_10.x86_64
kernelopts=root=UUID=384a38ac-d639-47a4-9224-ec792d528fb7 ro crashkernel=auto net.ifnames=0 rhgb console=tty0 console=ttyS0,115200n8 fips=1
boot_success=0
^-- see the appended fips=1 to kernelopts above.
In RHEL 9+ this does not happen anymore, AFAIK, as this apparently neat hack was considered too fragile in practice. Still, RHEL 8 is an important case to consider and also there are other important info (default boot entry is one of them, as we see from this issue).
@jsmeix
My main point of my reasoning is that
on a by "rear recover" recreated system there is no such thing as a meaningful previous bootso grubenv is meaningless on the recreated system. In more precise wording:
on a by "rear recover" recreated system there is no such thing as a meaningful previous boot of that exact system"The system" was booted before as the original system, but ReaR does not make an identical copy of the system (as 'dd' of the system's disk partitions could do it) so ReaR's recreated system was never booted before.
I would say that ReaR strives to make a copy of the system identical enough to make this meaningful - one should expect stuff that depends on previous boots to use the previous boot before making the backup. (Whether that's feasible is of course another question.)
Think about a system which had two boot entries "THIS" and "THAT" and "THAT" was the default boot entry but during "rear recover" only "THIS" gets recreated (for whatever reason). Then
saved_entry=THATin grubenv is wrong which could lead to whatever inexplicable or misleading error messages when booting the recreated system, e.g. of same kind as what I experienced in #1828 starting at #1828 (comment)
I would say that if "THAT" boot entry does not get recreated, it is a pretty big issue in ReaR and would need to be fixed in any case. Imagine "THAT" boot entry being the only working one - how would the recreated system boot? No amount of changes to grubenv would make the system bootable.
when a user knows that keeping grubenv in the recreated system is OK in his specific case, then something like
BACKUP_RESTORE_MOVE_AWAY_FILES=( )is the currently intended method.
I would say that we are in an opposite situation: we (maintainers) know
that removing grubenv is problematic in some cases, so I would keep it
by default and when a user knows that removing grubenv in the recreated
system is needed, they can set
BACKUP_RESTORE_MOVE_AWAY_FILES+=( /boot/grub2/grubenv ). Appending to
the variable is cleaner, setting it to an empty list should be
discouraged; who knows what other important stuff you could be missing?
Back to the situation on RHEL 8 and why it is problematic.
grub2-mkconfig generates this as part of the new config:
# This section was generated by a script. Do not modify the generated file - all changes
# will be lost the next time file is regenerated. Instead edit the BootLoaderSpec files.
#
# The blscfg command parses the BootLoaderSpec files stored in /boot/loader/entries and
# populates the boot menu. Please refer to the Boot Loader Specification documentation
# for the files format: https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/.
# The kernelopts variable should be defined in the grubenv file. But to ensure that menu
# entries populated from BootLoaderSpec files that use this variable work correctly even
# without a grubenv file, define a fallback kernelopts variable if this has not been set.
#
# The kernelopts variable in the grubenv file can be modified using the grubby tool or by
# executing the grub2-mkconfig tool. For the latter, the values of the GRUB_CMDLINE_LINUX
# and GRUB_CMDLINE_LINUX_DEFAULT options from /etc/default/grub file are used to set both
# the kernelopts variable in the grubenv file and the fallback kernelopts variable.
if [ -z "${kernelopts}" ]; then
set kernelopts="root=UUID=384a38ac-d639-47a4-9224-ec792d528fb7 ro crashkernel=auto net.ifnames=0 rhgb console=tty0 console=ttyS0,115200n8 fips=1 "
fi
insmod blscfg
blscfg
### END /etc/grub.d/10_linux ###
So it adds a fallback kernelopts value to the GRUB configuration file,
which mitigates the problem a bit - if one regenerates the configuration
file, the value can be used from there. (The value was also set in
/etc/default/grub:
GRUB_CMDLINE_LINUX="crashkernel=auto net.ifnames=0 rhgb console=tty0 console=ttyS0,115200n8 fips=1"
and /etc/kernel/cmdline:
root=UUID=384a38ac-d639-47a4-9224-ec792d528fb7 ro crashkernel=auto net.ifnames=0 rhgb console=tty0 console=ttyS0,115200n8 fips=1
).
Still, removing grubenv is problematic (in addition to the reasons
identified above). On UEFI RHEL 8, /boot/grub2/grubenv is a symlink to
../efi/EFI/redhat/grubenv. After removing it, it gets recreated, but
as a regular file, not a symlink. This causes discrepancy between the
two files - newly modified boot parameters are not applied as a result.
By the way, @svlv :
may contain boot_indeterminate, boot_success, boot_once , etc and these variables must be reset during recovery.
Do they really need to be reset? I suppose that if a system undergoes a hard reboot (crash, power failure), the values are the same as after recovery. Yet, the system must be able to boot after a hard reboot.
Anyway, @svlv , thank you a lot for having looked into the problem!
pcahyna commented at 2026-03-09 18:42:¶
@svlv
I've tried to reproduce what you experienced on a fresh SLES 15 but without success.
By the way: my understanding is that the issue actually was:
during boot there is still that grub error message
which I managed now to catch before it is too fast flashing away
by pausing the virtual machine in the right moment
Have you used a serial console when trying to reproduce that in order to capture all output? Perhaps the message flashes too fast?
@jsmeix are you able to reproduce the issue again with
BACKUP_RESTORE_MOVE_AWAY_FILES=( ) ?
svlv commented at 2026-03-10 06:52:¶
Thank you @jsmeix and @pcahyna
Finally I've reproduced the issue with env_block on Btrfs in GRUB
shell. (Yes, I didn't capture serial console - it was my mistake)
grub> load_env
grub> echo ${env_block}
512+1
grub> set env_block="(${root})${env_block}"
grub> load_env -f "${env_block}"
error: invalid environment block.
I've looked through the code. Likely it is because env_block doesn't
contain GRUB_ENVBLK_SIGNATURE.
I guess we can't simply leave grubenv as is without re-creating env
block and re-setting variables.
env_block itself is a read-only variable, meaning it cannot be changed
via grub-editenv.
Also, I've found the
patch
that was introducing env_block.
I'll keep researching in my spare time.
svlv commented at 2026-03-10 07:03:¶
I guess we can't simply leave grubenv as is without re-creating env block and re-setting variables.
I am not sure about this statement. It is rather first thoughts after
looking at env_block.
Also, if I am not mistaken, env_block is used only on Btrfs.
jsmeix commented at 2026-03-10 09:00:¶
As far as I see it looks now as if
each bootloader stuff is a whole mess
which changes arbitrarily over time
in each one of the Linux distributions.
SUSE does its specific hacks,
Red Hat does its specific hacks,
and so on...
I think ReaR should try to not get involved into
continuously changing specific details of various
Linux distribution specific bootloader hacks
because we at ReaR upstream will never ever be able
to properly support continuously changing details.
Perhaps there is a more generic way out of that mess?
I guess that each Linux distribution has a more or less
generic and stable method how each Linux distribution
does its bootloader setup and bootloader installation
(in particular during their initial system installation),
cf. the initial comments in
finalize/Linux-i386/660_install_grub2.sh
Perhaps it could be a generic way out of that mess
when ReaR only runs each Linux distribution's method
during "rear recover" from within the recreated system
(as usual via chroot /mnt/local ...)?
So instead of one
finalize/Linux-i386/660_install_grub2.sh
we may use separated distribution specific scripts like
usr/share/rear/finalize/SUSE_LINUX/660_install_grub2.sh
and
usr/share/rear/finalize/Fedora/660_install_grub2.sh
where each one runs the distribution specific method?
jsmeix commented at 2026-03-10 13:20:¶
@pcahyna regarding your
https://github.com/rear/rear/issues/3578#issuecomment-4025912872
(excerpts)
I would say that if "THAT" boot entry does not get recreated,
it is a pretty big issue in ReaR
not necessarily - in particular in MIGRATION_MODE the user
can modify the restored GRUB config files as he needs,
e.g. he may no longer want the "THAT" boot entry, cf.
finalize/default/520_confirm_finalize.sh
but likely users may not (yet) know that nowadays grubenv
is also a GRUB config file which may have to be kept
or may have to be removed depending on Linux distribution
specific details of their hacked bootloader setup.
I only like to tell that things could become complicated.
I don't know how to properly deal with it automatically in ReaR.
removing grubenv is problematic in some cases, so I would
keep it by default and when a user knows that removing grubenv
in the recreated system is needed, they can set
BACKUP_RESTORE_MOVE_AWAY_FILES+=( /boot/grub2/grubenv )
I agree - in particular because according to
https://github.com/rear/rear/issues/1828#issuecomment-398717889
when grubenv is kept a SUSE system boots
regardless of the GRUB error message
error: invalid environment block.
but according to this issue here a Rocky 9 system
fails to boot when grubenv is removed so this issue here
is more severe than only a GRUB error message annoyance.
jsmeix commented at 2026-03-10 14:00:¶
I did
https://github.com/rear/rear/pull/3581
as a quick proposal how to no longer move grubenv away.
As in
https://github.com/rear/rear/pull/3579#discussion_r2895897280
my main thing here is also to have comments which explain
why things are as they are so things could be properly
cleaned up later as needed - in particular if @svlv
and/or @pcahyna like to implement it differently.
pcahyna commented at 2026-03-10 15:52:¶
As far as I see it looks now as if each bootloader stuff is a whole mess which changes arbitrarily over time in each one of the Linux distributions.
SUSE does its specific hacks, Red Hat does its specific hacks, and so on...
I think ReaR should try to not get involved into continuously changing specific details of various Linux distribution specific bootloader hacks because we at ReaR upstream will never ever be able to properly support continuously changing details.
I don't see this issue as supporting this view. It is actually the contrary: it has led me to see some convergence in this area. I was stuck with that RHEL 8 issue because RHEL 8 does its own hack with grubenv and I did not know how to unify it with the code for other distributions. Now we see that there are more reasons to preserve grubenv than just the RHEL 8-specific behavior, which provides an opportunity to solve multiple issues in a generic way.
Regarding the SUSE-specific grubenv behavior: as it is often the case,
it helped to return to the issue after some rest and searching some
docs. Here's what I found :
https://en.opensuse.org/GRUB#GRUB2_on_btrfs_/boot . The key
observation is that env_block=512+1 introduces an indirection to a
reserved area outside of the filesystem data, which we don't preserve
and therefore merely keeping the file is broken. (It is also in the
documentation quoted earlier: "On Btrfs filesystems, a reserved area in
the filesystem header may be used to store the environment block.")
Keeping the file and removing the file lead to the same end result in
this case, which is that the environment block is lost, in the former
case GRUB just complains about it. I believe that the proper fix would
be to actually preserve the environment block in this case (probably as
a part of disk layout) and restore it during recovery, but that's a
bigger project of course.
The behavior is not really SUSE-specific though, it is btrfs-specific. The patch that introduced it was recently applied also to Grub 2 in Fedora and will be in Fedora 44: https://bugzilla.redhat.com/show_bug.cgi?id=2372973 . Fedora uses btrfs by default (although not on /boot, at least not yet). Thus we see more convergence and this again points to the opportunity and in fact the need of handling the situation in a common way instead of distribution-specific hacks.
svlv commented at 2026-03-10 16:12:¶
Things with env_block are quite simple.
There is grubenv file which GRUB loads by load_env. Next, if
grubenv contains env_block, this block loads as well.
set btrfs_relative_path="y"
export btrfs_relative_path
if [ -f ${config_directory}/grubenv ]; then
load_env -f ${config_directory}/grubenv
elif [ -s $prefix/grubenv ]; then
load_env
fi
if [ "${env_block}" ] ; then
set env_block="(${root})${env_block}"
export env_block
load_env -f "${env_block}"
fi
Next, during the boot process the env block is used to store some variables, e.g.,
function savedefault {
if [ -z "${boot_once}" ]; then
saved_entry="${chosen}"
if [ "${env_block}" ] ; then
save_env -f "${env_block}" saved_entry
else
save_env saved_entry
fi
fi
}
During recovery the env block must be properly initialised if it is
used. But now I don't how to do it.
Probably dd can be used or grub-editenv create should be able to do
it.
I started analysing grub-editenv create but didn't find out how to
init block_env.
How can I do it on running system? How env_block appears in grubenv
if it can't be changed by grub-editenv set ? - It is what I am trying
to figure out.
It is the function which creates block env - https://github.com/rhboot/grub2/blob/fd2a33b6f90f0ac052d4209752848c4c06236538/util/editenv.c#L39
pcahyna commented at 2026-03-10 16:22:¶
@svlv here's how to recreate the environment block: https://en.opensuse.org/GRUB#Recovering_from_broken_grubenv
svlv commented at 2026-03-10 16:24:¶
Yes, thanks, I checked the link you sent.
I’ll try it out tomorrow.
‘grub2-editenv - unset dummy’ is nice :)
svlv commented at 2026-03-11 07:34:¶
No issue to initialise block_env during recovery, in my case GRUB
found the root device correctly. Before I used
grub2-editenv /boot/grub2/grubenv set but it didn't work because it is
essential to use grub2-editenv - set.
I've checked the patch one more time and found the reason - `fs_envblk` is probed only when filename is `-`.
if (strcmp (filename, "-") == 0)
filename = DEFAULT_ENVBLK_PATH;
...
if (strcmp (filename, DEFAULT_ENVBLK_PATH) == 0)
probe_fs_envblk (fs_envblk_spec);
I used the following script to reset all variables. There is no issue in my case:
#!/bin/bash
grub2-editenv - list > myenv
rm /boot/grub2/grubenv
grub2-editenv - unset dummy
while IFS= read -r key_value; do
key="${key_value%=*}"
# env_block is read only
if [ "$key" = "env_block" ] ; then
continue
fi
grub2-editenv - set "$key_value"
done < myenv
# insure that my_var has been set
grub2-editenv - set my_var=my_value
rm myenv
After recovery:
sles15:~ # grub2-editenv - list
env_block=512+1
saved_entry=SLES 15
my_var=my_value
Probably, it needs to create a separate script in ReaR to do something similar.
Updated:
grub2-editenv - unset dummy is essential. I checked in GRUB shell and
got the error. Sorry. I am always in hurry and make mistakes.
svlv commented at 2026-03-11 09:03:¶
Regarding grub2-editenv - unset dummy.
In Fedora's patch setcommand will also trigger
create_env_on_block (); but in SUSE's patch it won't trigger.
I was looking though the Fedora code and came to the conclusion that
set command should trigger it as well.
At least I do see it in https://build.opensuse.org/projects/openSUSE:Leap:15.6:Update/packages/grub2.37794/files/grub2-grubenv-in-btrfs-header.patch?expand=0 SUSE.
Very likely patches are different. Fedora uses the patch reviewed by the upstream.
jsmeix commented at 2026-03-11 09:29:¶
@pcahyna
regarding your
https://github.com/rear/rear/issues/3578#issuecomment-4032535010
(excerpts)
env_block=512+1 introduces an indirection to a reserved area
outside of the filesystem data, which we don't preserve
...
"On Btrfs filesystems, a reserved area in the filesystem header
may be used to store the environment block."
...
I believe that the proper fix would be
to actually preserve the environment block in this case
(probably as a part of disk layout)
and restore it during recovery
This is exactly what must not be done because
we at ReaR upstream will never ever be able
to properly support continuously changing details
where "continuously changing details"
also means things like "implementaion details".
I liked to explain that with my
https://github.com/rear/rear/issues/3578#issuecomment-4029740208
but - as so often - unfortunately - I failed to get the
current specific implementaion details exactly right
so my basic reasoning became misunderstood.
Another try to make my basic reasoning clear:
ReaR recreates a system anew from scratch.
In particular ReaR recreates the so called "disk layout"
(basically partitioning and filesystems) anew from scratch.
ReaR does not preserve filesystem low-level binary (meta) data
from the original system and somehow tries to "insert"
or "merge" that into the recreated new filesystems.
All gets recreated by calling those tools which are
meant to create partitions, filesystems, and so on.
E.g. in case of SUSE's special btrfs structure,
ReaR calls SUSE's special helper tool to let it create
SUSE's special btrfs structure anew from scratch.
Accordingly also the GRUB environment block must be
recreated anew from scratch by calling those tools
which are meant to create a GRUB environment block.
Such tools must exist because otherwise it would not
have been possible to do the initial system installation
of the original system with a proper GRUB environment block.
What ReaR could do to recreate a GRUB environment block
which matches (a logical match - not a binary data match)
what the GRUB environment block was on the original system
is that ReaR stores values of interest (e.g. 'saved_entry')
from the GRUB environment block on the original system and
recreates the GRUB environment block with that values set
logically same as they have been on the original system
where "logically same" means that a value may have to be
adjusted to match what there is on the recreated system
(e.g. some ID may have changed or whatever).
jsmeix commented at 2026-03-11 09:45:¶
@svlv
not that it actually helps here but in my opinion
"every Linux distribution specific patch is a bug".
What system admins want is that things work as designed
and documented by the matching upstream projects
without the need for special attention.
This means things should work without unexpected behaviour
in particular without Linux distribution specific stuff
and without behavioural changes all of a sudden.
Cf.
https://en.opensuse.org/index.php?title=Concepts_printing&oldid=183171#System_admin_expectations
Clarification to avoid misunderstanding:
The exact wording "Linux distribution specific patch" is crucial.
I am only talking about when Linux distributions deviate from
upstream by their own specific modifications of a software.
Examples of patches which are perfectly OK:
- Backported upstream fixes to older software versions
which are provided by certain Linux distributions.
Such patches are specific for an older software version
but they are neither a deviation from upstream
nor specific to a particular Linux distribution. - Fixes and enhancements for current software versions
by particular Linux distributions which are meant to get
integrated into a future upstream release of that software.
Such patches are not meant as deviation from upstream and
likely also other Linux distributions could benefit from it.
pcahyna commented at 2026-03-11 11:56:¶
@pcahyna regarding your #3578 (comment) (excerpts)
env_block=512+1 introduces an indirection to a reserved area outside of the filesystem data, which we don't preserve ... "On Btrfs filesystems, a reserved area in the filesystem header may be used to store the environment block." ... I believe that the proper fix would be to actually preserve the environment block in this case (probably as a part of disk layout) and restore it during recoveryThis is exactly what must not be done because
we at ReaR upstream will never ever be able to properly support continuously changing detailswhere "continuously changing details" also means things like "implementaion details".
I liked to explain that with my #3578 (comment) but - as so often - unfortunately - I failed to get the current specific implementaion details exactly right so my basic reasoning became misunderstood.
Another try to make my basic reasoning clear:
ReaR recreates a system anew from scratch. In particular ReaR recreates the so called "disk layout" (basically partitioning and filesystems) anew from scratch. ReaR does not preserve filesystem low-level binary (meta) data from the original system and somehow tries to "insert" or "merge" that into the recreated new filesystems. All gets recreated by calling those tools which are meant to create partitions, filesystems, and so on. E.g. in case of SUSE's special btrfs structure, ReaR calls SUSE's special helper tool to let it create SUSE's special btrfs structure anew from scratch.
Accordingly also the GRUB environment block must be recreated anew from scratch by calling those tools which are meant to create a GRUB environment block. Such tools must exist because otherwise it would not have been possible to do the initial system installation of the original system with a proper GRUB environment block.
What ReaR could do to recreate a GRUB environment block which matches (a logical match - not a binary data match) what the GRUB environment block was on the original system is that ReaR stores values of interest (e.g. 'saved_entry') from the GRUB environment block on the original system and recreates the GRUB environment block with that values set logically same as they have been on the original system where "logically same" means that a value may have to be adjusted to match what there is on the recreated system (e.g. some ID may have changed or whatever).
@jsmeix , sorry, I now see that I was not clear either and my idea was also misunderstood. What I meant by
I believe that the proper fix would be
to actually preserve the environment block in this case
(probably as a part of disk layout)
and restore it during recovery
was not to save and restore the actual disk sectors where GRUB stores
the environment block on Btrfs, but to save and restore the content of
the environment block, using grub2-editenv or another appropriate
tool.
pcahyna commented at 2026-03-11 12:18:¶
Regarding
grub2-editenv - unset dummy. In Fedora's patchsetcommand will also triggercreate_env_on_block ();but in SUSE's patch it won't trigger. I was looking though the Fedora code and came to the conclusion thatsetcommand should trigger it as well.At least I do see it in https://build.opensuse.org/projects/openSUSE:Leap:15.6:Update/packages/grub2.37794/files/grub2-grubenv-in-btrfs-header.patch?expand=0 SUSE.
Very likely patches are different. Fedora uses the patch reviewed by the upstream.
I suppose it does no harm to do grub2-editenv - unset dummy in both
cases to be on the safe side?
jsmeix commented at 2026-03-11 12:35:¶
@pcahyna
thank you for your prompt clarification what you actually meant.
Phew!
This helps me a lot to sleep relaxed (pun intended) tonight :-)
svlv commented at 2026-03-11 13:02:¶
I suppose it does no harm to do grub2-editenv - unset dummy in both cases to be on the safe side?
Yes, it should be fine on both systems.
Perhaps, it needs to be tested on the latest Fedora version with Btrfs
on /boot to make sure that everything is OK.
jsmeix commented at 2026-03-11 13:53:¶
Regarding
https://github.com/rear/rear/issues/3578#issuecomment-4037600275
why the SUSE patch is different compared to the Fedora patch:
GRUB2 in openSUSE Leap 15.x comes from SLES 15.
GRUB2 is provided in SLES 15 as those versions:
# for p in SUSE:SLE-15-SP7:Update SUSE:SLE-15-SP6:Update \
SUSE:SLE-15-SP5:Update SUSE:SLE-15-SP4:Update \
SUSE:SLE-15-SP3:Update SUSE:SLE-15-SP2:Update \
SUSE:SLE-15-SP1:Update SUSE:SLE-15:Update ; \
do echo $p ; osc list $p grub2 | grep 'grub.*tar\.xz' ; echo ; done
SUSE:SLE-15-SP7:Update
grub-2.12.tar.xz
SUSE:SLE-15-SP6:Update
grub-2.12.tar.xz
SUSE:SLE-15-SP5:Update
grub-2.06.tar.xz
SUSE:SLE-15-SP4:Update
grub-2.06.tar.xz
SUSE:SLE-15-SP3:Update
grub-2.04.tar.xz
SUSE:SLE-15-SP2:Update
grub-2.04.tar.xz
SUSE:SLE-15-SP1:Update
grub-2.02.tar.xz
SUSE:SLE-15:Update
grub-2.02.tar.xz
All of them have a grub2-grubenv-in-btrfs-header.patch
which gets adapted from GRUB2 version to version.
In contrast in openSUSE:Factory there is GRUB2 version 2.14
# osc list openSUSE:Factory grub2 | grep 'grub.*tar\.xz'
grub-2.14.tar.xz
There is no longer a grub2-grubenv-in-btrfs-header.patch
because
# osc cat openSUSE:Factory grub2 grub2.changes | less
...
Fri Jan 16 08:52:19 UTC 2026 - Michael Chang <mchang@suse.com>
- Update to 2.14
- New in 2.14
...
* GRUB environment block inside the Btrfs header support.
...
- Patch removed
...
* grub2-grubenv-in-btrfs-header.patch
So GRUB2 versions before 2.14 need some kind of
'grub2-grubenv-in-btrfs-header.patch' to support
a "GRUB environment block inside the Btrfs header"
or in less implementation detail specific wording
to support a GRUB environment block also on Btrfs
(it is an implementation detail how a GRUB environment
block is actually stored in case of Btrfs), see also
https://lists.gnu.org/archive/html/grub-devel/2025-07/msg00129.html
I assume SUSE uses its specific long tested that it works
for what SUSE needs 'grub2-grubenv-in-btrfs-header.patch'
in SUSE's GRUB2 versions before 2.14.
svlv commented at 2026-03-11 14:12:¶
@jsmeix
SLES 16 uses GRUB2 2.12. So it means the change must be tested on SLES
15-16 to validate that everything is OK on GRUB2 with
grub2-grubenv-in-btrfs-header.patch, and must be tested on, e.g.,
openSUSE Tumbleweed with GRUB 2.14 after block_env was upstreamed and
grub2-grubenv-in-btrfs-header.patch was removed. Am I right?
pcahyna commented at 2026-03-11 14:27:¶
I suppose it does no harm to do grub2-editenv - unset dummy in both cases to be on the safe side?
Yes, it should be fine on both systems. Perhaps, it needs to be tested on the latest Fedora version with Btrfs on
/bootto make sure that everything is OK.
yes, I have been working on a Fedora test.
jsmeix commented at 2026-03-11 14:29:¶
@pcahyna @svlv
many thanks for digging out details and reasoning behind
why that grub2-grubenv-in-btrfs-header.patch is there.
I think now even I got some understanding of that stuff.
Somewhat offtopic:
What I still do not understand is
why on BIOS in case of Btrfs
not a 'bios_grub' partition could be used
with a filesystem that GRUB can read and write
so a GRUB environment block could be stored there
AND
if also on UEFI in case of Btrfs a GRUB environment block
is stored within the realm of the Btrfs filesystem
and not in the ESP?
My guess was that in both cases the reason could be that
what there is on a 'bios_grub' partition or on an ESP
cannot be snapshotted by Btrfs so certain GRUB stuff
could get out of sync with what there is on a
Btrfs subvolume that gets mounted at /?
But I think my guess does not make sense because
I assume that also the Btrfs header is not snapshotted
because only what is within a Btrfs subvolume is snapshotted.
So currently all that does not make sense to me.
Sorry in advance if such questions are stupid.
I became rather ignorant of bootloader details.
My personal reason is that I cannot understand
why since beginning of time bootloader stuff
is still so complicated nowadays because in the end
all the fuss is only about an "initial program load"
(cf. 'IPL' on IBM Z).
I remember LILO well: At that time I had helped
many S.u.S.E.[sic] users with booting issues
because I understood LILO's peculiarities which came
from BIOS limitations. At least from my point of view
LILO was rather simple and straightforward compared
to GRUB where I basically got lost step by step over time.
jsmeix commented at 2026-03-11 14:45:¶
@svlv
regarding your
https://github.com/rear/rear/issues/3578#issuecomment-4039500371
For ReaR 3.0 only SLES 16 does really matter.
It would be fine when ReaR 3.0 still works on SLES 15
but if not it is no problem because ReaR 2.9 works on SLES 15,
cf. the section "Version upgrades with Relax-and-Recover" in
https://en.opensuse.org/SDB:Disaster_Recovery
(excerpt)
When you have a working disaster recovery procedure,
do not upgrade ReaR ...
In general openSUSE Tumbleweed is only nice to have
because ReaR cannot support openSUSE Tumbleweed, see
https://github.com/rear/rear/blob/rear-2.9/doc/rear-release-notes.txt#L151
(excerpt)
In theory ReaR 2.9 should work on openSUSE Tumbleweed
but in practice arbitrary failures could happen at any time
because the Tumbleweed distribution is a pure rolling release
version of openSUSE containing the latest stable versions
of all software (cf. https://en.opensuse.org/Portal:Tumbleweed)
so arbitrary changes of any software are possible at any time
that could arbitrarily break how ReaR works.
pcahyna commented at 2026-03-11 15:56:¶
What I still do not understand is why on BIOS in case of Btrfs not a 'bios_grub' partition could be used with a filesystem that GRUB can read and write so a GRUB environment block could be stored there AND if also on UEFI in case of Btrfs a GRUB environment block is stored within the realm of the Btrfs filesystem and not in the ESP?
My guess was that in both cases the reason could be that what there is on a 'bios_grub' partition or on an ESP cannot be snapshotted by Btrfs so certain GRUB stuff could get out of sync with what there is on a Btrfs subvolume that gets mounted at
/? But I think my guess does not make sense because I assume that also the Btrfs header is not snapshotted because only what is within a Btrfs subvolume is snapshotted. So currently all that does not make sense to me.
@jsmeix
Fedora bug https://bugzilla.redhat.com/show_bug.cgi?id=2372973 description starts with: "Currently, if you set up /boot as a btrfs subvolume (...)". Indeed, this is not the default, by default /boot is on XFS, so this configuration is not (yet) standard. It does not mention why one wants to do that, neither why SLES/openSUSE choose so (you can ask your colleagues), but I can imagine that it is due to the (understandable) need of snapshotting the content of /boot. I suppose as well that the header is not snapshotted, so those who need this can live with the environment block not being snapshotted (after all you found that when it gets corrupted, the system still boots), but they still want to have the rest of /boot snapshotted (which does not include just the bootloader, but more importantly the kernel(s) and initrd(s)).
pcahyna commented at 2026-03-11 16:03:¶
@svlv I was able to get Fedora working with /boot on Btrfs.
# lsblk -o +fstype
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE
zram0 251:0 0 7.4G 0 disk [SWAP] swap
vda 253:0 0 54G 0 disk
├─vda1 253:1 0 5.4G 0 part [SWAP] swap
└─vda2 253:2 0 48.6G 0 part / btrfs
# grub2-editenv list
env_block=512+1
saved_entry=c46f5f76227c4cd1a73cdf05de426446-7.0.0-0.rc2.21.fc45.x86_64
blsdir=/root/boot/loader/entries
boot_success=0
# cat /boot/grub2/grubenv
# GRUB Environment Block
# WARNING: Do not edit this file by tools other than grub-editenv!!!
env_block=512+1
saved_entry=c46f5f76227c4cd1a73cdf05de426446-7.0.0-0.rc2.21.fc45.x86_64
blsdir=/root/boot/loader/entries
###################(...)
Interestingly, the variables are set in the file, but I suppose this could change if GRUB actually touches some of the variables.
Do you want me to test something?
svlv commented at 2026-03-11 16:32:¶
Vars are set both to the file and to the fs block.
If you use ‘-‘ as filename.
If you want to set var the fs block only, you can run ‘save_env -f
${env_block} my_var’ in grub shell
I’d shared the example before.
svlv commented at 2026-03-11 16:39:¶
About testing, I think the most important thing is to call load_env -f ${env_block} in grub shell.
pcahyna commented at 2026-03-11 17:09:¶
Indeed, I did foo=bar and save_env foo in GRUB shell complains about
not being able to save to a sparse file, save_env -f ${env_block} foo
works and after boot grub2-editenv list shows foo=bar, which is not
present in the grubenv file.
jsmeix commented at 2026-03-31 14:52:¶
With https://github.com/rear/rear/pull/3581 merged
this issue should be avioded but this is not yet a solution.
For a solution see
https://github.com/rear/rear/pull/3591
[Export of Github issue for rear/rear.]