#3410 Issue `open`: CrowdStrike Falcon may conflict with ReaR¶

Labels: support / question, external tool

jsmeix opened issue at 2025-02-27 11:50:¶

CrowdStrike Falcon could make "rear recover" fail

I noticed a SUSE internal issue where a customer reports that
running CrowdStrike Falcon (used as AV solution) results
that "rear mkrescue" created a ReaR recovery system ISO
but with that ISO "rear recover" failed to recreate the system.

After disabling the AV the ISO is working as intended.

So running CrowdStrike Falcon somehow conflicts with ReaR.

I don't know any more details.
I neither know how "rear recover" failed
nor how CrowdStrike Falcon conflicts with ReaR.

I report it only FYI
that running CrowdStrike Falcon could be a reason
for whatever failures during "rear recover".

jsmeix commented at 2025-02-28 08:31:¶

I got a "rear mkbackup" log file from this customer case
with a short text what had happened:

Here is a rear log claiming to successfully create the recovery image,
but the iso is not usable. The logs complains about not being able
to umount the tmp rear working path due to be still in use.
Once falcon agent is disabled, this error shows no more
and the resulting iso is valid.

Excerpts from the "rear mkbackup" log
(with possibly customer specific values replaced by 'XXX')

Relax-and-Recover 2.7 / 2022-07-13
Running rear mkbackup (PID 139816 date 2025-01-26 04:50:01)
Using log file: /var/log/rear/rear-XXX.log
Running workflow mkbackup on the normal/original system
Using backup archive '/var/tmp/rear.cIuQs8c4J3wbOM8/outputfs/XXX/backup.tar.gz'
Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1)
...
Using sysconfig bootloader 'grub2-efi' for 'rear recover'
...
Trying to find what to use as UEFI bootloader...
Trying to find a 'well known file' to be used as UEFI bootloader...
Using '/boot/efi/EFI/sles/grubx64.efi' as UEFI bootloader file
...
Created initrd.cgz with gzip default compression (521711241 bytes) in 30 seconds
GRUB2 modules to load: fat lvm part_gpt part_msdos xfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-XXX.iso (620M)
Copying resulting files to nfs location
...
Making backup (using backup method NETFS)
...
Archived 20354 MiB in 2607 seconds [avg 7995 KiB/sec]
Exiting rear mkbackup (PID 139816) and its descendant processes ...
Running exit tasks
Failed to 'rm -Rf --one-file-system /var/tmp/rear.cIuQs8c4J3wbOM8/tmp'
Could not remove build area /var/tmp/rear.cIuQs8c4J3wbOM8 (something still exists therein)
Something is still mounted within the build area
  /var/tmp/rear.cIuQs8c4J3wbOM8/tmp/isofs/boot/efiboot.img (deleted) on /var/tmp/rear.cIuQs8c4J3wbOM8/tmp/efi_virt type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
You must manually umount it, then you could manually remove the build area
To manually remove the build area use (with caution): rm -Rf --one-file-system /var/tmp/rear.cIuQs8c4J3wbOM8

Therein the

Something is still mounted within the build area
  /var/tmp/rear.cIuQs8c4J3wbOM8/tmp/isofs/boot/efiboot.img (deleted) on /var/tmp/rear.cIuQs8c4J3wbOM8/tmp/efi_virt type vfat ...

is known as
https://github.com/rear/rear/issues/2908
for OUTPUT=ISO
and also for OUTPUT=USB the same kind of issue is known as
https://github.com/rear/rear/issues/3397
where both should be fixed by
https://github.com/rear/rear/pull/3408

For some background summary what I think about it see
https://github.com/rear/rear/issues/3397#issuecomment-2685552225

For a more detailed discussion see starting at
https://github.com/rear/rear/issues/2908#issuecomment-1378811748

As far as I can see currently I think that failed umount
is not related to a CrowdStrike Falcon conflict with ReaR
because I fail to see how that failed umount could
make the ISO unusable.

What I could imagine how a running CrowdStrike Falcon
could make the ISO unusable is that perhaps
CrowdStrike is accessing the files of the ISO
e.g. for virus scanning or whatever things and perhaps
CrowdStrike may even somehow modify files of the ISO
e.g. to protect against (possibly falsely) detected
viruses in the files of the ISO or things like that.

jsmeix commented at 2025-02-28 13:19:¶

From my current point of view the issue seems to be
a support question how to deal with CrowdStrike Falcon
when ReaR should be used because I think we should not
even try to work around or against CrowdStrike Falcon in ReaR.
In particular not because - as far as I know - CrowdStrike Falcon
works basically on kernel level (via some special additional
CrowdStrike Falcon kernel modules - as far as I know)
so CrowdStrike Falcon's realm shoud be treated sacrosanct
for user programs (even if running as 'root' as ReaR does).

jsmeix commented at 2025-02-28 14:24:¶

Set "Dedicated Priority Support" by me for SUSE as long as
the SUSE internal issue for a SUSE customer is moving forward.

schlomo commented at 2025-03-02 10:33:¶

Maybe a timing problem? ReaR tries to umount and remove the vfat image before the Crowdstrike Falcon agent finishes examining it, so that the agent accessing it prevents the umount from going through?

Didn't we once have a fuser or lsof call there in the error message to show what prevents umounting?

jsmeix commented at 2025-03-13 14:36:¶

We have 'fuser' output in
output/ISO/Linux-i386/700_create_efibootimg.sh
since Mar 22, 2023 via
https://github.com/rear/rear/commit/b40513187752c0dfc2fb467f097812dbd03a17d2
from https://github.com/rear/rear/pull/2909
for ReaR 2.8
that moved into the umount_mountpoint_retry_lazy() function
via https://github.com/rear/rear/pull/3408

But we do not have 'fuser' output in ReaR 2.7
https://github.com/rear/rear/blob/rear-2.7/usr/share/rear/output/ISO/Linux-i386/700_create_efibootimg.sh#L47
which is what the SUSE customer uses because
ReaR 2.7 is currently the latest ReaR version for SLE15
that is officially provided by SUSE for SLE-HA customers.

By the way a side note FYI:

Currently I am working on providing ReaR 2.9
as new RPM package 'rear29a' for SUSE customers, cf.
https://en.opensuse.org/SDB:Disaster_Recovery#SUSE_support_for_Relax-and-Recover

In this case the trailing 'a' in the RPM package name
means again that it will not be the pristine ReaR 2.9 release
but some (reasonable) later state, cf.
https://en.opensuse.org/SDB:Disaster_Recovery#rear_/_rear116_/_rear1172a_/_rear118a_/_rear23a_/_rear27a

In particular I want to provide my new

TRUSTED_OWNERS and TRUSTED_PATHS protection
against code injection via 'source'

to SUSE customers so ReaR behaves more secure for them,
cf. https://github.com/rear/rear/pull/3424

Currently I don't know for what specific SLE products
it will be officially provided (and supported) by SUSE,
these are ongoing management decisions...

[Export of Github issue for rear/rear.]

#3410 Issue open: CrowdStrike Falcon may conflict with ReaR¶