#1936 Issue closed: cp: error writing '/tmp/rear-efi.XXXXX//EFI/BOOT/initrd.cgz': No space left on device

Labels: enhancement, support / question, fixed / solved / done

MagnetoOptical opened issue at 2018-10-19 13:28:

Relax-and-Recover (ReaR) Issue Template

Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):

  • ReaR version ("/usr/sbin/rear -V"):

Relax-and-Recover 2.4 / Git

  • OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"):

Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000

  • Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
    PC, bare metal

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
    x86_64

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
    UEFI, grub (GRUB) 2.02-2ubuntu8.6

  • Storage (lokal disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):

sda btrfs Home 42dc50e0-0a80-4f94-b497-ecda85c3f9f9 /home
sdb btrfs Home 42dc50e0-0a80-4f94-b497-ecda85c3f9f9
sdd
├─sdd1 vfat REAR-EFI 371D-53D1
└─sdd2 ext3 REAR-000 7f78a389-5f0b-45b8-992e-4b5c81b96d07
sr0
nvme0n1
├─nvme0n1p1 vfat DF06-E334 /boot/efi
└─nvme0n1p2 ext4 a87b33b8-cc40-47a0-8d83-2b57351ec3c4 /

  • Description of the issue (ideally so that others can reproduce it):
    sudo usr/sbin/rear -vD mkrescue fails. Complains of being out of space on device, however df -h / shows:

Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p2 439G 101G 316G 25% /

  • Work-around, if any:
    I'm not aware of any.

jsmeix commented at 2018-10-19 13:35:

See what is described about TMPDIR in default.conf:
https://raw.githubusercontent.com/rear/rear/master/usr/share/rear/conf/default.conf

MagnetoOptical commented at 2018-10-19 13:44:

I'm not sure I follow.
Are you suggesting that the default TMPDIR doensn't have sufficient space? Every drive on the system has more space available than there is total data on the system. Please elaborate.

gozora commented at 2018-10-19 14:05:

Some Ubuntu version use to setup /tmp as tmpfs, so it is not part of / ...
You might check df -h /tmp output to see where is yours /tmp mounted ...

Should you really have /tmp setup on tmpfs you have two options.

  1. Disable such setup by masking tmp.mount service in systemd
  2. Create some /rear_tmp directory and set export TMPDIR="/rear_tmp" into your /etc/rear/local.conf

V.

MagnetoOptical commented at 2018-10-23 17:04:

/tmp is on /. I have 315G remaining on that drive. I have less than that in data across all drives.

jsmeix commented at 2018-10-24 09:10:

@ProfElectric
your short information is not helpful for us
because things work for us on our systems
and we do not sit in front of your system so that
we cannot imagine what goes wrong on your system
in your particular case.

As a blind guess perhaps your issue might be somehow related
to what I described at
https://github.com/rear/rear/issues/1875#issuecomment-406987662

Simply put:
It does not matter how much disk space you have in the filesystem
where the /tmp/ directory belongs to.
What matters is how much disk space you have in the filesystem
where /tmp/rear-efi.XXXXX//EFI/BOOT/initrd.cgz belongs to.

Regarding debugging what the root cause is in your case
you need to run it in debugscript mode (see man rear)
and inspect the log file, see also the section about
"Debugging issues with Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery

MagnetoOptical commented at 2018-10-25 13:54:

@ProfElectric
your short information is not helpful for us
because things work for us on our systems
and we do not sit in front of your system so that
we cannot imagine what goes wrong on your system
in your particular case.

As a blind guess perhaps your issue might be somehow related
to what I described at
#1875 (comment)

Simply put:
It does not matter how much disk space you have in the filesystem
where the /tmp/ directory belongs to.
What matters is how much disk space you have in the filesystem
where /tmp/rear-efi.XXXXX//EFI/BOOT/initrd.cgz belongs to.

Regarding debugging what the root cause is in your case
you need to run it in debugscript mode (see man rear)
and inspect the log file, see also the section about
"Debugging issues with Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery

@jsmeix

For what it's worth, the last attempt generated "/tmp/rear-efi.nUXYO//EFI/BOOT/initrd.cgz ". /tmp/rear-efi.XXXXX is in fact on the drive with / which should have ample space. The behavior is puzzling. I hope you are able to glean something helpful from the log when it's provided.

Thank you for your help and your work on this software.

MagnetoOptical commented at 2018-10-25 17:22:

rear-HOSTNAME.log

jsmeix commented at 2018-10-26 08:34:

It is the same kind of issue (not the same but the same kind) as I described in
https://github.com/rear/rear/issues/1875#issuecomment-406987662

What matters is how much disk space there is in the filesystem
where /tmp/rear-efi.nUXYO//EFI/BOOT/initrd.cgz belongs to
and that filesystem is the one on /dev/disk/by-label/REAR-EFI
because that gets mounted at /tmp/rear-efi.nUXYO
(see the debug log excerpt 'Details' below).

With OUTPUT=USB the filesystem on /dev/disk/by-label/REAR-EFI
was created by the rear format workflow, cf. the code in
usr/share/rear/format/USB/default/300_format_usb_disk.sh

What the rear format workflow results can be specified by
various USB_... config variables that are described in
usr/share/rear/conf/default.conf

In this particular case USB_UEFI_PART_SIZE should be the right
config variable that needs to be adjusted to what is actually needed
to make things work in this particular case.

Perhaps the current default of 200 MiB via USB_UEFI_PART_SIZE="200"
cf. https://github.com/rear/rear/pull/1205
has again become too little to make things work on nowadays usual systems?

Details:

In https://github.com/rear/rear/files/2516052/rear-HOSTNAME.log
there is (excerpts):

+ source /home/XXXXXXXXXX/Downloads/git/rear/usr/share/rear/output/USB/Linux-i386/100_create_efiboot.sh
...
++ mount /dev/disk/by-label/REAR-EFI /tmp/rear-efi.nUXYO
...
++ mkdir -p /tmp/rear-efi.nUXYO//EFI/BOOT
...
++ cp -v /boot/efi/EFI/ubuntu/grubx64.efi /tmp/rear-efi.nUXYO//EFI/BOOT/BOOTX64.efi
...
++ cp -pL -v /boot/vmlinuz-4.15.0-38-generic /tmp/rear-efi.nUXYO//EFI/BOOT/kernel
...
++ cp -p -v /tmp/rear.PhVohMXHGUVdCAm/tmp/initrd.cgz /tmp/rear-efi.nUXYO//EFI/BOOT/initrd.cgz
...
cp: error writing '/tmp/rear-efi.nUXYO//EFI/BOOT/initrd.cgz': No space left on device

jsmeix commented at 2018-10-26 14:33:

I increased the default USB_UEFI_PART_SIZE from 200 to 400 MiB via
https://github.com/rear/rear/commit/89e6a47c82fbfe8d059a4647755f94750fe28acc

MagnetoOptical commented at 2018-10-31 19:11:

This works as expected

jsmeix commented at 2018-11-05 14:28:

@ProfElectric
thanks for your feedback!
It helps a lot to get an explicit confirmation when things work.

[Export of Github issue for rear/rear.]