boot': Read-only file system when recover¶

Labels: waiting for info, support / question, no-issue-activity

cvijayvinoth opened issue at 2020-11-25 06:49:¶

ReaR version ("/usr/sbin/rear -V"):
Relax-and-Recover 2.5 / Git
OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):

NAME="Ubuntu"
VERSION="16.04.7 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.7 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

OUTPUT=ISO
BACKUP=RSYNC
RSYNC_PREFIX="$HOSTNAME"
BACKUP_PROG="/var/www/html/imageBackup/rsync"
OUTPUT_URL=rsync://xxx@xxxxx::xxxx
BACKUP_URL=rsync://xxxx@xxxxx::xxxx
#ISO_DIR="$(</etc/rear/path.txt)/imageBackup/iso/$HOSTNAME"
ISO_DIR="/var/www/html/imageBackup/iso/$HOSTNAME"
MESSAGE_PREFIX="$$: "
BACKUP_RSYNC_OPTIONS+=(-z --progress --password-file=/etc/rear/xxxx)
BACKUP_PROG_EXCLUDE+=( "$(</etc/rear/path.txt)/imageBackup/iso/*"  )
PROGRESS_MODE="plain"
AUTOEXCLUDE_PATH=( /tmp )
PROGRESS_WAIT_SECONDS="1"
export TMPDIR="/var/www/html/imageBackup/iso/"
PXE_RECOVER_MODE=automatic
ISO_FILES=("/var/www/html/imageBackup/rsync")
ISO_PREFIX="${HOSTNAME}"
ISO_DEFAULT="automatic"
USE_DHCLIENT="Yes"

Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR):
Virtual Machine
System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device):
x86 compatible
Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot):
GRUB
Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe):
local disk
Storage layout ("lsblk -ipo NAME,KNAME,PKNAME,TRAN,TYPE,FSTYPE,SIZE,MOUNTPOINT" or "lsblk" as makeshift):

NAME        KNAME     PKNAME   TRAN   TYPE FSTYPE   SIZE MOUNTPOINT
/dev/sr0    /dev/sr0           ata    rom          1024M
/dev/sda    /dev/sda           sata   disk          500G
|-/dev/sda4 /dev/sda4 /dev/sda        part ext4   270.1G /home
|-/dev/sda2 /dev/sda2 /dev/sda        part ext4     954M /boot
|-/dev/sda3 /dev/sda3 /dev/sda        part swap     8.4G [SWAP]
`-/dev/sda1 /dev/sda1 /dev/sda        part ext4   220.6G /

Description of the issue (ideally so that others can reproduce it):
When i am trying to perform recover operation unable to restore data.
filesystem of type ext4 with mount point /boot on /dev/sda2 getting failed.
and getting error like
mkdir: cannot create directory '/mnt/local/boot': Read-only file system
Workaround, if any:
Attachments, as applicable ("rear -D mkrescue/mkbackup/recover" debug log files):
recover log attached -- rear-snigdhaubuntu32.log
rear-snigdhaubuntu32.log

jsmeix commented at 2020-11-25 10:23:¶

@cvijayvinoth
your https://github.com/rear/rear/files/5594867/rear-snigdhaubuntu32.log
is not a rear -D recover debug log file but it is a text file that begins with

Reg Ticket : ID110124129
Hi Deepak,
...

See "Debugging issues with Relax-and-Recover" at
https://en.opensuse.org/SDB:Disaster_Recovery
how to get a rear -D recover debug log file out of the running recovery system.

cvijayvinoth commented at 2020-11-25 11:02:¶

rear-snigdhaubuntu32.log
Sorry for the wrong log file

jsmeix commented at 2020-11-25 12:10:¶

Excerpts from your
https://github.com/rear/rear/files/5596237/rear-snigdhaubuntu32.log

++ echo 'mkdir -p "/mnt/local/"'
++ echo 'mount -o rw,relatime,errors=remount-ro,data=ordered /dev/sda1 "/mnt/local/"'
++ echo 'mount -o rw,relatime,errors=remount-ro,data=ordered,remount,user_xattr /dev/sda1 "/mnt/local/"'
...
++ echo 'mkdir -p "/mnt/local/boot"'
++ echo 'mount -o rw,relatime,data=ordered /dev/sda2 "/mnt/local/boot"'
++ echo 'mount -o rw,relatime,data=ordered,remount,user_xattr /dev/sda2 "/mnt/local/boot"'
...
++ source /var/lib/rear/layout/diskrestore.sh
...
+++ mkdir -p /mnt/local/
+++ mount -o rw,relatime,errors=remount-ro,data=ordered /dev/sda1 /mnt/local/
+++ mount -o rw,relatime,errors=remount-ro,data=ordered,remount,user_xattr /dev/sda1 /mnt/local/
...
+++ mkdir -p /mnt/local/boot
mkdir: cannot create directory '/mnt/local/boot': Read-only file system

In contrast excerpts from a "rear -D recover"
on one of my test systems (with LVM):

++ echo 'mkdir -p /mnt/local/'
++ echo 'mount -o rw,relatime,data=ordered /dev/mapper/system-root /mnt/local/'
++ echo 'mount -o rw,relatime,data=ordered,remount,user_xattr /dev/mapper/system-root /mnt/local/'
...
++ echo 'mkdir -p /mnt/local/home'
++ echo 'mount -o rw,relatime,data=ordered /dev/mapper/system-home /mnt/local/home'
++ echo 'mount -o rw,relatime,data=ordered,remount,user_xattr /dev/mapper/system-home /mnt/local/home'
...
++ source /var/lib/rear/layout/diskrestore.sh
...
+++ mkdir -p /mnt/local/
+++ mount -o rw,relatime,data=ordered /dev/mapper/system-root /mnt/local/
+++ mount -o rw,relatime,data=ordered,remount,user_xattr /dev/mapper/system-root /mnt/local/
...
+++ mkdir -p /mnt/local/home
+++ mount -o rw,relatime,data=ordered /dev/mapper/system-home /mnt/local/home
+++ mount -o rw,relatime,data=ordered,remount,user_xattr /dev/mapper/system-home /mnt/local/home

The difference is that in your case the mount options for /mnt/local/
contain errors=remount-ro while in my case they don't.

I would assume errors=remount-ro only means to remount in read-only mode
only in case of errors so that

mount -o rw,relatime,errors=remount-ro,data=ordered,remount,user_xattr /dev/sda1 /mnt/local/

would remount /mnt/local/ in read-write mode (because of the rw)
with the additioal option to remount it in read-only mode in case of errors.

But I am not a sufficient expert of subtle details and possible interdependencies
of mount options so my above assumption could be wrong.

The mount options are stored in var/lib/rear/layout/disklayout.conf
for example my disklayout.conf looks like this (excerpt):

fs /dev/mapper/system-root / ext4 uuid=b461ac56-7f9c-46a2-9611-8d4346b794ae label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16356 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered
fs /dev/mapper/system-home /home ext4 uuid=00b4b672-9a60-4966-9dc2-8c0765e0a4cc label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16356 default_mount_options=user_xattr,acl options=rw,relatime,data=ordered

Accordingly it should help to edit your var/lib/rear/layout/disklayout.conf
when you are logged in as root inside your booted ReaR recovery system and
remove the possibly disturbing errors=remount-ro mount option in your case
and afterwards run "rear -D recover".

This way your file systems should be recreated without the errors=remount-ro mount option
which is not exactly as it was before but you could at least get your system recreated.

github-actions commented at 2021-01-25 02:59:¶

Stale issue message

[Export of Github issue for rear/rear.]