#202 Issue closed: Rear cann't mount /tmp/rear.xxxxxxxxx/outputfs when rescue media created in CentOS x64¶
Labels: support / question
altring opened issue at 2013-02-22 12:11:¶
Here is the situation.
When I run:
$ rear format /dev/sdc
in Centos 6.3 i386 kernel 2.6.32-279.22.1.el6.i386.
Then I boot in Centos 6.3 x86_64 kernel 2.6.32-279.22.1.el6.x86_64
and
rear mkbackup
Backup and restore of Centos 6.3 x86_64 has done without errors.
But when I run
$ rear format /dev/sdc
$ rear mkbackup
both in Centos 6.3 x86_64 I have the following lines in /var/log/rear/rear-hostname.log after executing "rear recover" command.
$ cat /var/log/rear/rear-hostname.log
2013-02-22 08:56:42 Relax-and-Recover 1.14 / Git
2013-02-22 08:56:42 Command line options: /bin/rear recover
2013-02-22 08:56:42 Using log file: /var/log/rear/rear-localhost.log
2013-02-22 08:56:42 Including /etc/rear/os.conf
2013-02-22 08:56:42 Including conf/Linux-i386.conf
2013-02-22 08:56:42 Including conf/GNU/Linux.conf
2013-02-22 08:56:42 Including /etc/rear/local.conf
2013-02-22 08:56:42 Including /etc/rear/rescue.conf
2013-02-22 08:56:42 Using build area '/tmp/rear.1aS3iekWHblvBCn'
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/rootfs'
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/tmp'
2013-02-22 08:56:42 Running recover workflow
2013-02-22 08:56:42 Running 'setup' stage
2013-02-22 08:56:42 Including setup/default/01_pre_recovery_script.sh
2013-02-22 08:56:42 Finished running 'setup' stage in 0 seconds
2013-02-22 08:56:42 Running 'verify' stage
2013-02-22 08:56:42 Including verify/default/02_cciss_scsi_engage.sh
2013-02-22 08:56:42 Including verify/default/02_translate_url.sh
2013-02-22 08:56:42 Including verify/default/03_translate_tape.sh
2013-02-22 08:56:42 Including verify/NETFS/default/05_check_NETFS_requirements.sh
2013-02-22 08:56:42 Skipping ping test
2013-02-22 08:56:42 Including verify/GNU/Linux/05_sane_recovery_check.sh
2013-02-22 08:56:42 Including verify/NETFS/default/07_set_backup_archive.sh
2013-02-22 08:56:42 Including verify/NETFS/default/08_start_required_daemons.sh
2013-02-22 08:56:42 Including verify/NETFS/default/09_set_readonly_options.sh
2013-02-22 08:56:42 Including verify/NETFS/default/10_mount_NETFS_path.sh
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:42 Mounting with 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs'
mount: unknown filesystem type 'ext3'
mount: you didn't specify a filesystem type for /dev/sdc1
I will try type ext3
2013-02-22 08:56:43 ERROR: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
=== Stack trace ===
Trace 0: /bin/rear:248 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:27 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:79 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:40 Source
Trace 4: /usr/share/rear/verify/NETFS/default/10_mount_NETFS_path.sh:11 source
Trace 5: /usr/share/rear/lib/global-functions.sh:124 mount_url
Trace 6: /usr/share/rear/lib/_input-output-functions.sh:131 StopIfError
Message: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
===================
2013-02-22 08:56:43 Running exit tasks.
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:43 Finished in 1 seconds
2013-02-22 08:56:43 Removing build area /tmp/rear.1aS3iekWHblvBCn
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn'
2013-02-22 08:56:43 End of program reached
$ cat /etc/rear/local.conf
OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
gdha commented at 2013-02-25 07:16:¶
Are you sure that rear format /dev/sdc worked correctly? The script
/usr/share/rear/format/USB/default/30_format_usb_disk.sh is
responsible for the formatting of the USB stick with an ext3 filesystem.
Also, check if the label on the USB stick was created correctly:
$ e2label /dev/sdc1
The output should be REAR-000
altring commented at 2013-02-25 13:15:¶
everything is ok.
Here is how output of mount command looks like:
[root@localhost ~]# mount
...
...
/dev/sdc1 on /media/REAR-000 type ext3 (rw,nosuid,nodev,uhelper=udisks)
I've managed to create media and was able to boot from it.
But
rear recover
fails
gdha commented at 2013-02-25 14:20:¶
What do you see in the log-file of rear as error?
altring commented at 2013-02-26 07:19:¶
Here is what I have:
...
...
2013-02-22 08:56:42 Including verify/NETFS/default/10_mount_NETFS_path.sh
mkdir: created directory `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:42 Mounting with 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs'
mount: unknown filesystem type 'ext3'
mount: you didn't specify a filesystem type for /dev/sdc1
I will try type ext3
2013-02-22 08:56:43 ERROR: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
=== Stack trace ===
Trace 0: /bin/rear:248 main
Trace 1: /usr/share/rear/lib/recover-workflow.sh:27 WORKFLOW_recover
Trace 2: /usr/share/rear/lib/framework-functions.sh:79 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:40 Source
Trace 4: /usr/share/rear/verify/NETFS/default/10_mount_NETFS_path.sh:11 source
Trace 5: /usr/share/rear/lib/global-functions.sh:124 mount_url
Trace 6: /usr/share/rear/lib/_input-output-functions.sh:131 StopIfError
Message: Mount command 'mount -v -o ro /dev/disk/by-label/REAR-000 /tmp/rear.1aS3iekWHblvBCn/outputfs' failed.
===================
2013-02-22 08:56:43 Running exit tasks.
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn/outputfs'
2013-02-22 08:56:43 Finished in 1 seconds
2013-02-22 08:56:43 Removing build area /tmp/rear.1aS3iekWHblvBCn
rmdir: removing directory, `/tmp/rear.1aS3iekWHblvBCn'
2013-02-22 08:56:43 End of program reached
gdha commented at 2013-02-26 08:30:¶
can you do the following? # grep EXT3 /boot/config-*
altring commented at 2013-02-26 11:03:¶
[root@localhost ~]# grep EXT3 /boot/config-*
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS=m
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS_XATTR=y
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS_POSIX_ACL=y
/boot/config-2.6.32-279.22.1.el6.x86_64:CONFIG_EXT3_FS_SECURITY=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS=m
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS_XATTR=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS_POSIX_ACL=y
/boot/config-2.6.32-279.el6.x86_64:CONFIG_EXT3_FS_SECURITY=y
gdha commented at 2013-02-26 12:18:¶
Please check the content of cat /etc/modules. Please paste it in
here.
Try modprobe ext3 before running rear -v recover
altring commented at 2013-02-26 12:23:¶
In rescue system
RESCUE localhost:~ # cat /etc/modules
r8169
r8169
r8169
usb_storage
RESCUE localhost:~ # modprobe ext3
FATAL: Module ext3 not found.
gdha commented at 2013-02-26 15:52:¶
Ok, we found the problem, right? Module ext3 was not copied to the
image.
altring commented at 2013-02-27 07:06:¶
Yes, we did it :)
gdha commented at 2013-02-27 11:30:¶
Can you proof this via the rear.log? Try
grep ext /var/log/rear/rear-$(hostname).log
I find this rather strange. Was there a mix up with another kernel tree
perhaps?
Try a find /lib/modules -name "ext*"
altring commented at 2013-02-27 12:23:¶
grep ext /var/log/rear/rear-$(hostname).log gives huuuuuge output.
grep ext3 /var/log/rear/rear-$(hostname).log gives:
2013-02-26 12:08:39 Binaries being copied: /bin/awk /bin/basename /bin/bash /bin/cat /bin/chmod /bin/chown /bin/cp /bin/cpio /bin/cut /bin/date /bin/dd /bin/df /bin/dmesg /bin/dumpkeys /bin/egrep /bin/fgrep /bin/find /bin/grep /bin/gzip /bin/hostname /bin/ipcalc /bin/kbd_mode /bin/ln /bin/loadkeys /bin/ls /bin/mkdir /bin/mknod /bin/mktemp /bin/mount /bin/mv /bin/netstat /bin/ping /bin/ps /bin/readlink /bin/rm /bin/rmdir /bin/sed /bin/sleep /bin/sort /bin/stty /bin/sync /bin/tar /bin/touch /bin/true /bin/umount /bin/uname /bin/usleep /bin/vi /lib/udev/ata_id /lib/udev/cdrom_id /lib/udev/edd_id /lib/udev/path_id /lib/udev/usb_id /root/rear/usr/sbin/rear /sbin/agetty /sbin/arping /sbin/blkid /sbin/cfdisk /sbin/dhclient /sbin/ethtool /sbin/fdisk /sbin/fsck /sbin/fsck.ext2 /sbin/fsck.ext3 /sbin/fsck.ext4 /sbin/fsck.ext4dev /sbin/grub /sbin/grub-install /sbin/halt /sbin/ifconfig /sbin/ifenslave /sbin/init /sbin/initctl /sbin/insmod /sbin/ip /sbin/killall5 /sbin/lsmod /sbin/mdadm /sbin/mingetty /sbin/mkfs /sbin/mkfs.ext2 /sbin/mkfs.ext3 /sbin/mkfs.ext4 /sbin/mkfs.ext4dev /sbin/mkswap /sbin/modprobe /sbin/nameif /sbin/parted /sbin/partprobe /sbin/pidof /sbin/poweroff /sbin/reboot /sbin/rmmod /sbin/route /sbin/rsyslogd /sbin/scsi_id /sbin/sfdisk /sbin/shutdown /sbin/swapoff /sbin/swapon /sbin/telinit /sbin/tune2fs /sbin/udevadm /sbin/udevd /usr/bin/attr /usr/bin/bzip2 /usr/bin/chacl /usr/bin/diff /usr/bin/dirname /usr/bin/du /usr/bin/expand /usr/bin/expr /usr/bin/file /usr/bin/free /usr/bin/getent /usr/bin/getfacl /usr/bin/getfattr /usr/bin/getopt /usr/bin/head /usr/bin/id /usr/bin/join /usr/bin/killall /usr/bin/ldd /usr/bin/less /usr/bin/logger /usr/bin/md5sum /usr/bin/mkfifo /usr/bin/openvt /usr/bin/rsync /usr/bin/scp /usr/bin/seq /usr/bin/setfacl /usr/bin/setfattr /usr/bin/sftp /usr/bin/ssh /usr/bin/stat /usr/bin/strings /usr/bin/tac /usr/bin/tail /usr/bin/tee /usr/bin/top /usr/bin/tr /usr/bin/tty /usr/bin/uniq /usr/bin/wc /usr/bin/which /usr/bin/xargs /usr/bin/xz /usr/libexec/openssh/sftp-server /usr/sbin/chroot /usr/sbin/sshd
`/sbin/fsck.ext3' -> `/tmp/rear.BYBzaZFHTUzPCcs/rootfs/bin/fsck.ext3'
`/sbin/mkfs.ext3' -> `/tmp/rear.BYBzaZFHTUzPCcs/rootfs/bin/mkfs.ext3'
I will try type ext3
/dev/sdc1 on /tmp/rear.BYBzaZFHTUzPCcs/outputfs type ext3 (rw,noatime)
I will try type ext3
/dev/sdc1 on /tmp/rear.BYBzaZFHTUzPCcs/outputfs type ext3 (rw,noatime)
block 287118: /usr/include/asm/sigcontext32.h
block 764385: /usr/share/man/man8/mkfs.ext3.8.gz
block 764669: /usr/share/man/man8/fsck.ext3.8.gz
block 1504679: /lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3/
block 1504680: /lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3/ext3.ko
block 1692696: /lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3/
block 1692697: /lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3/ext3.ko
block 2230653: /sbin/fsck.ext3
block 2230654: /sbin/mkfs.ext3
Mix with another kernel possible as there is an update to
2.6.32-279.22.1.el6.x86_64 kernel!
Rescue media was built under latest kernel!
Here is
find /lib/modules -name "ext*"
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext4
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext4/ext4.ko
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext3/ext3.ko
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext2
/lib/modules/2.6.32-279.22.1.el6.x86_64/kernel/fs/ext2/ext2.ko
/lib/modules/2.6.32-279.22.1.el6.x86_64/extra
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext4
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext4/ext4.ko
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext3/ext3.ko
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext2
/lib/modules/2.6.32-279.el6.x86_64/kernel/fs/ext2/ext2.ko
/lib/modules/2.6.32-279.el6.x86_64/extra
gdha commented at 2013-03-04 15:40:¶
@altring could you give it another try with latest github repo?
altring commented at 2013-03-28 14:01:¶
It works!
Now we have in rescue system:
RESCUE localhost:~ # cat /etc/modules
e100
ext3
usb_storage
And restore works.
I think problem resolved.
gdha commented at 2013-03-31 15:13:¶
Thanks for testing it out...and glad it is resolved.
TimDowker commented at 2013-04-10 14:21:¶
Trying to do a recovery to dis-similar hardware and keep getting "Mount command failed" error. I have confirmed the ext3 module is loaded and I've tried re-running the backup just to be sure. I'm running the latest GIT - synced yesterday, with an external USB drive. Any thoughts?
Here's the output of my backup command - the format of the USB device also gave no errors.
sudo usr/sbin/rear -v mkbackup
Relax-and-Recover 1.14 / Git
Using log file: /home/tdowker/rear/var/log/rear/rear-MiddleSrv2.log
Creating disk layout
Creating root filesystem layout
WARNING: Could not determine network driver for 'bond0'. Please make
WARNING: sure that it loads automatically (e.g. via udev) or add
WARNING: it to MODULES_LOAD in
/home/tdowker/rear/etc/rear/{local,site}.conf!
TIP: To login as root via ssh you need to set up
/root/.ssh/authorized_keys
Copying files and directories
/home/tdowker/rear/etc/rear/local.conf' ->/tmp/rear.QdaY97YhodaPrae/rootfs/etc/rear/local.conf'
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Writing MBR to /dev/sdc
Copying resulting files to usb location
Encrypting disabled
Creating tar archive
'/tmp/rear.QdaY97YhodaPrae/outputfs/rear/MiddleSrv2/20130409.1009/backup.tar.gz'
Archived 150873 MiB [avg 16548 KiB/sec]OK
Archived 150873 MiB in 9337 seconds [avg 16546 KiB/sec]
[Export of Github issue for rear/rear.]