#160 Issue closed: vol_id doesn't exist on ubuntu anymore¶
Labels: bug, cleanup
kkaiser opened issue at 2012-09-19 10:08:¶
When trying to create a USB-Stick it fails because of the command vol_id in:
https://github.com/rear/rear/blob/master/usr/share/rear/format/USB/default/20_check_usb_layout.sh
(line 39)
blkid -o udev should work. Here is my /etc/rear/local.conf config:
# sample local configuration
### write the rescue initramfs to USB and update the USB bootloader
OUTPUT=USB
### create a backup using the internal NETFS method, using 'tar'
BACKUP=NETFS
### write both rescue image and backup to the device labeled REAR-000
BACKUP_URL=usb:///dev/sdb1/REAR-000
dagwieers commented at 2012-09-19 20:14:¶
As usual with bug-reports, we expect an exact version of Ubuntu and Relax-and-Recover together with all possible relevant information.
kkaiser commented at 2012-09-20 06:29:¶
Relax-and-Recover 1.13.0-git201209071215 / 2012-09-07
Using log file: /var/log/rear/rear-beryllium.log.lockless
Dumping out configuration and system information
This is a 'Linux-x86_64' system, compatible with 'Linux-i386'.
System definition:
ARCH = Linux-i386
OS = GNU/Linux
OS_MASTER_VENDOR = Debian
OS_MASTER_VERSION = 11.04
OS_MASTER_VENDOR_ARCH = Debian/i386
OS_MASTER_VENDOR_VERSION = Debian/11.04
OS_MASTER_VENDOR_VERSION_ARCH = Debian/11.04/i386
OS_VENDOR = Ubuntu
OS_VERSION = 11.04
OS_VENDOR_ARCH = Ubuntu/i386
OS_VENDOR_VERSION = Ubuntu/11.04
OS_VENDOR_VERSION_ARCH = Ubuntu/11.04/i386
Configuration tree:
Linux-i386.conf : OK
GNU/Linux.conf : OK
Debian.conf : missing/empty
Debian/i386.conf : missing/empty
Debian/11.04.conf : missing/empty
Debian/11.04/i386.conf : missing/empty
Ubuntu.conf : missing/empty
Ubuntu/i386.conf : missing/empty
Ubuntu/11.04.conf : missing/empty
Ubuntu/11.04/i386.conf : missing/empty
site.conf : missing/empty
local.conf : OK
Backup with NETFS
NETFS_KEEP_OLD_BACKUP_COPY =
NETFS_PREFIX = beryllium
BACKUP_MOUNTCMD =
BACKUP_OPTIONS =
BACKUP_RSYNC_OPTIONS = --sparse --archive --hard-links --verbose --numeric-ids --stats
BACKUP_SELINUX_DISABLE = 1
BACKUP_UMOUNTCMD =
BACKUP_URL = usb:///dev/sdb1/REAR-000
Backup program is 'tar':
BACKUP_PROG = tar
BACKUP_PROG_ARCHIVE = backup
BACKUP_PROG_COMPRESS_OPTIONS = --gzip
BACKUP_PROG_COMPRESS_SUFFIX = .gz
BACKUP_PROG_EXCLUDE = /tmp/* /dev/shm/* /a/* /afs/* /SCR /scr
BACKUP_PROG_INCLUDE =
BACKUP_PROG_OPTIONS =
BACKUP_PROG_OPTIONS_CREATE_ARCHIVE =
BACKUP_PROG_OPTIONS_RESTORE_ARCHIVE =
BACKUP_PROG_SUFFIX = .tar
BACKUP_PROG_WARN_PARTIAL_TRANSFER = 1
Output to USB
USB_DEVICE =
USB_FILES =
USB_RETAIN_BACKUP_NR = 2
RESULT_MAILTO =
/usr/share/rear/lib/validated/Ubuntu/11.04/i386.txt
Your system is not yet validated. Please carefully check all functions
and create a validation record with 'rear validate'. This will help others
to know about the validation status of Relax-and-Recover on this system.
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 11.04
Release: 11.04
Codename: natty
Relax-and-Recover 1.13.0-git201209071215 / 2012-09-07
kkaiser commented at 2012-09-20 06:53:¶
$ rear format /dev/sdb
USB device /dev/sdb must be formatted with ext2/3/4 or btrfs file system
Please type Yes to format /dev/sdb in ext3 format:
yes
ERROR: Abort USB format process by user
Aborting due to an error, check /var/log/rear/rear-beryllium.log for details
Terminated
root@beryllium:~ > cat /var/log/rear/rear-beryllium.log
2012-09-20 08:52:36 Relax-and-Recover 1.13.0-git201209071215 / 2012-09-07
2012-09-20 08:52:36 Command line options: /usr/sbin/rear format /dev/sdb
2012-09-20 08:52:36 Including conf/Linux-i386.conf
2012-09-20 08:52:36 Including conf/GNU/Linux.conf
2012-09-20 08:52:36 Including /etc/rear/local.conf
2012-09-20 08:52:36 Using build area '/tmp/rear.bAtr9iIaOgD380r'
2012-09-20 08:52:36 Running format workflow
2012-09-20 08:52:36 Running 'format' stage
2012-09-20 08:52:36 Including format/USB/default/20_check_usb_layout.sh
/usr/share/rear/format/USB/default/20_check_usb_layout.sh: line 39: vol_id: command not found
2012-09-20 08:52:45 ERROR: Abort USB format process by user
=== Stack trace ===
Trace 0: /usr/sbin/rear:245 main
Trace 1: /usr/share/rear/lib/format-workflow.sh:46 WORKFLOW_format
Trace 2: /usr/share/rear/lib/framework-functions.sh:79 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:40 Source
Trace 4: /usr/share/rear/format/USB/default/20_check_usb_layout.sh:48 source
Trace 5: /usr/share/rear/lib/_input-output-functions.sh:131 StopIfError
Message: Abort USB format process by user
===================
2012-09-20 08:52:45 Running exit tasks.
2012-09-20 08:52:45 Finished in 9 seconds
2012-09-20 08:52:45 Removing build area /tmp/rear.bAtr9iIaOgD380r
2012-09-20 08:52:45 End of program reached
kkaiser commented at 2012-09-20 07:09:¶
I tested "blkid -o udev" but doesn't work either. I also tried version: Relax-and-Recover 1.14-git201209200242 / 2012-09-20
heres the log with blkid when running rear format /dev/sdb
2012-09-20 09:06:02 Relax-and-Recover 1.14-git201209200242 / 2012-09-20
2012-09-20 09:06:02 Command line options: /usr/sbin/rear format /dev/sdb
2012-09-20 09:06:02 Including conf/Linux-i386.conf
2012-09-20 09:06:02 Including conf/GNU/Linux.conf
2012-09-20 09:06:02 Including /etc/rear/local.conf
2012-09-20 09:06:02 Using build area '/tmp/rear.x3kWlwW0uKt7ztJ'
2012-09-20 09:06:02 Running format workflow
2012-09-20 09:06:02 Running 'format' stage
2012-09-20 09:06:02 Including format/USB/default/20_check_usb_layout.sh
2012-09-20 09:06:08 ERROR: Abort USB format process by user
=== Stack trace ===
Trace 0: /usr/sbin/rear:245 main
Trace 1: /usr/share/rear/lib/format-workflow.sh:46 WORKFLOW_format
Trace 2: /usr/share/rear/lib/framework-functions.sh:79 SourceStage
Trace 3: /usr/share/rear/lib/framework-functions.sh:40 Source
Trace 4: /usr/share/rear/format/USB/default/20_check_usb_layout.sh:48 source
Trace 5: /usr/share/rear/lib/_input-output-functions.sh:131 StopIfError
Message: Abort USB format process by user
===================
2012-09-20 09:06:08 Running exit tasks.
2012-09-20 09:06:08 Finished in 6 seconds
2012-09-20 09:06:08 Removing build area /tmp/rear.x3kWlwW0uKt7ztJ
2012-09-20 09:06:08 End of program reached
thumm commented at 2012-09-20 07:52:¶
vol_id has been removed from udev (see <a href=http://git.kernel.org/?p=linux/hotplug/udev.git;a=commit;h=f07996885dab45102492d7f16e7e2997e264c725>here)
Could you check it after applying the following patch (which I already tested on Ubuntu 12.04 precise)
diff --git a/usr/share/rear/format/USB/default/20_check_usb_layout.sh b/usr/share/rear/format/USB/default/20_check_usb_layout.sh
index b76eb2b..533a66d 100644
--- a/usr/share/rear/format/USB/default/20_check_usb_layout.sh
+++ b/usr/share/rear/format/USB/default/20_check_usb_layout.sh
@@ -36,7 +36,12 @@ StopIfError "Unable to determine raw USB device for $REAL_USB_DEVICE"
answer=""
ID_FS_TYPE=
-eval $(vol_id "$REAL_USB_DEVICE")
+which vol_id > /dev/null 2>&1
+if [[ $? != 0 ]]; then
+ eval $(blkid -o udev "$REAL_USB_DEVICE")
+else
+ eval $(vol_id "$REAL_USB_DEVICE")
+fi
StopIfError "Could not determine filesystem info for '$REAL_USB_DEVICE'"
[[ "$ID_FS_TYPE" == @(btr*|ext*) ]]
kkaiser commented at 2012-09-20 08:34:¶
yep I can confirm that the patch works. The mistake I made when trying blkid was typing "yes" instead of "Yes". That is why "Abort USB format process by user" was displayed.
thank you
dagwieers commented at 2012-09-20 09:50:¶
@thumm Thanks for looking into this. However what we probably need to do
is ressurect rear_vol_id from the grave. We put it to rest back when
there was no longer a need to use vol_id (thanks to the new layout
code). So I was a bit surprised that there was still a need for vol_id
(and the related kludges it brings in).
You can find the removal of rear_vol_id here:
https://github.com/rear/rear/commit/8bcb4ba05d0efd04c947a0dc193f2ef77f74b171
If you search the issues/commits for vol_id and rear_vol_id, you can
get a feeling of what the context of these changes was. If there is a
way to get rid of vol_id altogether (maybe using the layout-code) that
would be even better...
thumm commented at 2012-09-25 14:56:¶
@dagwieers Do you really want to (re-) use rear_vol_id?
The format-workflow is the last one using vol_id (that's what
grep -nri "vol_id" . says) and would only need a very
small part of rear_vol_id: given a device -> does this
device already has a ext* or btr filesystem.
In the layout functions mount is used to determine the
filesystem of a device. I would prefer a solution which uses the same
approach, e. g. something like this
diff --git a/usr/share/rear/format/USB/default/20_check_usb_layout.sh b/usr/share/rear/format/USB/default/20_check_usb_layout
index b76eb2b..50263a0 100644
--- a/usr/share/rear/format/USB/default/20_check_usb_layout.sh
+++ b/usr/share/rear/format/USB/default/20_check_usb_layout.sh
@@ -36,7 +36,7 @@ StopIfError "Unable to determine raw USB device for $REAL_USB_DEVICE"
answer=""
ID_FS_TYPE=
-eval $(vol_id "$REAL_USB_DEVICE")
+ID_FS_TYPE=$(get_filesystem $REAL_USB_DEVICE)
StopIfError "Could not determine filesystem info for '$REAL_USB_DEVICE'"
[[ "$ID_FS_TYPE" == @(btr*|ext*) ]]
diff --git a/usr/share/rear/lib/layout-functions.sh b/usr/share/rear/lib/layout-functions.sh
index 8cfc865..60ba08c 100644
--- a/usr/share/rear/lib/layout-functions.sh
+++ b/usr/share/rear/lib/layout-functions.sh
@@ -447,3 +447,26 @@ get_block_size() {
fi
}
+# Get the filesystem of a partition
+get_filesystem () {
+ local name=$1
+ local tmp_mount_dir=$(mktemp -d $TMP_DIR/rear.formatXXX)
+
+ mkdir -p $tmp_mount_dir
+
+ [[ -d $tmp_mount_dir ]]
+ BugIfError "Unable to create dir $tmp_mount_dir"
+
+ mount $1 $tmp_mount_dir 2> /dev/null
+ if (( $? == 0 )); then
+ echo $(mount | grep $name | cut -d" " -f5)
+ umount $tmp_mount_dir
+ else
+ echo "unknown"
+ fi
+
+ rmdir $tmp_mount_dir
+
+ return 0
+}
+
dagwieers commented at 2012-09-25 22:16:¶
I wouldn't just mount, because that changes the filesystem, rather use readonly-mounting is better.
That said, I am not convinced. It seems quite costly to do this, and
there is a risk that a mount (attempt) gives problems we would otherwise
not see. I wonder how exactly udev/vol_id do this. I expect they scan
the filesystem header, rather than mounting. That's why I prefer that
approach. The only reason why rear_vol_id is such a kludge is because
it has a lot of backward compatibility built in.
I love to hear what others think.
dagwieers commented at 2012-09-26 11:00:¶
Ok, the preferred solution is to use file on, the block device. This
is lightweight. So something like this should do:
file_output=$(file -sbL "$device")
case "$file_output" in
(*ext2 filesystem*)
filesystem=ext2
;;
(*ext3 filesystem*)
filesystem=ext3
;;
(*ext4 filesystem*)
filesystem=ext4
;;
### FIXME: Also test for btrfs
(*)
Error "Unsupported filesystem found on device $device ($file_output)"
;;
Is this acceptable ? Can you modify the patch in this way ?
thumm commented at 2012-09-26 16:04:¶
Hm, a miracle has happened between mount and
file :-)
Yes, I can modify the patch, but I would like to unterstand the
difference between using file and mount. Why
is file more lightweight?
dagwieers commented at 2012-09-26 17:40:¶
With file you simply read a few blocks (in my tests it is the first
262k) of the disk, with mount you do much more:
- register the mount to the kernel
- verify filesystem metadata
- open the filesystem
- update filesystem metadata
and you require to umount it, which undoes the above again. Besides
mount and umount can fail in considerably more ways than file can,
although it probably knows more about filesystems itself. One advantage
for file is that you don't need the kernel modules in order to make it
work. One disadvantage may be that the magic files may not support new
or experimental filesystems. But that's maybe a good thing.
thumm commented at 2012-09-27 12:41:¶
Ok, that's clear now. Thank you very much. I'll send a pull request.
dagwieers commented at 2012-09-28 06:25:¶
This fixes #160.
[Export of Github issue for rear/rear.]