#335 Issue closed: Error in a recover operation with Debian and rear 1.14¶
Labels: documentation
vincentCegedim opened issue at 2013-11-22 11:26:¶
Hi,
I have a big trouble restoring a Debian 7.1 machine with rear 1.14;
partitionning the disk fails if destination disk is bigger than original
one. In this case, it seems that logical partition is made too small to
contain LVM one.
If disk has the same size, it works fine.
But anyway, thank you for that wonderful piece of code that saves me
multiple times
Vincent
jhoekx commented at 2013-11-22 11:47:¶
Can you post the /var/lib/rear/layout/disklayout.conf file and the
rear -D recover logfile?
If the destination disk is smaller, then it's possible that LVM fails. When it's larger, it should only increase the size.
vincentCegedim commented at 2013-11-22 12:29:¶
In my exemple, the new disk was much larger (25Go => 45Go)
Here the disklayout file:
disk /dev/sda 26843545600 msdos part /dev/sda 499122176 1048576 primary boot /dev/sda1 part /dev/sda 1024 501218304 extended none /dev/sda2 part /dev/sda 26341277696 501219328 logical lvm /dev/sda5 lvmdev /dev/vgroot /dev/sda5 XeeNbl-BV4G-Clgr-TWIn-7E1b-gP0c-uEiQyW 51447808 lvmgrp /dev/vgroot 4096 6280 25722880 lvmvol /dev/vgroot lvvar 953 7806976 lvmvol /dev/vgroot lvhome 238 1949696 lvmvol /dev/vgroot lvopt 715 5857280 lvmvol /dev/vgroot lvtmp 384 3145728 lvmvol /dev/vgroot lvswap 476 3899392 lvmvol /dev/vgroot lvroot 1907 15622144 fs /dev/mapper/vgroot-lvroot / ext4 uuid=bd61f8d4-b5d6-49e3-9b48-26788ab2965b label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16368 options=rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered fs /dev/sda1 /boot ext4 uuid=ddf5cb3d-d86b-4dfd-a8b1-ff9b7e445110 label= blocksize=1024 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=4093 options=rw,relatime,user_xattr,barrier=1,data=ordered fs /dev/mapper/vgroot-lvhome /home ext4 uuid=25bd415d-241a-4724-b6f9-4664352cf0a5 label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16384 options=rw,relatime,user_xattr,barrier=1,data=ordered fs /dev/mapper/vgroot-lvopt /opt ext4 uuid=af751048-859e-4de9-ae14-6d4b5123c69b label= blocksize=4096 reserved_blocks=5% max_mounts=-1 check_interval=0d bytes_per_inode=16363 options=rw,relatime,user_xattr,barrier=1,data=ordered fs /dev/mapper/vgroot-lvtmp /tmp ext4 uuid=61f06855-c6de-46f9-b250-f3b8f37b314e label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=17623 options=rw,relatime,user_xattr,barrier=1,data=ordered fs /dev/mapper/vgroot-lvvar /var ext4 uuid=727dc7e2-c49d-45f8-8362-a7549bdd3f7f label= blocksize=4096 reserved_blocks=4% max_mounts=-1 check_interval=0d bytes_per_inode=16360 options=rw,relatime,user_xattr,barrier=1,data=ordered swap /dev/mapper/vgroot-lvswap uuid=b0048c2d-3360-477f-8136-f3f820099b3d label=
vincentCegedim commented at 2013-11-22 12:57:¶
And some logs
+++ echo '2013-11-22 13:45:26 Creating partitions for disk /dev/sda (msdos)'
2013-11-22 13:45:26 Creating partitions for disk /dev/sda (msdos)
+++ Print 'Creating partitions for disk /dev/sda (msdos)'
+++ test 1
+++ echo -e 'Creating partitions for disk /dev/sda (msdos)'
+++ parted -s /dev/sda mklabel msdos
+++ parted -s /dev/sda mkpart primary 32768B 499154943B
Warning: The resulting partition is not properly aligned for best performance.
+++ parted -s /dev/sda set 1 boot on
+++ parted -s /dev/sda mkpart extended 499159040B 48318382079B
Warning: The resulting partition is not properly aligned for best performance.
+++ parted -s /dev/sda mkpart logical 499163136B 2646646782B
Warning: The resulting partition is not properly aligned for best performance.
+++ parted -s /dev/sda set 5 lvm on
+++ partprobe -s /dev/sda
/dev/sda: msdos partitions 1 2 <5>
[...]
2013-11-22 13:45:36 Creating LVM PV /dev/sda5
+++ Print 'Creating LVM PV /dev/sda5'
+++ test 1
+++ echo -e 'Creating LVM PV /dev/sda5'
+++ lvm vgchange -a n vgroot
Volume group "vgroot" not found
+++ true
+++ lvm pvcreate -ff --yes -v --uuid XeeNbl-BV4G-Clgr-TWIn-7E1b-gP0c-uEiQyW --norestorefile /dev/sda5
Set up physical volume for "/dev/sda5" with 4194304 available sectors
Writing physical volume data to disk "/dev/sda5"
Physical volume "/dev/sda5" successfully created
+++ component_created pv:/dev/sda5 lvmdev
+++ local device=pv:/dev/sda5
+++ local type=lvmdev
+++ local touchfile=lvmdev-pv:-dev-sda5
+++ touch /tmp/rear.eOwfN97I2KNNYF8/tmp/touch/lvmdev-pv:-dev-sda5
+++ create_component /dev/vgroot lvmgrp
+++ local device=/dev/vgroot
+++ local type=lvmgrp
+++ local touchfile=lvmgrp--dev-vgroot
+++ '[' -e /tmp/rear.eOwfN97I2KNNYF8/tmp/touch/lvmgrp--dev-vgroot ']'
+++ return 0
+++ LogPrint 'Creating LVM VG vgroot'
+++ Log 'Creating LVM VG vgroot'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2013-11-22 13:45:36 Creating LVM VG vgroot'
2013-11-22 13:45:36 Creating LVM VG vgroot
+++ Print 'Creating LVM VG vgroot'
+++ test 1
+++ echo -e 'Creating LVM VG vgroot'
+++ '[' -e /dev/vgroot ']'
+++ rm -rf /dev/vgroot
+++ lvm vgcreate --physicalextentsize 4096k vgroot /dev/sda5
Volume group "vgroot" successfully created
+++ lvm vgchange --available y vgroot
0 logical volume(s) in volume group "vgroot" now active
+++ component_created /dev/vgroot lvmgrp
+++ local device=/dev/vgroot
+++ local type=lvmgrp
+++ local touchfile=lvmgrp--dev-vgroot
+++ touch /tmp/rear.eOwfN97I2KNNYF8/tmp/touch/lvmgrp--dev-vgroot
+++ create_component /dev/mapper/vgroot-lvvar lvmvol
+++ local device=/dev/mapper/vgroot-lvvar
+++ local type=lvmvol
+++ local touchfile=lvmvol--dev-mapper-vgroot-lvvar
+++ '[' -e /tmp/rear.eOwfN97I2KNNYF8/tmp/touch/lvmvol--dev-mapper-vgroot-lvvar ']'
+++ return 0
+++ LogPrint 'Creating LVM volume vgroot/lvvar'
+++ Log 'Creating LVM volume vgroot/lvvar'
+++ test 1 -gt 0
++++ Stamp
++++ date '+%Y-%m-%d %H:%M:%S '
+++ echo '2013-11-22 13:45:36 Creating LVM volume vgroot/lvvar'
2013-11-22 13:45:36 Creating LVM volume vgroot/lvvar
+++ Print 'Creating LVM volume vgroot/lvvar'
+++ test 1
+++ echo -e 'Creating LVM volume vgroot/lvvar'
+++ lvm lvcreate -l 953 -n lvvar vgroot
Volume group "vgroot" has insufficient free space (511 extents): 953 required.
++ (( 1 == 0 ))
++ LogPrint 'An error occured during layout recreation.'
++ Log 'An error occured during layout recreation.'
++ test 1 -gt 0
+++ Stamp
+++ date '+%Y-%m-%d %H:%M:%S '
++ echo '2013-11-22 13:45:36 An error occured during layout recreation.'
2013-11-22 13:45:36 An error occured during layout recreation.
gdha commented at 2013-11-22 17:51:¶
Strange, you did not get a question about disk migration?
vincentCegedim commented at 2013-11-26 09:54:¶
Yes, I think. I cannot write here the full log file, it is too large (21253 lines). I don't know where to put it to give you acces. If you have an idea...
gdha commented at 2013-11-26 10:00:¶
you can use gist - https://gist.github.com/ - to attach a large file
vincentCegedim commented at 2013-11-26 16:42:¶
OK. I had to cut the file in two parts (I did not find how to simply
upload a file, I am brand new to Github).
Part 1
Part 2
gdha commented at 2013-11-27 15:20:¶
Thank you for the logs - it was very useful!
see script layout/prepare/default/40_autoresize_disks.sh and a piece
out of the 2th log from you:
new_size=$(echo "$partition_size $resizeable_space $available_space" | awk '{ printf "%d", ($1/$2)*$3; }')
++ for data in '"${partitions[@]}"'
++ name=/dev/sda5
++ partition_size=26341277696
+++ awk '{ printf "%d", ($1/$2)*$3; }'
+++ echo '26341277696 26341278720 47819259904'
++ new_size=2147483647
echo '26341277696 26341278720 47819259904' | awk '{ printf "%d", ($1/$2)*$3; }'
47819258045
As you can see the math calculated on your system is wrong (don't ask me
why). The awk version I used was GNU Awk 3.1.8
@jhoekx have you ever seen such behavior before? I didn't
jhoekx commented at 2013-11-27 15:26:¶
It looks like some overflow. I was using awk because bash couldn't handle those large numbers.
What's the awk version you have? Is your OS 32 or 64 bit?
I checked on my system (4.1.0), RHEL 5 (3.1.5 i386) and RHEL 6 (3.1.7) and all of them are doing it correctly.
vincentCegedim commented at 2013-11-27 15:34:¶
It is a debian 7.1 64 bits.
awk => Version: 1:4.0.1+dfsg-2.1
gdha commented at 2013-11-27 15:38:¶
@vincentCegedim could you execute the following please?
echo '26341277696 26341278720 47819259904' | awk '{ printf "%d", ($1/$2)*$3; }'
awk --version
vincentCegedim commented at 2013-11-27 15:40:¶
# echo '26341277696 26341278720 47819259904' | awk '{ printf "%d", ($1/$2)*$3; }'
2147483647
# awk --version
awk: not an option: --version
# apt-cache show gawk
Package: gawk
Version: 1:4.0.1+dfsg-2.1
Installed-Size: 2269
Maintainer: Arthur Loiret
Architecture: amd64
Provides: awk
Pre-Depends: libc6 (>= 2.11), libreadline6 (>= 6.0), libsigsegv2
Suggests: gawk-doc
Description-en: GNU awk, a pattern scanning and processing language
`awk', a program that you can use to select particular records in a
file and perform operations upon them.
.
Gawk is the GNU Project's implementation of the AWK programming language.
It conforms to the definition of the language in the POSIX 1003.2 Command
Language And Utilities Standard. This version in turn is based on the
description in The AWK Programming Language, by Aho, Kernighan, and
Weinberger, with the additional features defined in the System V Release
4 version of UNIX awk. Gawk also provides more recent Bell Labs awk
extensions, and some GNU-specific extensions.
Multi-Arch: foreign
Homepage: http://www.gnu.org/software/gawk/
Description-md5: 0dce81fe543994c3caea814e2777732e
Tag: devel::interpreter, implemented-in::c, interface::commandline,
role::program, scope::utility, suite::gnu, use::converting,
use::filtering, use::scanning, works-with::text
Section: interpreters
Priority: optional
Filename: pool/main/g/gawk/gawk_4.0.1+dfsg-2.1_amd64.deb
Size: 971710
MD5sum: 984730ae49afce4befd7e31c0298a5a7
SHA1: 4d6065666181a430b961a0a6393db2562f908038
SHA256: 3e5be94893eb76d9c396e111111cf40fee63b351166e4b9735e36c4c64974540
vincentCegedim commented at 2013-11-27 15:48:¶
I just try an apt-get install gawk before retrying the
test. Here is the result
# echo '26341277696 26341278720 47819259904' | awk '{ printf "%d", ($1/$2)*$3; }'
47819258045
I do a new cycle mkrescure/recover and tell you the result.
Thank you.
gdha commented at 2013-11-27 15:51:¶
@vincentCegedim Did you install a new version of gawk ? if yes, where
was the previous awk coming from??
vincentCegedim commented at 2013-11-27 16:03:¶
YES ! That was it.
Thank you very much for your coding expertise !
vincentCegedim commented at 2013-11-27 16:11:¶
I don't know where that awk comes from; here is what I can found on a fresh installed debian box...
# ls -l /usr/bin/awk lrwxrwxrwx 1 root root 21 oct. 3 11:55 /usr/bin/awk -> /etc/alternatives/awk # ls -l /etc/alternatives/awk lrwxrwxrwx 1 root root 13 oct. 3 11:55 /etc/alternatives/awk -> /usr/bin/mawk
and on the gawk one
# ls -l /usr/bin/awk lrwxrwxrwx 1 root root 21 Nov 19 16:03 /usr/bin/awk -> /etc/alternatives/awk # ls -l /etc/alternatives/awk lrwxrwxrwx 1 root root 13 Nov 27 16:46 /etc/alternatives/awk -> /usr/bin/gawk
schlomo commented at 2013-11-27 21:30:¶
That is why we love Debian :-) Maybe we should be calling gawk instead
of
awk? And depend on the package? After all we also depend on bash and
don't
support dash...
On 27 November 2013 17:11, vincentCegedim notifications@github.com wrote:
I don't know where that awk comes from; here is what I can found on a
fresh installed debian box...ls -l /usr/bin/awk¶
lrwxrwxrwx 1 root root 21 oct. 3 11:55 /usr/bin/awk -> /etc/alternatives/awk
ls -l /etc/alternatives/awk¶
lrwxrwxrwx 1 root root 13 oct. 3 11:55 /etc/alternatives/awk -> /usr/bin/mawk
and on the gawk one
ls -l /usr/bin/awk¶
lrwxrwxrwx 1 root root 21 Nov 19 16:03 /usr/bin/awk -> /etc/alternatives/awk
ls -l /etc/alternatives/awk¶
lrwxrwxrwx 1 root root 13 Nov 27 16:46 /etc/alternatives/awk -> /usr/bin/gawk
—
Reply to this email directly or view it on GitHubhttps://github.com/rear/rear/issues/335#issuecomment-29396567
.
gdha commented at 2013-11-28 07:24:¶
submitted a case at https://code.google.com/p/original-mawk/issues/detail?id=23 (mawk maintainer for this issue)
answer from mawk member:
This is a known limitation: mawk's format for %d is limited by the format.
The limitation is done to improve performance.
You can get more precision using one of the floating formats (and can construct
one which prints like a %d, e.g., by putting a ".0" on the end of the format).
gdha commented at 2013-12-10 08:11:¶
@schlomo should we add the following the spec file?
### Dependencies on all distributions
Requires: gawk
schlomo commented at 2013-12-10 09:00:¶
IMHO yes as we need GNU awk and not any other
On 10 December 2013 09:11, gdha notifications@github.com wrote:
@schlomo https://github.com/schlomo should we add the following the
spec file?Dependencies on all distributions¶
Requires: gawk
—
Reply to this email directly or view it on GitHubhttps://github.com/rear/rear/issues/335#issuecomment-30206473
.
gdha commented at 2014-06-06 07:29:¶
Just for completeness https://code.google.com/p/original-mawk/issues/detail?id=23 status changed to "wont fix"
[Export of Github issue for rear/rear.]