#1469 PR closed: Use en_US.UTF-8 instead of creating custom rear.UTF-8¶
Labels: won't fix / can't fix / obsolete
andreamtp opened issue at 2017-09-03 09:12:¶
Since we're using en_US.UTF-8 at the end of the day, better check
existence of en_US.UTF-8 rather then creating a single purpose locales,
with the risk also of having failing backups if the locales cannot be
created. Since trending is to make glibc-locale-source optional,
better avoid the risk of failure in localedef and to create a barely
used locales.
gozora commented at 2017-09-03 12:43:¶
@andreamtp
First of all, I must ask, did you FULLY tested this PR (backup +
restore) ?
This might been changed lately, but as far as I remember during writing
of this patch, ReaR recovery system did not included UTF-8 locales but
rather worked with LC_ALL=C which caused Borg recovery to fail ...
V.
jsmeix commented at 2017-09-04 12:08:¶
In general see "Character encoding" at
https://github.com/rear/rear/wiki/Coding-Style
jsmeix commented at 2017-09-04 12:14:¶
See also https://github.com/rear/rear/issues/1035
gozora commented at 2017-09-04 15:42:¶
@jsmeix I was looking for issue mentioned in your https://github.com/rear/rear/pull/1469#issuecomment-326948052 for 30 minutes and could not find it :-)
Thanks for that!
V.
andreamtp commented at 2017-09-05 15:25:¶
@gozora this PR do not change the behaviour of ReaR with Borg
integration.
It just avoid the creation of a custom locale for rear, named
rear.UTF-8
and instead check the existence of
en_US.UTF-8
, that is the source for the old rear.UTF-8 , to use it during
restore.
The essence of the integration is preserved, it just avoid the creation of rear.UTF-8 that can fail on modern distros since the source files of locales are not always shipped by default and the error message that the user get is not very clear.
I've tested the backup part without any problem, and do not expect any trouble with the restore as well, since the LC_ALL used is the same, despite the name change.
gozora commented at 2017-09-05 15:38:¶
Hello @andreamtp,
I'm afraid it does break the integration.
The problem is that ReaR recovery system does not have UTF-8 support by
default (see
https://github.com/rear/rear/issues/1035)
and in order to restore from Borg backup you need to have UTF-8 support,
so that localedef is used just for this purpose.
A way how to solve this problem according your proposal, would be to
check if UTF-8 locales exists on original system and if they does, find
their location and copy them to ReaR recovery system.
I was thinking of this approach when I was writing Borg backup
implementation but using localedef looked to me more straight forward
compared to searching for locales in every available Linux distro you
can imagine ...
V.
gozora commented at 2017-11-06 16:26:¶
As there was no reply since some time and I personally think this PR would break Borg integration, I'm closing it.
GaLaKtIkUs commented at 2017-12-23 22:09:¶
Hi,
Since I'm expecting an error related to this issue, I would like to ask
if you plan to solve it.
To be more explicit, I'm getting the following message when trying to
backup:
00:28:51.387745142 Including prep/BORG/default/200_prep_borg.sh
character map file `UTF-8' not found: No such file or directory
default character map file `ANSI_X3.4-1968' not found: No such file or directory
2017-12-24 00:28:51.393723113 ERROR: Could not create locales
gozora commented at 2017-12-24 09:33:¶
Hi, what is the distro you are using?
V.
GaLaKtIkUs commented at 2017-12-24 10:12:¶
Hi,
Fedora 25, 26 and 27. Tried it on 25 and 27.
Using also CentOS 7. Will try it later if you ask.
Mehdi
gozora commented at 2017-12-24 10:19:¶
hmm, i'd say that testing on Centos was done when Borg support was written. Maybe I just missed something. Anyhow I'll run couple of tests on Fedora and Centos next days (but not sooner then 26th of Dec ;-) ), and let you konw ...
V.
GaLaKtIkUs commented at 2017-12-24 14:22:¶
Ok,
Thanks in advance :)
Happy holiday season
gozora commented at 2017-12-27 08:47:¶
Hello @GaLaKtIkUs
I just tested ReaR with Borg backend on my Fedora 26, and all seems to work correctly.
Can you please send me output of following commands:
# locale -a
# locale
# rpm -qa | grep glibc
# localedef -f UTF-8 -i en_US /tmp/mylocale && ls -al /tmp/mylocale
Thanks
V.
GaLaKtIkUs commented at 2017-12-27 08:58:¶
Hi,
I was about to write about how I solved the problem for Fedora 27.
First of all, I used the patch from this pull request.
I also used a standalone binary of Borg (generated using pyinstaller)
and included in the rescue disk using COPY_AS_IS_BORG.
I'll continue extensive testing on other versions of fedora and CentOS.
Thank you )
gozora commented at 2017-12-27 09:10:¶
Hello @GaLaKtIkUs,
Be careful, I actually never tested this patch, but I suspect that if you use it, you will end up with ReaR rescue system that will NOT be able to restore from Borg repository! To cut things short you will be able to backup but restore will fail.
ReaR by default does not have UTF-8 locales included.
V.
GaLaKtIkUs commented at 2017-12-27 09:10:¶
The localedef -f UTF-8 -i en_US /tmp/mylocale && ls -al /tmp/mylocale
gave errors as described in earlier posts. That's why I used the patch.
PS.
The results of the commands are attached
test.txt
GaLaKtIkUs commented at 2017-12-27 09:13:¶
I successfully restored a system (a virtual machine). I made a backup,
deleted the HDD and created a new empty one, than restored the whole VM
))
I boots successfully.
I will now a more complex case and report the results back here.
gozora commented at 2017-12-27 09:20:¶
Good, looking forward for that ;-).
I maybe know why you are having this problem. Could you try to install
glibc-locale-source ?
That should solve this problem without further hacks needed.
V.
gozora commented at 2017-12-27 09:25:¶
Without glibc-locale-source I have same message as you have:
fedora:(/root)(root)# rpm -q glibc-locale-source
package glibc-locale-source is not installed
fedora:(/root)(root)# localedef -f UTF-8 -i en_US /tmp/mylocale
character map file `UTF-8' not found: No such file or directory
default character map file `ANSI_X3.4-1968' not found: No such file or directory
with installed glibc-locale-source
fedora:(/root)(root)# rpm -q glibc-locale-source
glibc-locale-source-2.25-12.fc26.x86_64
fedora:(/root)(root)# localedef -f UTF-8 -i en_US /tmp/mylocale
fedora:(/root)(root)# ls -al /tmp/mylocale/
total 1576
drwxr-xr-x 3 root root 227 Dec 27 10:25 .
drwxrwxrwt. 11 root root 276 Dec 27 10:25 ..
-rw-r--r-- 1 root root 167 Dec 27 10:25 LC_ADDRESS
-rw-r--r-- 1 root root 1244054 Dec 27 10:25 LC_COLLATE
-rw-r--r-- 1 root root 328180 Dec 27 10:25 LC_CTYPE
-rw-r--r-- 1 root root 368 Dec 27 10:25 LC_IDENTIFICATION
-rw-r--r-- 1 root root 23 Dec 27 10:25 LC_MEASUREMENT
drwxr-xr-x 2 root root 29 Dec 27 10:25 LC_MESSAGES
-rw-r--r-- 1 root root 286 Dec 27 10:25 LC_MONETARY
-rw-r--r-- 1 root root 77 Dec 27 10:25 LC_NAME
-rw-r--r-- 1 root root 54 Dec 27 10:25 LC_NUMERIC
-rw-r--r-- 1 root root 34 Dec 27 10:25 LC_PAPER
-rw-r--r-- 1 root root 59 Dec 27 10:25 LC_TELEPHONE
-rw-r--r-- 1 root root 2454 Dec 27 10:25 LC_TIME
V.
GaLaKtIkUs commented at 2017-12-27 09:27:¶
Ok. I'm already testing your solution. Backup created. Will start restoring ...
GaLaKtIkUs commented at 2017-12-27 10:54:¶
Hi,
It seems it works. Thank you very much.
I will continue testing more complex cases. I'll report if I find any
issues.
gozora commented at 2017-12-27 11:11:¶
You are welcome!
If you find any, please open separate issue for that, as your problem was not related to this pull request.
V.
[Export of Github issue for rear/rear.]