#1781 PR merged: Also include other keyboard mappings to support non-US keyboards

Labels: enhancement, bug, cleanup, fixed / solved / done

jsmeix opened issue at 2018-04-24 14:29:

Also include other keyboard mappings to also support users with a non-US keyboard
because without the right keyboard mapping it could become an awful annoyance
to work in the recovery system.

  • Type: Bug Fix and Enhancement
    Actually this provides a workaround when in
    etc/scripts/system-setup.d/10-console-setup.sh
    the loadkeys /etc/dumpkeys.out fails.

  • Impact: Low and High
    For US-keyboard users the impact is at most low,
    for non-US-keyboard users the impact could be even critical.

  • How was this pull request tested?
    It works for me on a SUSE system.
    I don't know where on other systems the other keyboard mapping files are stored.
    But it is now at least a starting point that can be easily enhanced for other systems.

  • Brief description of the changes in this pull request:

By default the current keyboard mapping is dumped to etc/dumpkeys.out
so that it can be set during recovery system startup by
etc/scripts/system-setup.d/10-console-setup.sh
via loadkeys /etc/dumpkeys.out.

But depending on the keyboard mapping
the loadkeys /etc/dumpkeys.out command
may not work in the recovery system
so that some other keyboard mapping files
are also included so that the user can manually set
his keyboard mapping (e.g. via a command like loadkeys de-latin1).

This increases the recovery system size by about 1 MB
(an usual recovery system size is about 500 MB uncompressed)
but without the right keyboard mapping it could be an awful annoyance
to work in the recovery system.

jsmeix commented at 2018-04-24 14:32:

@gdha @gozora @schabrolles
because my knowledge about keyboard setup is basically zero
(I did this pull request based on a lot of trial and error)
I would very much appreciate it if you could have a look.
Perhaps I did something stupid and the real solution is "just simple".

jsmeix commented at 2018-04-25 09:17:

In my case the keyboad mapping on the original system
fails to load in the recovery system via loadkeys /etc/dumpkeys.out
with this error message:

adding map 3 violates explicit keymaps line

and I get a keyboard with a default (presumably US) layout
but actually I use a German qwertz keyboard.

On the original system I have a /etc/vconsole.conf file that contains

KEYMAP=de-latin1-nodeadkeys
FONT=eurlatgr.psfu
FONT_MAP=none
FONT_UNIMAP=

The unexpected thing is that in the recovery system
I can run loadkeys de-latin1-nodeadkeys
which "just works" so that one could assume
the dumped keyboad mapping on the original system
is not de-latin1-nodeadkeys but something different
but on the original system

# dumpkeys -f >/tmp/dumpkeys.out

# loadkeys de-latin1-nodeadkeys

# dumpkeys -f >/tmp/dumpkeys.de-latin1-nodeadkeys

# diff -s /tmp/dumpkeys.out /tmp/dumpkeys.de-latin1-nodeadkeys
Files /tmp/dumpkeys.out and /tmp/dumpkeys.de-latin1-nodeadkeys are identical

# loadkeys /tmp/dumpkeys.out

# loadkeys /tmp/dumpkeys.de-latin1-nodeadkeys

so something subtle makes in the recovery system
loadkeys /etc/dumpkeys.out fail...

gdha commented at 2018-04-25 09:51:

@jsmeix Perhaps it would be better we load all the keyboard keys like the plain Linux OS? Is lot of overhead but this way we have all keyboards. Just a though...

jsmeix commented at 2018-04-25 09:51:

It is the -f1 in dumpkeys -f1 >$ROOTFS_DIR/etc/dumpkeys.out
in rescue/GNU/Linux/500_clone_keyboard_mappings.sh
that makes things fail:

# dumpkeys -f1 >/tmp/dumpkeys.out2

# loadkeys /tmp/dumpkeys.out2
adding map 3 violates explicit keymaps line

On none of my systems down to SLE10 a -f1
is mentioned in man dumpkeys.
It is always only plain -f and that works for me.

jsmeix commented at 2018-04-25 09:54:

git log -p --follow usr/share/rear/rescue/GNU/Linux/500_clone_keyboard_mappings.sh
shows that the non-working -f1 was there since the beginning of git history
so that I have no idea if there was a reason or if it is just a typo.

jsmeix commented at 2018-04-25 10:00:

@gdha
including the whole keymaps default directory /usr/share/kbd/keymaps/
increases the recovery system size by about 3 MB in my case

# du -hs /usr/share/kbd/keymaps
2.8M    /usr/share/kbd/keymaps

and I know that on POWER architecture the recovery system must be kept very small
(bigger recovery systems on POWER architecture can cause totally weird failures).

But I agree that it can be crucial that the recovery system supports any keyboad
in particular for system migration when the keyboard of the new hardware
may not be known in advance when "rear mkrescue" is run.

So - as always - the right way out is to introduce config variables
where the user can specify what he actually needs.
I will implement that...

gozora commented at 2018-04-25 10:40:

@jsmeix honestly, my knowledge in this area is equal to 0 :-(

V.

jsmeix commented at 2018-04-25 10:49:

@gozora
no worries!
My knowledge in this area was equal to 0 but meanwhile it became a little bit more ;-)

jsmeix commented at 2018-04-25 12:08:

An addedum to
https://github.com/rear/rear/pull/1781#issuecomment-384230363

Interestingly on SLE10 things work with dumpkeys -f1

# dumpkeys -f1 >/tmp/dumpkeys.out

# loadkeys /tmp/dumpkeys.out && echo OK
Loading /tmp/dumpkeys.out
OK

but also with plain dumpkeys -f

# dumpkeys -f >/tmp/dumpkeys.out

# loadkeys /tmp/dumpkeys.out && echo OK
Loading /tmp/dumpkeys.out
OK

and there is no -f1 described in man dumpkeys or via dumpkeys -h on SLE10:

# man dumpkeys
...
SYNOPSIS
       dumpkeys [ -hilfn -ccharset --help --short-info --long-info --numeric --full-table --funcs-only --keys-only --compose-only --charset=charset ]
...
       -f --full-table
              This makes dumpkeys skip all the short-hand heuristics (see keymaps(5)) and output the key bindings in the canonical form. First a keymaps  line  describing
              the  currently defined modifier combinations is printed. Then for each key a row with a column for each modifier combination is printed. For example, if the
              current keymap in use uses seven modifiers, every row will have seven action code columns. This format can be useful for example to programs that  post-pro‐
              cess the output of dumpkeys.

# dumpkeys -h
dumpkeys version 1.12
...
       -f --full-table     don't use short-hand notations, one row per keycode

so that I will use plain dumpkeys -f in /rescue/GNU/Linux/500_clone_keyboard_mappings.sh

jsmeix commented at 2018-04-25 12:49:

Since
https://github.com/rear/rear/pull/1781/commits/e626787fa5e5a5293fdb2e9254953c5ddfb0f15f
including and setting keyboard mappings in the recovery system
works well for me (and sufficiently fail-safe) for the first time.

Now there are the new (but intentionally not yet documented) config variables

KEYMAPS_DEFAULT_DIRECTORY which specifies the default directory for keymaps

KEYMAPS_DIRECTORIES which is a string of keymap directories that should
get included in the recovery system

KEYMAP which is a keymap file for loadkeys in the recovery system

e.g. what I used in local.conf

KEYMAPS_DEFAULT_DIRECTORY=""
KEYMAPS_DIRECTORIES="/usr/share/kbd/keymaps/legacy"
KEYMAP="de-latin1-nodeadkeys"

jsmeix commented at 2018-04-25 13:24:

@gdha @schabrolles
if you do not object against the new KEYMAP* config variables
I would document them tomorrow in default.conf
and then I would like to merge this pull request.

I think with the new KEYMAP* config variables there cannot be
a severe regression because if somehow the default behaviour
does no longer work the user can now manually specify what
is needed to make it work in his particular environment.

jsmeix commented at 2018-04-25 14:15:

With
https://github.com/rear/rear/pull/1781/commits/4719c5ad14015bbe69018115137feb092e3dba98
I describe KEYMAPS_DEFAULT_DIRECTORY, KEYMAPS_DIRECTORIES, and KEYMAP
in default.conf so that you can have a look how I meant them to be used.

jsmeix commented at 2018-04-25 15:08:

@gdha @schabrolles @gozora
could you do me a favour and report here
what the default directory for keymaps is
on whatever non-SUSE systems you have.

On my SUSE systems I have in man loadkeys

FILES
       /usr/share/kbd/keymaps
              default directory for keymaps

which is actually true because I get things like

# rpm -qa | grep -i kbd
kbd-...
kbd-legacy-...

# rpm -ql kbd kbd-legacy | grep de-latin1
/usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1-nodeadkeys.map.gz
/usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1.map.gz
/usr/share/kbd/keymaps/legacy/mac/all/mac-de-latin1-nodeadkeys.map.gz
/usr/share/kbd/keymaps/legacy/mac/all/mac-de-latin1.map.gz
/usr/share/kbd/keymaps/legacy/sun/sunt5-de-latin1.map.gz

I would like to know how those things look on non-SUSE systems.

gozora commented at 2018-04-25 15:35:

Hello @jsmeix,

  • Debian 9.2 (stretch)
FILES
       /usr/share/keymaps
              default directory for keymaps

       /usr/src/linux/drivers/char/defkeymap.map
              default kernel keymap
  • Centos 7.3.1611 (Core)
FILES
       /lib/kbd/keymaps
              default directory for keymaps

       /usr/src/linux/drivers/char/defkeymap.map
              default kernel keymap
  • Fedora release 26 (Twenty Six)
FILES
       /lib/kbd/keymaps
              default directory for keymaps

       /usr/src/linux/drivers/char/defkeymap.map
              default kernel keymap
  • Arch Linux
FILES
       /usr/share/kbd/keymaps
              default directory for keymaps

       /usr/src/linux/drivers/char/defkeymap.map
              default kernel keymap
  • Ubuntu 12.04.5 LTS (Precise)
FILES
       /usr/share/keymaps
              default directory for keymaps

       /usr/src/linux/drivers/char/defkeymap.map
              default kernel keymap

Hope it helps.

V.

schabrolles commented at 2018-04-25 15:38:

@gozora ... too quick for me ;-)

gozora commented at 2018-04-25 15:44:

Btw, to check whether defaults in man page are really used by loadkeys binary, strace -e open loadkeys any_non_existing_file can be used ...

V.

jsmeix commented at 2018-04-26 06:59:

@gozora
many thanks for your https://github.com/rear/rear/pull/1781#issuecomment-384331316
it helps a lot!
I will enhance KEYMAPS_DEFAULT_DIRECTORY autodetection accordingly.

jsmeix commented at 2018-04-26 07:02:

@schabrolles @gdha
perhaps you can provide what the default directory for keymaps is
on whatever RHEL systems you have (ideally also on older ones).

jsmeix commented at 2018-04-26 07:35:

Regarding
https://github.com/rear/rear/pull/1781#issuecomment-384332401

This time @gozora won even with a lot of text!
Cf.
https://github.com/rear/rear/issues/1775#issuecomment-381120658
;-)
He is getting better with impressive speed!
I get a bit anxious what may happen next.
Perhaps an elaborated answer before the question?

schabrolles commented at 2018-04-26 08:40:

RHEL 6/7 are like CentOS/Fedora ones ... but I don't have anything into /usr/src/linux/ ...

FILES
       /lib/kbd/keymaps
              default directory for keymaps

jsmeix commented at 2018-04-26 09:06:

@schabrolles
many thanks for your prompt reply!

jsmeix commented at 2018-04-26 09:24:

FYI related to https://github.com/rear/rear/pull/1781#issuecomment-384334494

I found out how one can determine what keymap files plus include files
one particular keyboad mapping needs so that it can be loaded.

In my case for loadkeys de-latin1-nodeadkeys it is "just as simple as" ;-)

# for f in $( strace -f -e open loadkeys de-latin1-nodeadkeys 2>&1 | grep '/usr/share/kbd/keymaps/legacy/' | grep -v 'O_DIRECTORY' | grep -o 'open.*,' | cut -d '"' -f2 ) ; do readlink -e $f ; done

/usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1-nodeadkeys.map.gz
/usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1.map.gz
/usr/share/kbd/keymaps/legacy/i386/include/qwertz-layout.inc
/usr/share/kbd/keymaps/legacy/i386/include/compose.inc
/usr/share/kbd/keymaps/legacy/i386/include/linux-with-alt-and-altgr.inc
/usr/share/kbd/keymaps/legacy/i386/include/linux-keys-bare.inc
/usr/share/kbd/keymaps/legacy/i386/include/euro2.map.gz
/usr/share/kbd/keymaps/legacy/include/compose.latin1

Simplifying that command is left as an exercise to the reader ;-)

Accordingly with

KEYMAPS_DEFAULT_DIRECTORY="/usr/share/kbd/keymaps"
KEYMAPS_DIRECTORIES="/usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1-nodeadkeys.map.gz /usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1.map.gz /usr/share/kbd/keymaps/legacy/i386/include/qwertz-layout.inc /usr/share/kbd/keymaps/legacy/i386/include/compose.inc /usr/share/kbd/keymaps/legacy/i386/include/linux-with-alt-and-altgr.inc /usr/share/kbd/keymaps/legacy/i386/include/linux-keys-bare.inc /usr/share/kbd/keymaps/legacy/i386/include/euro2.map.gz /usr/share/kbd/keymaps/legacy/include/compose.latin1"
KEYMAP="de-latin1-nodeadkeys"

(one can as well specify plain files as KEYMAPS_DIRECTORIES values)
I get a minimal recovery system that contains only what is needed

# find /tmp/rear.owdPbH9RbpT2GqN/rootfs/ | grep keymaps
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/include
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/include/compose.latin1
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/include
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/include/qwertz-layout.inc
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/include/linux-keys-bare.inc
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/include/linux-with-alt-and-altgr.inc
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/include/compose.inc
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/include/euro2.map.gz
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/qwertz
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1-nodeadkeys.map.gz
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/qwertz/de-latin1.map.gz
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/qwerty
/tmp/rear.owdPbH9RbpT2GqN/rootfs/usr/share/kbd/keymaps/legacy/i386/qwerty/defkeymap.map.gz

(defkeymap.map.gz gets added in any case by 500_clone_keyboard_mappings.sh)
to run loadkeys de-latin1-nodeadkeys successfully in the recovery system
by etc/scripts/system-setup.d/10-console-setup.sh

jsmeix commented at 2018-04-26 10:44:

My "latest greatest" current code here works so well for me
so that I would like to merge it today soon
unless there are objections.

[Export of Github issue for rear/rear.]