Multi-level keyboard layouts: Control keys produce glyphs from private use plane

Hello,

This is a downstream-issue I originally reported for cool-retro-term at https://github.com/Swordfish90/cool-retro-term/issues/11.

I am using an XKB layout based on the Neo2 layout (part of Xorg), where some of the control characters (delete, up/down, insert, etc.) are on an additional level on top of the alphabetic keys and are accessed using an extra modifier. This seems to confuse the Ubuntu terminal library.

E. g., when I enter return, this will not yield a newline, but instead produces a unicode glyph from the private use plane. Someone more knowledgeable in XKB stuff once informed me that such bugs were indicative of incorrect handling of XKB modifier locks, but I cannot ascertain whether this is the case here.

Pressing return once yields nothing. Pressing return a second time yields the following private-use-unicode character: 􊴓

The following is the xev output when pressing return on my layout.

KeyPress event, serial 33, synthetic NO, window 0x3400001,
    root 0x267, subw 0x0, time 4344370, (295,293), root:(296,294),
    state 0x0, keycode 108 (keysym 0xfe0c, ISO_First_Group), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

KeyPress event, serial 33, synthetic NO, window 0x3400001,
    root 0x267, subw 0x0, time 4344562, (295,293), root:(296,294),
    state 0x0, keycode 36 (keysym 0xff0d, Return), same_screen YES,
" XLookupString gives 1 bytes: (0d) "
" XmbLookupString gives 1 bytes: (0d) "
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x3400001,
    root 0x267, subw 0x0, time 4344825, (295,293), root:(296,294),
    state 0x0, keycode 36 (keysym 0xff0d, Return), same_screen YES,
" XLookupString gives 1 bytes: (0d) "
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x3400001,
    root 0x267, subw 0x0, time 4345226, (295,293), root:(296,294),
    state 0x82, keycode 108 (keysym 0xff7f, Num_Lock), same_screen YES,
    XKeysymToKeycode returns keycode: 94
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

A related problem occurs when I am using keys on the numeric keypad, which on my layout is accessed using a modifier key. These keypad keys only generate numeric input and never output control characters, but the terminal application treats them as if numlock were disabled and sees them as control chars. E. g., KP_4 will move the cursor left, while it should rather yield the character 4.

Pertinent xev output for the keypad event:

KeyPress event, serial 33, synthetic NO, window 0x3600001,
    root 0x267, subw 0x0, time 5196219, (658,583), root:(659,584),
    state 0x0, keycode 94 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
    XKeysymToKeycode returns keycode: 92
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

KeyPress event, serial 33, synthetic NO, window 0x3600001,
    root 0x267, subw 0x0, time 5197039, (658,583), root:(659,584),
    state 0x0, keycode 83 (keysym 0xffb4, KP_4), same_screen YES,
    XLookupString gives 1 bytes: (34) "4"
    XmbLookupString gives 1 bytes: (34) "4"
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x3600001,
    root 0x267, subw 0x0, time 5197255, (658,583), root:(659,584),
    state 0x0, keycode 83 (keysym 0xffb4, KP_4), same_screen YES,
    XLookupString gives 1 bytes: (34) "4"
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x3600001,
    root 0x267, subw 0x0, time 5197539, (658,583), root:(659,584),
    state 0x82, keycode 94 (keysym 0xff7f, Num_Lock), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

Delete, backspace, up/down seem to work, but require two keypresses each. E.g., when using the overlay arrow keys, I noticed that pressing Up requires two presses of the key, while it should only require a single press. Again, xev output for a single press of Up.

KeyPress event, serial 33, synthetic NO, window 0x3200001,
    root 0x2bb, subw 0x0, time 16800117, (476,705), root:(477,706),
    state 0x0, keycode 108 (keysym 0xfe0c, ISO_First_Group), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

KeyPress event, serial 33, synthetic NO, window 0x3200001,
    root 0x2bb, subw 0x0, time 16800589, (476,705), root:(477,706),
    state 0x0, keycode 111 (keysym 0xff52, Up), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x3200001,
    root 0x2bb, subw 0x0, time 16800661, (476,705), root:(477,706),
    state 0x0, keycode 111 (keysym 0xff52, Up), same_screen YES,
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x3200001,
    root 0x2bb, subw 0x0, time 16801141, (476,705), root:(477,706),
    state 0x82, keycode 108 (keysym 0xff7f, Num_Lock), same_screen YES,
    XKeysymToKeycode returns keycode: 94
    XLookupString gives 0 bytes:
    XFilterEvent returns: False

Best regards,
Z.