Usability: "localectl set-x11-keymap" doesn't work in mantic

Looks like the change was intentional: https://launchpad.net/debian/+source/systemd/253-4

Though the referenced LP bug doesn't exist, so I'm not sure why. I'll keep digging.

The localectl change in systemd breaks cloud-init as it uses "localectl set-x11-keymap".

The commit message says "Integration is not set up for now, but GDM needs to query the configured values. Allow reading, but disallow setting keymaps."

Is this integration expected to be setup before Mantic releases? If so, would that fix our behavior? If not, do we need to switch to writing /etc/vconsole.conf?

Also, it appears that bug referenced in the commit message (#1038762) does not exist. Is this a typo or a private bug?

> Also, it appears that bug referenced in the commit message (#1038762) does not exist. Is this a typo or a private bug?

This is a Debian bug number: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1038762. However, the broader change here is that Debian (and now in Ubuntu), we stopped carrying a patch that had support for Debian specific config files (/etc/default/keyboard, /etc/default/locale, etc.). Ultimately, we would like to be able to just use /etc/vconsole.conf, but things are not there yet, and this error message is a stop-gap to indicate that.

I will check in with Debian maintainers on this. In the mean time, I guess it would be best for cloud-init to use Debian-specific ways to update the keyboard layout, i.e. see https://manpages.ubuntu.com/manpages/lunar/en/man5/keyboard.5.html.

cloud-init PR up at https://github.com/canonical/cloud-init/pull/4361

@enr0n, while I think the approach James is taking in PR 4361 to write /etc/default/keyboard directly (followed by running: systemctl restart console-setup ) will persist the keyboard values provided, I find it hard to confirm that the keyboard settings are properly set and maybe this warrants a separate bug?.

  Are there suggestions for how in Ubuntu we can report status of configured keyboard layout as `localectl status` doesn't seem to report values from /etc/default/keyboard anymore either.

After such a change, localectl status still does not represent the updated settings on Ubuntu:

Note as well that any call to localectl status initially reports "Could not get properties: Access Denied" on first call while systemd-localed.service is being started up by udev/systemd.

root@schema-m:~# localectl status
Could not get properties: Access denied
root@schema-m:~# journalctl -b 0 -f
Aug 23 03:25:08 schema-m systemd[1]: systemd-localed.service: Deactivated successfully.
Aug 23 03:27:05 schema-m dbus-daemon[209]: [system] Activating via systemd: service name='org.freedesktop.locale1' unit='dbus-org.freedesktop.locale1.service' requested by ':1.39' (uid=0 pid=2003 comm="localectl status" label="unconfined")
Aug 23 03:27:05 schema-m systemd[1]: Starting systemd-localed.service - Locale Service...
Aug 23 03:27:05 schema-m dbus-daemon[209]: [system] Successfully activated service 'org.freedesktop.locale1'
Aug 23 03:27:05 schema-m systemd[1]: Started systemd-localed.service - Locale Service.
Aug 23 03:27:55 schema-m systemd[1]: systemd-localed.service: Deactivated successfully.
Aug 23 03:28:08 schema-m dbus-daemon[209]: [system] Activating via systemd: service name='org.freedesktop.locale1' unit='dbus-org.freedesktop.locale1.service' requested by ':1.43' (uid=0 pid=2011 comm="localectl status" label="unconfined")
Aug 23 03:28:08 schema-m systemd[1]: Starting systemd-localed.service - Locale Service...
Aug 23 03:28:08 schema-m dbus-daemon[209]: [system] Successfully activated service 'org.freedesktop.locale1'
Aug 23 03:28:08 schema-m systemd[1]: Started systemd-localed.service - Locale Service.
^^^^^^^^^^^^^^^^^ Note systemd
root@schema-m:~# localectl status
System Locale: LANG=C.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset) <------------- Note layout still unset despite /etc/default/keyboard "gb" value

`localectl status` will report the correct output as long as the symlink from `/etc/vconsole.conf` to `/etc/default/keyboard` is setup correctly; this is configured by /usr/lib/tmpfiles.d/debian.conf. What is the status of systemd-tmpfiles-setup.service on your system?

E.g., on a fresh mantic container I see:

root@mantic:~# systemctl status systemd-tmpfiles-setup.service --no-pager
● systemd-tmpfiles-setup.service - Create Volatile Files and Directories
     Loaded: loaded (/lib/systemd/system/systemd-tmpfiles-setup.service; static)
     Active: active (exited) since Wed 2023-08-23 13:53:54 UTC; 12min ago
       Docs: man:tmpfiles.d(5)
             man:systemd-tmpfiles(8)
   Main PID: 141 (code=exited, status=0/SUCCESS)
        CPU: 18ms

Aug 23 13:53:54 mantic systemd[1]: Starting systemd-tmpfiles-setup.service …s...
Aug 23 13:53:54 mantic systemd[1]: Finished systemd-tmpfiles-setup.service …ies.
Hint: Some lines were ellipsized, use -l to show in full.
root@mantic:~# localectl
System Locale: LANG=C.UTF-8
    VC Keymap: (unset)
   X11 Layout: us
    X11 Model: pc105
root@mantic:~# readlink -f /etc/vconsole.conf
/etc/default/keyboard

Nick, thanks for the comment. I had a conspiracy theory that there was a race condition here in early boot during cloud-config.service when cloud-init tried writing /etc/default/keyboard directly to write out before invoking systemctl restart console-setup. The conditions I was seeing was the error message:
    Could not get properties: Access Denied

This generic message which lead me to believe something was interacting incorrectly with dbus during systemd-localed activation.

Turns, out this was a programming issue in the cloud-init PR we put up to address this. When cloud-init is trying to write /etc/default/keyboard, the content is correct, but the file permissions were off.

The utility function we were using was trying to set perms on /etc/default/keyboard to 644 providing a decimal value of 644 instead of an octal value of mode=0o644

```
        util.write_file(
            filename="/etc/default/keyboard",
            content=contents,
            mode=644,
            omode="w",
        )
```

This resulted in setting permissions 1204 on /etc/default/keyboard .... which is equivalent of:

 chmod u=w,o=,g=r,+t /etc/default/keyboard.

leaving us with a file with no read permissions for root and a sticky bit set:

root@testlocale-manticdaily:~# ls -l /etc/default/keyboard
-rw-r----T 1 root root 120 Aug 25 02:31 /etc/default/keyboard

This results in a strange state and error message where first call to "localectl status" will first report as systemd-localed.service is activated, and the 2nd call to "localectl status" uses the cached values of the currently active systemd-localed.service so it doesn't report the permissions error and just dumps empty unset values.

root@testlocale-manticdaily:~# localectl status
Could not get properties: Access denied
root@testlocale-manticdaily:~# localectl status
System Locale: LANG=C.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

I think there are two issues here with localectl in Mantic, neither of which really seem critical from a functional standpoint, but from a usability standpoint they may warrant bugs of effort to improve understanding best practices and/or discovering errors in keymap setting:

  1. localectl set-x11-keymap <keymap_layout> <model> could provide a pointer to manpage, breadcrumb or suggested best practice for writing /etc/default/keyboard directly instead of an unhelpful message:
   Setting X11 and console keymaps is not supported in Debian.

  2. localectl status could be more informative about permissions errors or formatting errors in /etc/default/keyboard and could probably announce what type of permissions errors it is hitting:
   Could not get properties: Access denied to file /etc/default/keyboard

Again the 2nd case is really user/program-failure to do the right thing so not as likely a candidate to address.

I've pushed a commit to fix cloud-init's PR 4361 to ensure we write /etc/default/keyboard correctly instead of calling localectl set-x11-keymap directly.

I have also published preliminary 23.3 cloud-init deb packages for both Jammy and Mantic to https://launchpad.net/~chad.smith/+archive/ubuntu/cloud-init-uploads that can be used to validate that writing /etc/default/keyboard directly works as advertised

 This proposed changeset is small enough that cloud-init user-data can actually modify itself on disk before cc_keyboard runs using write_files directives.

@DaveJones,

I've attached test_write_etc_default_keyboard.yaml which is cloud-config user-data that will leverage the proposal in PR #4361.

Would you be able to launch daily mantic arm64 images on raspi with the following user-data test_write_etc_default_keyboard.yaml to confirm the proposed cloud-init fixes from PR 4361 properly set the keymap/layout for you?

Once we validate that behavior on Focal, Jammy and Lunar doesn't degrade, we will include this in 23.3 release (tomorrow/Monday) and SRU it into Focal/Jammy and Lunar.

@chad.smith that's ... horrifically and *wonderfully* hacky! And yes, I can confirm it works :) My keyboard was indeed GB afterwards, and localectl reported things properly.

Yes, Dave and thank you!

We'll sort with Foundations review whether there are usability improvements warranted for how localectl now behaves, this may be something that is taken upstream in debian to improve this "magic" behavior of symlinking /etc/vconsole.conf -> /etc/default/keyboard and things "just working".

Cloud-init will publish upstream release 23.3 which will contain a 'fix' that will avoid calling the localectl set-x11-keymap utility in favor of writing /etc/default/keyboard given that the sub-command is no longer functional in debian.

This bug is fix released in cloud-init 23.3-0ubuntu1 in Ubuntu mantic.