mariadbcheck@ service is called with uniq parameter every time causing high cardinality

Hi,

Eventually, it is required to have templated name for the service, that is followed by the socket with Accept=yes
https://www.freedesktop.org/software/systemd/man/systemd.socket.html#Accept=

In order to overcome this issue, we'd need to disable `Accept`, which requires to have a proper daemon behind it, rather then just a script, that was moved from xinetd.

At the moment we don't have time to refactor clustercheck [1] as we wanted to work on replacing HAproxy with ProxySQL for MariaDB balancing.
So contributions to that in the meanwhile are welcome.

[1] https://opendev.org/openstack/openstack-ansible-galera_server/src/branch/master/templates/clustercheck.j2

A solution here might be to construct a regex to exclude the mariadbcheck processes from node-exporter using the --collector.systemd.unit-exclude flag, see the documentation here https://github.com/prometheus/node_exporter/blob/master/README.md#include--exclude-flags.

Yes, that's one of the workarounds. It can be node-exporter skip setting, it can be label drop or rewrite in prometheus (and Loki, which suffer from the same cardinality problem), but none of them is a proper solution.

I'm not sure there is an alternative - xinetd is deprecated and systemd socket activated services are the modern replacement.

You have suggested we "Proposition: remove timestamp from parametrization for calling mariadbcheck@ service." but I cannot see a way to do that with systemd.socket.

This will happen in more places, I see Ironic starting to use socket activated services for tftp which will behave in exactly the same way.

So, cardinality problem will become more and more of the issue. Even local journalctl is struggling if server is running for long time (which it should).

May be we can escalate it to systemd and ask them not to create a new name of each socket activation...

https://github.com/systemd/systemd/issues/29103

Well, as I said, the only alternative is to re-write check from xinetd format to be a proper daemon, like some dummy flask or aiohttp application.

As what systemd wrintes in their docs for Accept property [1]: " For performance reasons, it is recommended to write new daemons only in a way that is suitable for Accept=no."

So they pretty much aware that such usecases are far from being perfect.

[1] https://www.freedesktop.org/software/systemd/man/systemd.socket.html#Accept=