Comment 23 for bug 1923607

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/795150
Committed: https://opendev.org/openstack/tripleo-ansible/commit/33637b4ddf6b0561e740b9bf93f391b52f468605
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 33637b4ddf6b0561e740b9bf93f391b52f468605
Author: Michele Baldessari <email address hidden>
Date: Thu Jun 3 11:07:30 2021 +0200

    Add podman's events_logger option by default set to journald

    By default podman 3.0.x sets the [engine]/events_logger to "file".
    This causes every exec in podman to create a line of text in
    /run/libpod/events/events.log like the following:

      {"ID":"412b6770c0b418e6d49a4801e71a198ddb81bbbefdaf1c9aad4d7948f77910ee","Image":"quay.io/centos/centos:latest","Name":"leak-test-7","Status":"exec","Time":"2021-06-03T08:36:05.237964012Z","Type":"container","Attributes":{"org.label-schema.build-date":"20201204","org.label-schema.license":"GPLv2","org.label-schema.name":"CentOS Base Image","org.label-schema.schema-version":"1.0","org.label-schema.vendor":"CentOS"}}

    Since by default /run is mounted on tmpfs, this has the side-effect of
    increasing kernel slab objects over time indefinitely eventually causing
    an OOM of the box.

    We initially wanted to switch to the 'none' backend, but the podman
    folks recommended using the journald backend because events logs are
    used by podman in case of a rare race when running "podman run --rm".
    Given that we call run with --rm from in a multithreaded fashion this
    seems to be the safest approach. The drawback of using journald is
    that events won't be logged for rootless containers unless the user
    is part of the 'wheel' group. We believe we're not using those
    containers in tripleo anyways, so this should be safe.

    Tested by applying a backport of this patch to Train + podman 3.0.x and
    got the following:
    [root@controller-0 containers]# ls -la /run/libpod/events/
    total 0
    drwx------. 2 root root 40 Jun 3 11:55 .
    drwxr-x--x. 5 root root 140 Jun 3 11:55 ..

    [root@controller-0 containers]# more /etc/containers/containers.conf
    [containers]
    pids_limit = 4096
    [engine]
    events_logger = "journald"

    Also tested the override via the corresponding THT change in
    Ieffe2852111c3ec8347343a042dd78bbf691d79a.

    Closes-Bug: #1923607

    Change-Id: I780103e17f1bb42a0546c30bd6c001c642ad88b3
    (cherry picked from commit f31bab878bfd3332c20a10bf9ca26d443028d214)
    (cherry picked from commit 79be78bba35199c5b26632e51d8bda411a8239c5)
    (cherry picked from commit 637db1c401c6c6a0d2e3cef26ab8a97cc3b31bf2)