prometheus doesn't restart when adding a new relation

Bug #1933868 reported by Garrett Neugent
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Prometheus libvirt exporter charm
New
Undecided
Unassigned

Bug Description

When adding the prometheus-libvirt-exporter relation to a customer cloud, we found that the prometheus config file was correctly updated with the new endpoints for pulling data, but the snap was not restarted as part of config-changed:

$ service snap.prometheus.prometheus status
● snap.prometheus.prometheus.service - Service for snap application prometheus.prometheus
   Loaded: loaded (/etc/systemd/system/snap.prometheus.prometheus.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-09-28 19:02:18 UTC; 8 months 29 days ago

$ ls -al /var/snap/prometheus/32/prometheus.yml
-r--r----- 1 root root 8025 Jun 28 15:59 /var/snap/prometheus/32/prometheus.yml

Charm info:

  prometheus:
    charm: cs:prometheus2-19
    series: bionic
    os: ubuntu
    charm-origin: jujucharms
    charm-name: prometheus2
    charm-rev: 19
    charm-version: ae30894
    can-upgrade-to: cs:prometheus2-22
    exposed: false
    application-status:
      current: active
      message: Ready
      since: 28 Jun 2021 15:59:41Z
    relations:
      grafana-source:
      - grafana
      juju-info:
      - canonical-livepatch
      - filebeat
      - landscape-client
      - logrotate
      - ntp
      - telegraf-prometheus
      nrpe-external-master:
      - nrpe-host
      target:
      - prometheus-ceph-exporter
      - prometheus-libvirt-exporter
      - prometheus-openstack-exporter
      - telegraf
      - telegraf-maas-infra
      - telegraf-prometheus

$ juju config keystone openstack-origin
cloud:bionic-ussuri

As a workaround, restarting the prometheus snap worked:

$ sudo service snap.prometheus.prometheus restart

Revision history for this message
Garrett Neugent (thogarre) wrote :

Note, here were the commands used to deploy and relate the charm:

juju deploy prometheus-libvirt-exporter
juju relate prometheus-libvirt-exporter nova-compute-kvm
juju relate prometheus-libvirt-exporter:scrape prometheus:target
juju relate prometheus-libvirt-exporter:dashboards grafana:dashboards
juju relate prometheus-libvirt-exporter:nrpe-external-master nrpe-host:nrpe-external-master

Revision history for this message
Stephan Pampel (stephanpampel) wrote :

In general Prometheus does not need to restart to reload its' config: "Prometheus can reload its configuration at runtime." https://prometheus.io/docs/prometheus/latest/configuration/configuration/

There is already an bug for this (if you are using prometheus2 charm): https://bugs.launchpad.net/charm-prometheus2/+bug/1940565

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.