OpenStack Ceilometer Charm

ceilometer-upgrade action fails with a Keystone error

Bug #1906624 reported by Giuseppe Petralia on 2020-12-03

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Ceilometer Charm	New	Undecided	Unassigned

Bug Description

Description
===============================
When running ceilometer-update on ceilometer/leader,
action fails with error:

"ERROR ceilometer The Keystone service is temporarily unavailable."

Versions
===============================
Cloud focal-victoria
Charm ceilometer rev: 278

Steps to reproduce
===============================
1. Deploy a focal-victoria bundle (i.e. openstack-on-lxd bundle https://pastebin.ubuntu.com/p/pW2NGJYG2x/ )

2. Ceilometer unit is in "blocked" state with message:
"Run the ceilometer-upgrade action on the leader to initialize ceilometer and gnocchi"

3. Run ceilomer-upgrade action with:
"juju run-action ceilometer/0 ceilometer-upgrade --wait"

4. Action fail with the error: https://pastebin.ubuntu.com/p/PjnzssXTXf/

$ openstack token issue
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| expires | 2020-12-03T10:51:40+0000 |
| id | gAAAAABfyLUsbBdlmnDpuP8kHib-7-GM_oDo2KXAvxwKrpvW00flsytT48lEd73dOJO34zC2AYqdMZcCuH_BoCia6xkpWsMiS35SOooi9UleeTRi0sIW5QZowyLreiHPmg_jpMiF50dWCLE32yNXFjLevqs7bzkb94vJat2cIsoFM3RVBnhujFU |
| project_id | 5c19cc143cb746b6a9e765adcd9f7595 |
| user_id | 4434a217512a4bb8aa569e8f68a48ae8 |
+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Expected result
===============================
ceilometer-upgrade action complete and the unit is "active" state.

Revision history for this message

Giuseppe Petralia (peppepetra) wrote on 2020-12-03:

I found that keystone was using https endpoint configured with the relatioon:

vault:certificates keystone:certificates

While gnocchi was using http endpoint. I have added the relation with vault with:

juju add-relation vault:certificates gnocchi:certificates

and now I am able to complete the ceilometer-upgraade action:

$ juju run-action ceilometer/0 ceilometer-upgrade --wait
unit-ceilometer-0:
  UnitId: ceilometer/0
  id: "32"
  results:
    Stdout: "lxc\n2020-12-03 10:44:14.825 139161 DEBUG ceilometer.cmd.storage [-]
      Upgrading Gnocchi resource types upgrade /usr/lib/python3/dist-packages/ceilometer/cmd/storage.py:42\e[00m\nlxc\nactive\n"
    outcome: success, ceilometer-upgrade completed.
  status: completed
  timing:
    completed: 2020-12-03 10:44:30 +0000 UTC
    enqueued: 2020-12-03 10:44:06 +0000 UTC
    started: 2020-12-03 10:44:06 +0000 UTC