Use OpenStack API instead of MySQL to collect the status of the workers/agents

Bug #1546188 reported by Simon Pasquier
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StackLight
Fix Released
High
guillaume thouvenin

Bug Description

Right now the LMA collector queries the MySQL service to compute the status of the Nova/Cinder/Neutron services. This has led to miscellaneous issues [1][2][3] that would be all fixed by using the existing APIs to get the same information.

[1] https://bugs.launchpad.net/lma-toolchain/+bug/1536072
[2] https://bugs.launchpad.net/lma-toolchain/+bug/1546017
[3] https://bugs.launchpad.net/lma-toolchain/+bug/1531541

Tags: collectd mos9
Changed in lma-toolchain:
importance: Wishlist → High
Revision history for this message
Simon Pasquier (simon-pasquier) wrote :

Raising the importance to Critical because using MySQL makes the LMA collector crash with MOS 9:

2016/03/01 08:41:12 Decoder 'collectd_httplisten-collectd_decoder' error: FATAL: process_message() /usr/share/lma_collector/decoders/collectd.lua:52: services.osapi_compute.disabled

Content of the services table (note the nova-osapi_compute entry):

mysql> select `binary`, `host` from services;
+--------------------+--------------------------+
| binary | host |
+--------------------+--------------------------+
| nova-metadata | 0.0.0.0 |
| nova-osapi_compute | 10.109.1.6 |
| nova-cert | node-5.test.domain.local |
| nova-conductor | node-5.test.domain.local |
| nova-consoleauth | node-5.test.domain.local |
| nova-scheduler | node-5.test.domain.local |
| nova-compute | node-6.test.domain.local |
+--------------------+--------------------------+

And the output of the nova service-list command is correct:
+----+------------------+--------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+----+------------------+--------------------------+----------+---------+-------+----------------------------+-----------------+
| 1 | nova-consoleauth | node-5.test.domain.local | internal | enabled | up | 2016-03-01T08:44:07.000000 | - |
| 2 | nova-cert | node-5.test.domain.local | internal | enabled | up | 2016-03-01T08:44:08.000000 | - |
| 3 | nova-scheduler | node-5.test.domain.local | internal | enabled | up | 2016-03-01T08:43:43.000000 | - |
| 4 | nova-conductor | node-5.test.domain.local | internal | enabled | up | 2016-03-01T08:43:44.000000 | - |
| 8 | nova-compute | node-6.test.domain.local | nova | enabled | up | 2016-03-01T08:43:58.000000 | - |
+----+------------------+--------------------------+----------+---------+-------+----------------------------+-----------------+

Changed in lma-toolchain:
assignee: LMA-Toolchain Fuel Plugins (mos-lma-toolchain) → guillaume thouvenin (guillaume-thouvenin)
tags: added: collectd
tags: added: mos9
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-plugin-lma-collector (master)

Fix proposed to branch: master
Review: https://review.openstack.org/291073

Changed in lma-toolchain:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/291177

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/291738

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-plugin-lma-collector (master)

Reviewed: https://review.openstack.org/291073
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-lma-collector/commit/?id=4e5f35e3a626c82448876a1356483665b1b4fe5b
Submitter: Jenkins
Branch: master

commit 4e5f35e3a626c82448876a1356483665b1b4fe5b
Author: Guillaume Thouvenin <email address hidden>
Date: Mon Mar 7 10:33:15 2016 +0100

    Use OpenStack API to collect the status of Nova workers

    This change uses the Nova API to get the status of the Nova workers
    instead of querying the MySQL database.

    Change-Id: I24e84b21f988e4c748d0ead134d60df4bf9dd8b1
    Partial-Bug: #1546188

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/291177
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-lma-collector/commit/?id=e60e187f32cb65f7297a611049648c9da377742a
Submitter: Jenkins
Branch: master

commit e60e187f32cb65f7297a611049648c9da377742a
Author: Guillaume Thouvenin <email address hidden>
Date: Thu Mar 10 14:22:37 2016 +0100

    Use OpenStack API to collect the status of Cinder workers

    This change uses the Cinder API to get the status of the Cinder workers
    instead of querying the MySQL database.

    Change-Id: If92596b3cee8a4c9f0dcf84454fdff2a2532160f
    Partial-Bug: #1546188

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/291738
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-lma-collector/commit/?id=bcdedbf1aa7f4904d09e08da28a00fdb0be6d4ea
Submitter: Jenkins
Branch: master

commit bcdedbf1aa7f4904d09e08da28a00fdb0be6d4ea
Author: Guillaume Thouvenin <email address hidden>
Date: Fri Mar 11 15:23:13 2016 +0100

    Use OpenStack API to collect the status of Neutron agents

    This change uses the Neutron API to get the status of the Neutron
    agents instead of querying the MySQL database.

    Change-Id: I60fa2386a887e9dac2fe4f1234d225ad6402bf2d
    Partial-Bug: #1546188

Changed in lma-toolchain:
status: In Progress → Fix Committed
Changed in lma-toolchain:
milestone: 1.0.0 → 0.10.0
Changed in lma-toolchain:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.