ceilometer can's collect the instance

Bug #1711487 reported by accjiyun
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ceilometer
Confirmed
Undecided
Unassigned

Bug Description

Thank you for viewing my post. I hope you can help me.

I deployed the ceilometer service on Ocata and use gnocchi as it is database backend. I need to monitor some indexs of OpenStack such as cpu, memory etc. And I had launch an instances, but the ceilometer-collector's log show it not exist.

# openstack server list
+--------------------------------------+-------------------+--------+-------------------------+------------+
| ID | Name | Status | Networks | Image Name |
+--------------------------------------+-------------------+--------+-------------------------+------------+
| ac0f56a0-7de1-4433-a682-cb50edd34ff8 | provider-instance | ACTIVE | provider=192.168.32.206 | cirros |
+--------------------------------------+-------------------+--------+-------------------------+------------+

When i list the gnocchi resource :

# gnocchi resource list

Its output only one resource and that type is `generic` and `image`:

# gnocchi resource list
+------------------+---------+------------------+---------+----------------------+------------------+----------+------------------+-----+
| id | type | project_id | user_id | original_resource_id | started_at | ended_at | revision_start | r |
+------------------+---------+------------------+---------+----------------------+------------------+----------+------------------+-----+
| 83746b69-87e8-4e | generic | None | None | 83746b69-87e8-4ee2 | 2017-08-17T11:39 | None | 2017-08-17T11:39 | N |
| e2-b9af- | | | | -b9af-39769cc32b75 | :46.861664+00:00 | | :46.861693+00:00 | |
| 39769cc32b75 | | | | | | | | |
| 078094a0-d02d- | image | c9e5e8bd3e0f4303 | None | 078094a0-d02d-4e55 | 2017-08-17T12:05 | None | 2017-08-17T12:05 | Ncd |
| 4e55-a8be- | | 93163890a81cbeca | | -a8be-4cd897b64a2b | :53.440542+00:00 | | :53.440708+00:00 | d0 |
| 4cd897b64a2b | | | | | | | | b8 |
| | | | | | | | | |
+------------------+---------+------------------+---------+----------------------+------------------+----------+------------------+-----+

There are some logs and configurations:

> gnocchi-api -p 8041 &>> /var/log/gnocchi/gnocchi-uwsgi.log

192.168.32.134 - - [18/Aug/2017 09:43:38] "PATCH /v1/resource/instance/ac0f56a0-7de1-4433-a682-cb50edd34ff8 HTTP/1.1" 404 114
192.168.32.134 - - [18/Aug/2017 09:43:38] "PATCH /v1/resource/instance/ac0f56a0-7de1-4433-a682-cb50edd34ff8 HTTP/1.1" 404 114
192.168.32.134 - - [18/Aug/2017 09:43:38] "POST /v1/batch/resources/metrics/measures?create_metrics=True HTTP/1.1" 202 4
192.168.32.134 - - [18/Aug/2017 09:43:38] "PATCH /v1/resource/instance/ac0f56a0-7de1-4433-a682-cb50edd34ff8 HTTP/1.1" 404 114
192.168.32.134 - - [18/Aug/2017 09:43:38] "PATCH /v1/resource/instance/ac0f56a0-7de1-4433-a682-cb50edd34ff8 HTTP/1.1" 404 114

> ceilometer/collector.log

    2017-08-18 09:43:38.459 5478 ERROR ceilometer.dispatcher.gnocchi [-] Resource ac0f56a0-7de1-4433-a682-cb50edd34ff8 does not exist (HTTP 404)
    2017-08-18 09:43:38.498 5478 ERROR ceilometer.dispatcher.gnocchi [-] Resource ac0f56a0-7de1-4433-a682-cb50edd34ff8 does not exist (HTTP 404)
    2017-08-18 09:43:38.502 5478 DEBUG ceilometer.dispatcher.gnocchi [-] 4 measures posted against 4 metrics through 1 resources batch_measures /usr/lib/python2.7/site-packages/ceilometer/dispatcher/gnocchi.py:433
    2017-08-18 09:43:38.569 5478 ERROR ceilometer.dispatcher.gnocchi [-] Resource ac0f56a0-7de1-4433-a682-cb50edd34ff8 does not exist (HTTP 404)
    2017-08-18 09:43:38.597 5478 ERROR ceilometer.dispatcher.gnocchi [-] Resource ac0f56a0-7de1-4433-a682-cb50edd34ff8 does not exist (HTTP 404)
    2017-08-18 09:43:39.343 5478 ERROR ceilometer.dispatcher.gnocchi [-] Resource ac0f56a0-7de1-4433-a682-cb50edd34ff8 does not exist (HTTP 404)

> ceilometer.conf:

    [DEFAULT]
    meter_dispatchers=gnocchi
    event_dispatchers=gnocchi
    transport_url = rabbit://openstack:pass123456@controller3

    [api]
    gnocchi_is_enabled = true

    [dispatcher_gnocchi]
    filter_service_activity = False
    archive_policy = low
    filter_project=service

    [service_credentials]
    auth_type = password
    auth_url = http://controller3:5000/v3
    project_domain_name = default
    user_domain_name = default
    project_name = service
    username = ceilometer
    password = pass123456
    interface = internalURL
    region_name = RegionOne

> gnocchi.conf

    [DEFAULT]
    debug = true
    verbose = true
    log_dir = /var/log/gnocchi

    [api]
    workers = 4
    auth_mode = keystone
    middlewares = oslo_middleware.cors.CORS

    [indexer]
    url = mysql+pymysql://gnocchi:pass123456@controller3/gnocchi?charset=utf8

    [keystone_authtoken]
    auth_type = password
    auth_url = http://controller3:5000/v3
    auth_uri = http://controller3:35357/v3/
    memcached_servers = controller3:11211
    project_domain_name = Default
    user_domain_name = Default
    project_name = service
    username = gnocchi
    password = pass123456
    interface = internalURL
    region_name = RegionOne

    [statsd]
    host = controller3
    resource_id = 83746b69-87e8-4ee2-b9af-39769cc32b75
    archive_policy_name = low

accjiyun (accjiyun)
description: updated
description: updated
Revision history for this message
gordon chung (chungg) wrote :

did you run gnocchi-upgrade?

did you run ceilometer-upgrade --skip-metering-database?

gordon chung (chungg)
Changed in ceilometer:
status: New → Incomplete
Revision history for this message
accjiyun (accjiyun) wrote :

@gordon chung (chungg)
Yes, many times:)

Revision history for this message
gordon chung (chungg) wrote :

can you create a new instance and see what error you see in collector and gnocchi-api logs? the errors you have in descriptions are errors post-creation so i'm not sure if the actual creation worked.

Revision history for this message
gordon chung (chungg) wrote :

closing since no feedback. please reopen if still an issue.

Changed in ceilometer:
status: Incomplete → Invalid
Revision history for this message
davidzhubo (davidzhubo) wrote :

I have got the same problem.

Revision history for this message
davidzhubo (davidzhubo) wrote :

ceilometer -> gnocchi-api,/v1/resource/instance/xxx 404 error .

Changed in ceilometer:
status: Invalid → Confirmed
Revision history for this message
davidzhubo (davidzhubo) wrote :

I have got the same problem.the following is the error log(gnocchi-api):
 [26/Mar/2018:10:55:58 +0800] "PATCH /v1/resource/instance/13f27322-618f-45d3-b567-c7de223d0cc8 HTTP/1.1" 404 114 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:02 +0800] "POST /v1/batch/resources/metrics/measures?create_metrics=True HTTP/1.1" 400 580 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:03 +0800] "POST /v1/resource/instance HTTP/1.1" 400 117 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:04 +0800] "POST /v1/batch/resources/metrics/measures?create_metrics=True HTTP/1.1" 202 4 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:04 +0800] "PATCH /v1/resource/instance/13f27322-618f-45d3-b567-c7de223d0cc8 HTTP/1.1" 404 114 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:06 +0800] "POST /v1/batch/resources/metrics/measures?create_metrics=True HTTP/1.1" 400 580 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:07 +0800] "POST /v1/resource/instance HTTP/1.1" 400 117 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:08 +0800] "POST /v1/batch/resources/metrics/measures?create_metrics=True HTTP/1.1" 202 4 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
[26/Mar/2018:10:56:08 +0800] "PATCH /v1/resource/instance/13f27322-618f-45d3-b567-c7de223d0cc8 HTTP/1.1" 404 114 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:15 +0800] "POST /v1/batch/resources/metrics/measures?create_metrics=True HTTP/1.1" 202 4 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"
 [26/Mar/2018:10:56:16 +0800] "POST /v1/batch/resources/metrics/measures?create_metrics=True HTTP/1.1" 202 4 "-" "ceilometer-collector keystoneauth1/2.18.0 python-requests/2.12.5 CPython/2.7.5"

Changed in ceilometer:
assignee: nobody → davidzhubo (davidzhubo)
assignee: davidzhubo (davidzhubo) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.