Ceilometer

ceilometer-upgrade error

Bug #1745806 reported by Yang Shi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceilometer
In Progress
Undecided
gordon chung

Bug Description

I'm trying to install and configure ceilometer on the controller node according to the install guide.
When I run the command:
# ceilometer-upgrade --skip-metering-database
2017-11-29 15:59:06.536 4097 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
2017-11-29 15:59:06.536 4097 INFO ceilometer.cmd.storage [-] Skipping metering database upgrade
2017-11-29 15:59:07.165 4097 CRITICAL ceilometer [-] ConnectFailure: Unable to establish connection to http://controller:8041/v1/resource_type/ceph_account: HTTPConnectionPool(host='controller', port=8041): Max retries exceeded with url: /v1/resource_type/ceph_account (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f7bb28eec10>: Failed to establish a new connection: [Errno 111] Connection refused',))
2017-11-29 15:59:07.165 4097 ERROR ceilometer Traceback (most recent call last):
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/bin/ceilometer-upgrade", line 10, in <module>
2017-11-29 15:59:07.165 4097 ERROR ceilometer sys.exit(upgrade())
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/ceilometer/cmd/storage.py", line 53, in upgrade
2017-11-29 15:59:07.165 4097 ERROR ceilometer gnocchi_client.upgrade_resource_types(conf)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/ceilometer/gnocchi_client.py", line 113, in upgrade_resource_types
2017-11-29 15:59:07.165 4097 ERROR ceilometer gnocchi.resource_type.get(name=name)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/gnocchiclient/v1/resource_type.py", line 44, in get
2017-11-29 15:59:07.165 4097 ERROR ceilometer headers={'Content-Type': "application/json"}).json()
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/gnocchiclient/v1/base.py", line 37, in _get
2017-11-29 15:59:07.165 4097 ERROR ceilometer return self.client.api.get(*args, **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 217, in get
2017-11-29 15:59:07.165 4097 ERROR ceilometer return self.request(url, 'GET', **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/gnocchiclient/client.py", line 35, in request
2017-11-29 15:59:07.165 4097 ERROR ceilometer **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 142, in request
2017-11-29 15:59:07.165 4097 ERROR ceilometer return self.session.request(url, method, **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 101, in inner
2017-11-29 15:59:07.165 4097 ERROR ceilometer return wrapped(*args, **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 616, in request
2017-11-29 15:59:07.165 4097 ERROR ceilometer resp = send(**kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 690, in _send_request
2017-11-29 15:59:07.165 4097 ERROR ceilometer raise exceptions.ConnectFailure(msg)
2017-11-29 15:59:07.165 4097 ERROR ceilometer ConnectFailure: Unable to establish connection to http://controller:8041/v1/resource_type/ceph_account: HTTPConnectionPool(host='controller', port=8041): Max retries exceeded with url: /v1/resource_type/ceph_account (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f7bb28eec10>: Failed to establish a new connection: [Errno 111] Connection refused',))
2017-11-29 15:59:07.165 4097 ERROR ceilometer

Revision history for this message
gordon chung (chungg) wrote :

what does gnocchi-api logs say? you should probably paste your gnocchi.conf and ceilometer.conf files if you aren't able to find the error in gnocchi-api

gordon chung (chungg)
Changed in ceilometer:
status: New → Incomplete
Revision history for this message
Frank (f-sardis) wrote :

I have experienced this also. I followed the Pike guide for ceilometer and I get
Unable to establish connection to http://controller:8041/v1/resource_type/ceph
when i try to run ceilometer-upgrade --skip-metering-database
I do not use ceph in my openstack and i do not see anything in the ceilometer.conf that mentions ceph so it is not clear where this comes from.

I have tried gnocchi via apt and via pip. Same problem always. Gnocchi api doesn't seem to be installed via pip and via apt it does not register as a service at all so I cannot (re)start it.

Also the guide specifies using coordination_url = redis://controller:6379 but this produces a similar error during gnocchi-upgrade in that the redis service cannot be reached and the guide does not mention anything about installing it (nor does the gnocchi install guide). If i comment out the line, gnocchi-upgrade proceeds without a problem and uses the file driver.

There is no clarity in the guide on what is needed and for what reason which makes it difficult to troubleshoot.

Revision history for this message
gordon chung (chungg) wrote :

in the docs[1], there's a step that says: systemctl start openstack-gnocchi-api.service. can you run systemctl status openstack-gnocchi-api.service to check status? or journalctl to read logs?

if you deployed behind httpd, can you show the logs there?

regarding coordination_url, yes, that appears to be unexplained. i'll add explanation.

[1] https://docs.openstack.org/ceilometer/latest/install/install-base-rdo.html

Revision history for this message
gordon chung (chungg) wrote :

if you're wondering why it references ceph, it's becasuse ceilometer will create resource types (schemas) for all the possible resources it can theoretically monitor whether you enable them in ceilometer or not.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/540870

Changed in ceilometer:
assignee: nobody → gordon chung (chungg)
status: Incomplete → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (master)

Reviewed: https://review.openstack.org/540870
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=bd60ada44ae1948ef668fdf145879b5b215ae2f4
Submitter: Zuul
Branch: master

commit bd60ada44ae1948ef668fdf145879b5b215ae2f4
Author: gord chung <email address hidden>
Date: Mon Feb 5 14:30:21 2018 +0000

    add note explaining gnocchi coordination_url

    Change-Id: I632663679d0649e1d5912aef98a65c610fdeaac2
    Partial-Bug: #1745806

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.