This is most likely another error rather than the cache file itself. The snap doesn't expose a backend error in a nice way to the /metrics endpoint.
In my case, it was an exception in keystone client with SSL verification failure:
HTTPSConnectionPool(host='192.168.151.118', port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),))
$ curl localhost:9183/metrics
Traceback (most recent call last):
File "/snap/prometheus-openstack-exporter/29/bin/prometheus-openstack-exporter", line 740, in do_GET
collectors = [COLLECTORS[collector]() for collector in get_collectors(config.get('enabled_collectors'))]
File "/snap/prometheus-openstack-exporter/29/bin/prometheus-openstack-exporter", line 326, in __init__
with open(config['cache_file'], 'rb') as f:
IOError: [Errno 2] No such file or directory: '/var/snap/prometheus-openstack-exporter/common/RegionOne'
$ journalctl -u snap.prometheus-openstack-exporter.prometheus-openstack-exporter.service
...
Aug 11 01:05:19 juju-2ac58b-1-lxd-7 prometheus-openstack-exporter.prometheus-openstack-exporter[15486]: 127.0.0.1 - - [11/Aug/2021 01:05:19] "GET /metrics HTTP/1.1" 500 -
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 python2[15450]: Client setup done, keystone ver 3
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 python2[15450]: Error getting tenants.list, continue with projects.list
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 prometheus-openstack-exporter.prometheus-openstack-exporter[15450]: Client setup done, keystone ver 3
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 prometheus-openstack-exporter.prometheus-openstack-exporter[15450]: Error getting tenants.list, continue with projects.list
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 python2[15450]: Error getting stats: Traceback (most recent call last): File "/snap/prometheus-openstack-exporter/29/bin/prometheus-openstack-exporter", line 204, in run prodstack.update(self._get_keystone_info(keystone)) File "/snap/prometheus-openstack-exporter/29/bin/prometheus-openstack-exporter", line 152, in _get_keystone_info info['tenants'] = [x._info for x in keystone.projects.list()] File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/positional/__init__.py", line 108, in inner return wrapped(*args, **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneclient/v3/projects.py", line 119, in list **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneclient/base.py", line 75, in func return f(*args, **new_kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneclient/base.py", line 390, in list self.collection_key) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneclient/base.py", line 125, in _list resp, body = self.client.get(url, **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 386, in get return self.request(url, 'GET', **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 545, in request resp = super(LegacyJsonAdapter, self).request(*args, **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 248, in request return self.session.request(url, method, **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/session.py", line 747, in request auth_headers = self.get_auth_headers(auth) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/session.py", line 1158, in get_auth_headers return auth.get_headers(self, **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/plugin.py", line 95, in get_headers token = self.get_token(session) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 88, in get_token return self.get_access(session).auth_token File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access self.auth_ref = self.get_auth_ref(session) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 178, in get_auth_ref authenticated=False, log=False, **rkwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/session.py", line 1106, in post return self.request(url, 'POST', **kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/session.py", line 888, in request resp = send(**kwargs) File "/snap/prometheus-openstack-exporter/29/lib/python2.7/site-packages/keystoneauth1/session.py", line 983, in _send_request raise exceptions.SSLError(msg) SSLError: SSL exception connecting to https://192.168.151.118:5000/v3/auth/tokens: HTTPSConnectionPool(host='192.168.151.118', port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),))
This is most likely another error rather than the cache file itself. The snap doesn't expose a backend error in a nice way to the /metrics endpoint.
In my case, it was an exception in keystone client with SSL verification failure:
HTTPSConnection Pool(host= '192.168. 151.118' , port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError( SSLError( 1, u'[SSL: CERTIFICATE_ VERIFY_ FAILED] certificate verify failed (_ssl.c:590)'),))
$ curl localhost: 9183/metrics prometheus- openstack- exporter/ 29/bin/ prometheus- openstack- exporter" , line 740, in do_GET collector] () for collector in get_collectors( config. get('enabled_ collectors' ))] prometheus- openstack- exporter/ 29/bin/ prometheus- openstack- exporter" , line 326, in __init__ 'cache_ file'], 'rb') as f: prometheus- openstack- exporter/ common/ RegionOne'
Traceback (most recent call last):
File "/snap/
collectors = [COLLECTORS[
File "/snap/
with open(config[
IOError: [Errno 2] No such file or directory: '/var/snap/
$ journalctl -u snap.prometheus -openstack- exporter. prometheus- openstack- exporter. service openstack- exporter. prometheus- openstack- exporter[ 15486]: 127.0.0.1 - - [11/Aug/2021 01:05:19] "GET /metrics HTTP/1.1" 500 - openstack- exporter. prometheus- openstack- exporter[ 15450]: Client setup done, keystone ver 3 openstack- exporter. prometheus- openstack- exporter[ 15450]: Error getting tenants.list, continue with projects.list
File "/snap/ prometheus- openstack- exporter/ 29/bin/ prometheus- openstack- exporter" , line 204, in run
prodstack. update( self._get_ keystone_ info(keystone) )
File "/snap/ prometheus- openstack- exporter/ 29/bin/ prometheus- openstack- exporter" , line 152, in _get_keystone_info
info['tenants' ] = [x._info for x in keystone. projects. list()]
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ positional/ __init_ _.py", line 108, in inner
return wrapped(*args, **kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneclient/ v3/projects. py", line 119, in list
**kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneclient/ base.py" , line 75, in func
return f(*args, **new_kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneclient/ base.py" , line 390, in list
self.collection _key)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneclient/ base.py" , line 125, in _list
resp, body = self.client. get(url, **kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ adapter. py", line 386, in get
return self.request(url, 'GET', **kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ adapter. py", line 545, in request
resp = super(LegacyJso nAdapter, self).request( *args, **kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ adapter. py", line 248, in request
return self.session. request( url, method, **kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ session. py", line 747, in request
auth_headers = self.get_ auth_headers( auth)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ session. py", line 1158, in get_auth_headers
return auth.get_ headers( self, **kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ plugin. py", line 95, in get_headers
token = self.get_ token(session)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ identity/ base.py" , line 88, in get_token
return self.get_ access( session) .auth_token
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ identity/ base.py" , line 134, in get_access
self.auth_ ref = self.get_ auth_ref( session)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ identity/ v3/base. py", line 178, in get_auth_ref
authenticated= False, log=False, **rkwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ session. py", line 1106, in post
return self.request(url, 'POST', **kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ session. py", line 888, in request
resp = send(**kwargs)
File "/snap/ prometheus- openstack- exporter/ 29/lib/ python2. 7/site- packages/ keystoneauth1/ session. py", line 983, in _send_request
raise exceptions. SSLError( msg)
SSLError: SSL exception connecting to https:/ /192.168. 151.118: 5000/v3/ auth/tokens: HTTPSConnection Pool(host= '192.168. 151.118' , port=5000): Max retries exceeded with url: /v3/auth/tokens (Caused by SSLError( SSLError( 1, u'[SSL: CERTIFICATE_ VERIFY_ FAILED] certificate verify failed (_ssl.c:590)'),))
...
Aug 11 01:05:19 juju-2ac58b-1-lxd-7 prometheus-
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 python2[15450]: Client setup done, keystone ver 3
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 python2[15450]: Error getting tenants.list, continue with projects.list
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 prometheus-
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 prometheus-
Aug 11 01:10:10 juju-2ac58b-1-lxd-7 python2[15450]: Error getting stats: Traceback (most recent call last):