Service tokens do not work on cinder-volume charm

Bug #2024676 reported by Jorge Merlino
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Cinder Charm
Status tracked in Trunk
2023.1
Fix Released
Undecided
Unassigned
Trunk
Fix Committed
Undecided
Unassigned
Yoga
Fix Released
Undecided
Unassigned
Zed
Fix Released
Undecided
Unassigned

Bug Description

There is a bug with the service token configuration when cinder is separated between cinder and cinder-volume.
Plain cinder uses the identity-service relation with keystone while cinder-volume uses identity-credentials.
Cinder-volume can't create a token with the credentials obtained from identity-credentials. They fail to authenticate on keystone.
I can reproduce this on a focal/yoga openstack installation using cinder-volume and cinder-ceph but it also happens with pure-storage. Using the default service token configuration cinder-volume can not create a volume based on an image because it can't create the service token. This can be fixed by removing the [service-user] section on cinder-volume or by using the same credentials as those obtained by the identity-service relation.

This is the exception shown on the cinder-volume.log:

2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server [req-a49bac91-585d-47bd-9ed2-5a858f4ca2de 1d0e71f8854a4cddab77ac23b88dd841 38560b4f0be54e17af9a90ca2e90144f - - -] Exception during message handling: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f0458e09-ee73-4506-b053-f7b6553f6317)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "<decorator-gen-719>", line 2, in create_volume
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/objects/cleanable.py", line 208, in wrapper
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 815, in create_volume
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server _run_flow()
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 807, in _run_flow
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server flow_engine.run()
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 339, in reraise_if_any
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server failures[0].reraise()
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 346, in reraise
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server raise value
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server result = task.execute(**arguments)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", line 322, in execute
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server 'image_location': image_service.get_location(context,
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 336, in get_location
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server _reraise_translated_image_exception(image_id)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 635, in _reraise_translated_image_exception
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server raise new_exc.with_traceback(exc_trace)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 334, in get_location
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server image_meta = client.call(context, 'get', image_id)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 236, in call
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/v2/images.py", line 197, in get
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self._get(image_id)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/common/utils.py", line 670, in inner
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return RequestIdProxy(wrapped(*args, **kwargs))
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/v2/images.py", line 190, in _get
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server resp, body = self.http_client.get(url, headers=header)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 395, in get
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.request(url, 'GET', **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/common/http.py", line 361, in request
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server resp = super(SessionClient,
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 257, in request
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.session.request(url, method, **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 790, in request
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server auth_headers = self.get_auth_headers(auth)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1201, in get_auth_headers
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return auth.get_headers(self, **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/service_token.py", line 29, in get_headers
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server token = self.service_auth.get_token(session, **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 88, in get_token
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.get_access(session).auth_token
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 134, in get_access
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server self.auth_ref = self.get_auth_ref(session)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self._plugin.get_auth_ref(session, **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/v3/base.py", line 187, in get_auth_ref
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server resp = session.post(token_url, json=body, headers=headers,
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1149, in post
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.request(url, 'POST', **kwargs)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 986, in request
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server raise exceptions.from_response(resp, method, url)
2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f0458e09-ee73-4506-b053-f7b6553f6317)

Tags: sts
description: updated
description: updated
tags: added: sts
description: updated
Revision history for this message
Corey Bryant (corey.bryant) wrote :
Felipe Reyes (freyes)
summary: - Service tokens do not work on cider-volume charm
+ Service tokens do not work on cinder-volume charm
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.