Keystone App Creds Access Rules don´t work

I was able to reproduce the problem on the trunk (trunks for Keystone and Glance).

According to my tests, it is not important if the application credentials were created by an admin user or a regular user (with no admin role).

The result is, the access rules are ignored if they are applied to service "identity". But they work for other services as intended. The app creds work as intended otherwise, and setting _only_ OS_AUTH_TYPE=v3applicationcredential, OS_APPLICATION_CREDENTIAL_SECRET=yyyyy, OS_APPLICATION_CREDENTIAL_ID=zzzzzzzzzz is sufficient to authenticate with user's normal roles. Seems exactly as Simon reported.

BTW, Devstack does not configure service_type=xxxx in [keystone_authtoken] for any of its services, so it needs adding as documentation explains.

Isn't it the case that Keystone API does not have keystonemiddleware in its pipeline?

I looked a little at the cause and I found the difference.

All other services just run keystonemiddleware normally, but Keystone uses a middleware known as "auth_context". It has a class AuthContextMiddleware that inherits from BaseAuthProtocol, provided by the auth_token module in keystonemiddleware. That class has its own process_request, which literally stubs all the exceptions and invokes the parent's process_request. But unfortunately, the process_request in keystonemiddleware expects to report errors by raising!

Look at

keystone/server/flask/request_processing/middleware/auth_context.py:
class AuthContextMiddleware(...., auth_token.BaseAuthProtocol):
@middleware_exceptions
def process_request(self, request):

and

keystonemiddleware/auth_token/__init__.py:
class BaseAuthProtocol(object):
def process_request(self, request):
def validate_allowed_request(self, request, token):

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/keystone/+/874346

I'm going to change my mind about the root cause. It appears that we simply forgot to check if the authorization has set the flag request.user_token_valid, which keystonemiddleware checks in AuthProtocol.process_request(). I copied the missing chunk to AuthContextMiddleware.process_request() and it cleared the problem.

See WIP patch https://review.opendev.org/c/openstack/keystone/+/874346

This issue was fixed in the openstack/keystone 24.0.0.0rc1 release candidate.