Admin users cannot authenticate properly via admin_domain

Are you creating a new admin user and trying to do a domain list? Or are you getting a domain scoped token from the 'admin' user in the admin domain as per https://wiki.ubuntu.com/OpenStack/OpenStackCharms/ReleaseNotes1604#Keystone_v3_API_support ?

I'm going to grab this bug and update the README off the back of it because this is not the first time people have had issues with authenticating against a v3 cloud.

I've tried to authenticate against the API with the following environment variables populated:

OS_REGION_NAME=redacted
OS_USER_DOMAIN_NAME=admin_domain
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=redacted
OS_DOMAIN_NAME=admin_domain
OS_AUTH_URL=http://redacted:35357/v3
OS_USERNAME=admin

I still get a 403 when I try to list users or domains via the openstack cli client.

Here is a copy of policy.json from the environment. To me, this looks like it should work for listing users or domains, unless admin_and_matching_domain_id on line 9 is incorrect.

The templates in the charm that are used for policy.json have been edited by hand on the units to add the correct domain id for the cloud_admin rule on line 3. This is the workaround for bug #1637453, which I see is now fixed.

-Greg

This auth is failing for use with the admin_domain scope. The auth for these operations is failing for the user "admin" and any other users with the role "Admin" in the domain "admin_domain".

Greg, when you're setting up the environment variables are you ensuring that no other OS_* variables are set ? fwiw to ensure no other OS_* env vars pollute the environment my novarc looks like this:

_OS_PARAMS=$(env | awk 'BEGIN {FS="="} /^OS_/ {print $1;}' | paste -sd ' ')
for param in $_OS_PARAMS; do
    unset $param
done
unset _OS_PARAMS
export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -f keystone`:5000/v3
export OS_USERNAME=admin
export OS_PASSWORD=openstack
export OS_REGION_NAME=RegionOne
export OS_DOMAIN_NAME=admin_domain
export OS_USER_DOMAIN_NAME=admin_domain
export OS_IDENTITY_API_VERSION=3
# Swift needs this:
export OS_AUTH_VERSION=3

Yes, the novarc I'm using has that function.

When I've tested different combinations of environment variables, I've been sure to use unset to clean up the environment, and checking the output of env to be sure of what the environment has set. The variables I provided earlier were in the output of "env | grep OS_" rather than a novarc.

The issue was the roles granted to the admin user by the charm. The 'admin' user should be granted the 'Admin' role against the 'admin_domain' but for some reason the 'admin' user only had the 'Member' role against the 'admin_domain'.

Querying the keystone database directly shows how the grants should look:

select a.type, c.name, d.name
from assignment a, local_user b, role c, project d
where a.actor_id = b.user_id
and b.name = 'admin'
and c.id = a.role_id
and d.id = a.target_id;

+-------------+--------+--------------+
| type | name | name |
+-------------+--------+--------------+
| UserProject | Admin | admin |
| UserProject | Member | admin |
| UserDomain | Admin | admin_domain |
+-------------+--------+--------------+

However in the case of the deployment gmason was having issues I think the query would have shown:

+-------------+--------+--------------+
| type | name | name |
+-------------+--------+--------------+
| UserProject | Admin | admin |
| UserProject | Member | admin |
| UserDomain | Member | admin_domain |
+-------------+--------+--------------+

@gmason Could you confirm what the keystone charm setting admin-role is set to in your deployment?

@gmason the logs from /var/log/juju/* on the keystone units would also be useful.

I have been unable to reproduce this bug so far but will keep trying.

the juju keystone admin-role is configured to "Admin"

Here's the output of the MySQL query:
+-------------+--------+--------------+
| type | name | name |
+-------------+--------+--------------+
| UserProject | Admin | admin |
| UserProject | Admin | admin_domain |
| UserDomain | Admin | admin_domain |
| UserDomain | Member | admin_domain |
+-------------+--------+--------------+

Here are sanitized versions of the requested logs from the environment.

Hi

Is there a plan to fix this bug? we are hitting this bug that user with Admin role is unable to list role assignment but operations via Horizon succeeds

Hi Linda

Can you verify whether that user can perform the action using the CLI rather than horizon. It would help to verify whether this is a policy issue (in horizon), or a policy/misconfiguration issue in keystone.

Also: which version of OpenStack/Ubuntu and charms is this on please?
Has this OpenStack been upgraded?

Thanks.

Hi Alex.

The user with admin role can perform action (list role) using Horizon, but CLI failed. I have reviewed the keystone policy file, it looks good.

"admin_required": "role:Admin",

"identity:get_role": "rule:admin_required",
    "identity:list_roles": "rule:admin_required",
    "identity:create_role": "rule:cloud_admin",
    "identity:update_role": "rule:cloud_admin",
    "identity:delete_role": "rule:cloud_admin",

openstack/ubuntu version:
Ubuntu 18.04 (Bionic)
OpenStack Queens

keystone charm is at version 312

Openstack hasn't been upgraded

Hi Linda

Then it's almost certainly a mis-configuration of the admin's roles as per comment #8 in this thread. It would be useful to explore the database and see what the admin user and roles look like w.r.t. to the outputs shown in #8.