tenantId/default_project_id missing on Keystone service user in Mitaka

What do you see if you use the openstack cli (remember that the keystone cli is deprecated)?

(as an aside, the default_project_id is stored in the users table not the local_users table)

same with openstack cli:

# openstack user show a481a1f43ec0463083b7a30d20493d38
+----------+----------------------------------+
| Field | Value |
+----------+----------------------------------+
| email | nova@localhost |
| enabled | True |
| id | a481a1f43ec0463083b7a30d20493d38 |
| name | nova |
| username | nova |
+----------+----------------------------------+

user DB table only lists the postgres user:

keystone=# \dt

                    List of relations
 Schema | Name | Type | Owner
--------+------------------------+-------+----------------
 public | access_token | table | admin-keystone
 public | assignment | table | admin-keystone
 public | config_register | table | admin-keystone
 public | consumer | table | admin-keystone
 public | credential | table | admin-keystone
 public | domain | table | admin-keystone
 public | endpoint | table | admin-keystone
 public | endpoint_group | table | admin-keystone
 public | federated_user | table | admin-keystone
 public | federation_protocol | table | admin-keystone
 public | group | table | admin-keystone
 public | id_mapping | table | admin-keystone
 public | identity_provider | table | admin-keystone
 public | idp_remote_ids | table | admin-keystone
 public | implied_role | table | admin-keystone
 public | local_user | table | admin-keystone
 public | mapping | table | admin-keystone
 public | migrate_version | table | admin-keystone
 public | password | table | admin-keystone
 public | policy | table | admin-keystone
 public | policy_association | table | admin-keystone
 public | project | table | admin-keystone
 public | project_endpoint | table | admin-keystone
 public | project_endpoint_group | table | admin-keystone
 public | region | table | admin-keystone
 public | request_token | table | admin-keystone
 public | revocation_event | table | admin-keystone
 public | role | table | admin-keystone
 public | sensitive_config | table | admin-keystone
 public | service | table | admin-keystone
 public | service_provider | table | admin-keystone
 public | token | table | admin-keystone
 public | trust | table | admin-keystone
 public | trust_role | table | admin-keystone
 public | user | table | admin-keystone
 public | user_group_membership | table | admin-keystone
 public | whitelisted_config | table | admin-keystone
(37 rows)

keystone=# select * from user;
 current_user
--------------
 postgres
(1 row)

is the user you are doing show for a postgres user or an LDAP/AD user?

it is a services user stored in the SQL identity backend (not using LDAP for service users)

(Sorry for all the questions)...and is your client configured to talk v2 or v3 of the Identity API?

So for the V2 API, if there is no default project defined for a user, the tenantID attribute will not appear in the returned entity (which is a bit odd, I agree)

Did these users have default projects before the migration? If so, then one thing to look at (if possible) is a dump of the Users table, to see if the entries that match these do or do not have a default_proejct_id defined.

Thanks Henry, your responses lended to me some important clues which is if the project is not provided during user creation then the default_project_id mapping will not be added. Subsequently even if a role is granted for this user on said project then also no default_project_id is created.

In our deployment we are using Packstack/Puppet to create the services users.

Mitaka's puppet-keystone (unlike Kilo/Liberty's puppet-keystone) does not provide a project when creating Keystone services users {ceilometer, nova, neutron etc}. It subsequently assigns an admin role for each service user to the services project and ideally that should have created a project Id mapping within the user reference but it doesn't.

You can also reproduce this issue through standard openstack user create/openstack role add cmds

Therefore the users objects in keystone end up with NO default_project_id/tenantId fields which has the following two disadvantages:
1) The Users detail view in Horizon's Identity pane shows ProjectID as "None"
2) RBAC policies pertaining to the services project do not work since it cannot correlate the services users to the services project

I have a fix for this in my deployment which is to update the user reference with project Id during role grants.

I'll raise a review for this shortly incase this is something the community is interested in.

Ron, I already have a fix for this.

I am a bit confused by this bug.

1. Are we saying existing users with default_project_id field set to some project prior to upgrading to Mitaka are now seeing default_project_id set to None after upgraded to Mitaka?
2. Is this impacting ALL users or just the service users? Comments in #8 seem to suggest this is only impacting the service users.
3. Why would assigning the 'admin' role to the services users for the service project automatically set the 'default_project_id' field? I don't remember we ever support that.
4. Horizon behavior is expected if user have no default project.
5. Not sure if I understand this statement, "RBAC policies pertaining to the services project do not work since it cannot correlate the services users to the services project". If the given token is scoped to a project, RBAC policies should work accordingly. Are we saying we have policies that act on the 'default_project_id'?

 See inline with [Kam]:

1. Are we saying existing users with default_project_id field set to some project prior to upgrading to Mitaka are now seeing default_project_id set to None after upgraded to Mitaka?

[Kam]: Yes, as part of our general Openstack upgrade, we also updated the puppet modules. In Kilo's puppet-keystone, a project Id was provided when creating the services users. Whereas in puppet-keystone for Mitaka, the services users are created in two stages, 1) create user (without specifying project Id), 2) grant a member/admin role for these users to the services project.

However I would like to add that puppet just exposed the problem and is not specific to Puppet. This issue can be reproduced with openstack client.

2. Is this impacting ALL users or just the service users? Comments in #8 seem to suggest this is only impacting the service users.

[Kam]: Impacts all users that do not have projectId specified during creation and subsequently have a role granted to them on said project.

3. Why would assigning the 'admin' role to the services users for the service project automatically set the 'default_project_id' field? I don't remember we ever support that.

[Kam]: This has nothing to do with the 'admin' role in general and can be seen for any role/any user.

4. Horizon behavior is expected if user have no default project.

[Kam]: If a user does NOT have a default project(none specified during creation), shouldn't granting the first role on this user to a project set that as default project? Otherwise you end up with a scenario where Horizon indicates that user is projectless whereas it has a role on a project

5. Not sure if I understand this statement, "RBAC policies pertaining to the services project do not work since it cannot correlate the services users to the services project". If the given token is scoped to a project, RBAC policies should work accordingly. Are we saying we have policies that act on the 'default_project_id'?

[Kam]: Yes, this is specific to our deployment as we have modified the RBAC policies. I was mentioning it for posterity, it should not affect Upstream. Horizon not showing ProjectID, as well as logic that expects default_project_id to be within the user reference breaks because of this issue

On the the initial report: it looks suspiciously like postgres is executing a view when doing select * from user;

The user table does not have a current_user field. That looks like a postgres entity.

So a few additional comments:

1) In Keystone V2, if a user has a default project, then they are automatically assigned a role on it (the member role). This is not true for V3 (and this is by design). Having a default project with a user created via the V3 API does not give them any role assignments. You have to add these manually.

2) Sounds to me like the puppet models for deploying keystone you are using do not take this into account

3) For a user that does not have a default project, granting them a role on a project will absolutely not make this project their default project (and nor should it).

4) The one thing I am not clear on, is whether there is an issue for a client (e.g. Horizon) talking the keystone V2 API with the fact that if a user has no default project, then the tennatID attribute will not appear at all in the user entity returned (whereas using V3 you would get default_project_id=None in the entity)

Fix proposed to branch: master
Review: https://review.openstack.org/349059

Henry, sent out the code review since it was already in my pipeline and so as to visualize the changes but if this conflicts with Keystone's design philosophy then feel free to indicate that on the code inspection.

If there is no fix in Keystone then we need to modify the puppet-keystone modules to account for no default project being assigned when creating roles(puppet-kilo/liberty used to do this)

Hi

I think the puppet change is the right thing, I don't think there will be much support for changing the keystone design here.

I've marked this as invalid, but let me know if there are other aspects of this that we have not discussed.

Summary thus far:

The change to modify Keystone server to assign a default project Id to a user reference, if one does not exist, has been withdrawn since Identity V3 is functioning as designed.

Instead, we will address this as a puppet-keystone change to:
Specify project Id when creating keystone admin/services users so that:
       a) Horizon>Identity panel does not show an empty value for Project ID under the User detailed view
       b) RBAC policy(ies) that relies on target.user.default_project_id continues to function as intended

Change abandoned by Steve Martinelli (<email address hidden>) on branch: master
Review: https://review.openstack.org/349059
Reason: bug is invalid and no change in months

using V3 API we cannot access user email?

users = keystone.users.list()
for row in users:
        print row.email

Getting below error :
/*******************************************/
 File "/usr/lib/python2.7/site-packages/keystoneclient/base.py", line 496, in __getattr__
    raise AttributeError(k)
AttributeError: email

/*******************************************/
becuase in the user details, email field does not exist:
<User domain_id=e07a8e155862444eb9e966121fd161ce, enabled=True, id=8cda896fe33e4ed58e39b97157d38280, links={u'self': u'http::5000/v3/users/8cda896fe33e4ed58e39b97157d38280'}, name=, password_expires_at=None>

if deafult project exist then only we can access email field?
<User default_project_id=8cb63b3a72894e18ad18b1c56b4dc3e9, domain_id=e07a8e155862444eb9e966121fd161ce,
<email address hidden>, enabled=True, id=9ba608bc868d4333be1d06a8db3e8cbc,
links={u'self': u'http://xyz:5000/v3/users/9ba608bc868d4333be1d06a8db3e8cbc'}, name=anil, password_expires_at=None>

So how to access user email field when default_project_id does not exist?

I set up a test locally to see if I could recreate what you're seeing [0]. I am able to list users regardless of the default project id or the email attribute being present. Are you using a specific version of python-keystoneclient. I attempted this with master of both keystone and python-keystoneclient.

[0] http://cdn.pasteraw.com/d79a18f4pnydnqboni2svg5zdolsmos