keystone user-role-* operations fails when user no longer exists in backend

Bug #1321378 reported by Mike Dorman
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
Medium
Lance Bragstad
python-keystoneclient
Invalid
Undecided
Unassigned
python-openstackclient
Invalid
Undecided
Unassigned

Bug Description

When using an external user catalog (in our case, AD), if the user is removed on the backend catalog, the user-role-* keystone CLI commands no longer work, because keystone cannot look up the user.

The specific situation is a user had been granted roles on some projects, but then that user left the company and was removed from the backend directory. When going back to remove the roles assigned to that user, the keystone commands fail.

It may still be possible to do these operations directly through the API, I didn't check that. But ultimately was able to work around it by directly removing the entries in the keystone user_project_metadata table.

Revision history for this message
Dolph Mathews (dolph) wrote :

You're specifically looking for user-role-delete support, correct? (not -add?)

Changed in keystone:
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
Mike Dorman (mdorman-m) wrote :

That's right. For our particular use case, user-role-delete is what's needed.

Revision history for this message
Dolph Mathews (dolph) wrote : Re: keystone user-role-delete operation fails when user no longer exists in underlying catalog

I've narrowed the scope of this bug to match then -- I think that's completely legitimate. We have a history of getting bug reports from confused users when we allow "invalid" data to enter the database though (bad user_id references, etc) -- you should definitely be able to clean them out though!

summary: - keystone user-role-* operations fail when user no longer exists in
+ keystone user-role-delete operation fails when user no longer exists in
underlying catalog
Dolph Mathews (dolph)
summary: keystone user-role-delete operation fails when user no longer exists in
- underlying catalog
+ backend
Revision history for this message
Ajaya Agrawal (ajayaa) wrote : Re: keystone user-role-delete operation fails when user no longer exists in backend

If they are deleting the user directly and not through the call "openstack user delete" this problem will arise. The deletion of user should also call assignment_api.delete_user().
Alternative:
We could add a periodic job which would check the table assignment and user in keystone schema and remove the assignments which have no corresponding entry in user table.

Revision history for this message
Mike Dorman (mdorman-m) wrote :

For our use case, we cannot enforce an "openstack user delete" and/or delete_user() call. We use AD for backend authentication, and only have read-only access to it. User creation and deletion is handled at the corporate level and from the OpenStack perspective, we have no control over that.

Changed in keystone:
milestone: none → juno-rc1
Revision history for this message
Dolph Mathews (dolph) wrote :

Keystone should attempt the delete the assignment, and only 404 if no records were affected. The input doesn't need to be discretely validated.

description: updated
Henry Nash (henry-nash)
Changed in keystone:
assignee: nobody → Henry Nash (henry-nash)
Revision history for this message
Lance Bragstad (lbragstad) wrote :

Running some tests against this for V3 and I don't think this is an issue with with V3.

http://paste.openstack.org/show/108332/

Also working on a functional test for V3

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/119843

Changed in keystone:
assignee: Henry Nash (henry-nash) → Lance Bragstad (lbragstad)
status: Triaged → In Progress
Changed in keystone:
assignee: Lance Bragstad (lbragstad) → Henry Nash (henry-nash)
Changed in keystone:
assignee: Henry Nash (henry-nash) → Lance Bragstad (lbragstad)
Revision history for this message
Lance Bragstad (lbragstad) wrote : Re: keystone user-role-delete operation fails when user no longer exists in backend

This fix is going to be dependent on Brant's work here:

https://review.openstack.org/#/c/119629/6

which is gating.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/119843
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ea31841ad4d719840c1afaa3ddf7a4fe7bd39a6c
Submitter: Jenkins
Branch: master

commit ea31841ad4d719840c1afaa3ddf7a4fe7bd39a6c
Author: Lance Bragstad <email address hidden>
Date: Fri Sep 5 20:33:11 2014 +0000

    Allow users to clean up role assignments

    This change allows users the ability to remove role assignments in the
    event the user has been removed from the backend.

    Closes-Bug: #1321378
    Change-Id: I895e6e9718bc48344a53f192d5d56e6990da888e

Changed in keystone:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in keystone:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in keystone:
milestone: juno-rc1 → 2014.2
Revision history for this message
Adam Young (ayoung) wrote : Re: keystone user-role-delete operation fails when user no longer exists in backend

So...this is a continuing Saga. The fix that went in for Keystone only allows the V3 AP call to continue. However, there is currently no way to call that API except for CURL.

Something like:

 curl -X DELETE -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" $OS_AUTH_URL/projects/e9d504e8524e4c8d9876d179420dab89/users/tuser/roles/95a2366f8b514d43a5584342aefe448e

Will work, but there is no way to invoke that from python-keystoneclient or python-openwstackclient as both will attempt to list the users and do a lookup.

We probably need a --userid option that indicates that the passed in value is a userid, and do not attempt to look it up.

Revision history for this message
Adam Young (ayoung) wrote :

Reopening the issue against the Keystone server. The fix was not sufficient, as it was just a workaround, and one that we can't apply via the CLI.

The real fix requires avoiding the exception from the identity backend when performing any assignment-backend calls.

Changed in keystone:
status: Fix Released → Confirmed
summary: - keystone user-role-delete operation fails when user no longer exists in
+ keystone user-role-* operations fails when user no longer exists in
backend
Revision history for this message
Adam Young (ayoung) wrote :

Note that, while supporting Delete allowed the customer to work around the issue, a change made to LDAP (AD) external to Keystone has the effect of breaking Keystone, and that is the real problem here. It has been seen multiple times now in live deployments.

Revision history for this message
Adam Young (ayoung) wrote :

Closing the Keystone server component again, as I just confirmed the user-list error does not happen in this code base, and thus it is a new bug and a regression. Will open a separate ticket for that.

Changed in keystone:
status: Confirmed → Fix Released
Revision history for this message
QiangTang (qtang) wrote :

Does this issue still openstackclient or keystoneclient impact?

Changed in python-keystoneclient:
status: New → Invalid
QiangTang (qtang)
Changed in python-openstackclient:
status: New → Invalid
Revision history for this message
Lance Bragstad (lbragstad) wrote :

This is going to require changes to python-openstackclient

https://storyboard.openstack.org/#!/story/2006635

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.