Performance issue when delete tokens for users

I think the root cause is in the keystone/keystone/identity/controllers.py, we delete the token for user one by one in the for loop instead of do it in one session.

    def _delete_tokens_for_user(self, context, user_id, project_id=None):
        #First delete tokens that could get other tokens.
        for token_id in self.token_api.list_tokens(context,
                                                   user_id,
                                                   tenant_id=project_id):
            try:
                self.token_api.delete_token(context, token_id)
            except exception.NotFound:
                pass

The better solution is we provide a new api like token_api.delete_token_by_user(context, user_id) to batch delete tokens.

Please help me understand why there are so many tokens.
In your test scenario, total 300 users, why not just 300 tokens, one per user? Not introducing new token till earlier one for the same user expires. I was recently examining db tables in my dev environment and wondered about it.

Regards
Malini

Our test cases are trying to simulate many users login/logout concurrency, disable/enable user, create/delete user in long run env.
Each time you login will generate a new token recorder in db. The token will not be deleted even it expires before the user is deleted or disabled.

I understand your need to stress test, many user environment. I would basically create random users and logins associated from each .. 100, 200, 500, 1000 etc and see when the system breaks, where etc, response times.
But for each user a single token for scalability. After they login, and are validated, link to the same token, if anything extend the expire time of the token by whatever default lifetime interval.

Here is another bug stemming from a token explosion.
https://bugs.launchpad.net/keystone/+bug/1168399

So it looks like for every user action (e.g. login/logout) tokens are generated, how fast can a user log in and out, I'm guessing through the API and a high speed network connection they can do this pretty fast (especially since they can run it concurrently?). This issue may be security related if an attacker can cause enough tokens to be created to trigger a denial of service condition. Also how do we flag public bugs with the "security" tag (I don't want to make it private, I just want to make sure the right people see it)?

For the security issue I think it is another story should not be covered in this bug, I believe we already have some proposal regarding it, such as use OAuth to generate token, reduce the token expiration date, renew token and etc.

Here, I just want to improve the performance of token deletion and make it more efficiency.

Fix proposed to branch: master
Review: https://review.openstack.org/28866

Reviewed: https://review.openstack.org/28866
Committed: http://github.com/openstack/keystone/commit/d6cfe4f2e2c7c1b3eb23bf9f864779940a645555
Submitter: Jenkins
Branch: master

commit d6cfe4f2e2c7c1b3eb23bf9f864779940a645555
Author: gengjh <email address hidden>
Date: Sat May 11 17:20:10 2013 +0800

    Improve the performance of tokens deletion for user

    Provide new delete the tokens api 'delete_tokens' to support
    delete all the tokens for user in one session in the sql backend. For
    the kvs and memcache, I also provide the corresponding implementation.

    Fix bug 1178063

    Change-Id: I986a583e5900ea04e26cbdb7c49638a33818bca7

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/31907

Reviewed: https://review.openstack.org/31907
Committed: http://github.com/openstack/keystone/commit/ef467b2281ba40c858faa76ec8315c637481f8cf
Submitter: Jenkins
Branch: stable/grizzly

commit ef467b2281ba40c858faa76ec8315c637481f8cf
Author: gengjh <email address hidden>
Date: Sat May 11 17:20:10 2013 +0800

    Improve the performance of tokens deletion for user

    Provide new delete the tokens api 'delete_tokens' to support
    delete all the tokens for user in one session in the sql backend. For
    the kvs and memcache, I also provide the corresponding implementation.

    Fix bug 1178063

    Change-Id: I986a583e5900ea04e26cbdb7c49638a33818bca7
    (cherry picked from commit d6cfe4f2e2c7c1b3eb23bf9f864779940a645555)