[OSSA-2012-018] Removing user from a tenant isn't invalidating user access to tenant

I have subscribed the vulnerability management team to this bug. We need to evaluate this as a potential security vulnerability.

If confirmed, it's in the "extension of validity" class like the ones where you could extend the lifetime of a token. A bit hard to exploit (since you need to be given the role in the first place) but definitely needs to be fixed. I would issue an OSSA on this for the same reason we issued one on the previous token issues.

The EC2 mechanisms disregard role entirely, and work from the relationship of the user to the tenant. If that relationship didn't exist, the user shouldn't be able to access the tenant's resources through EC2. WIth the V2 API, just associating a user with a tenant is an implicit "membership" role - and in fact that's how Horizon displays this information.

Actual "roles" as Keystone defines them are irrelevant to the EC2 credential access, and with the V2 API's and the EC2 mechanisms, roles have no function there. They're not checked as a form of RBAC for allowing EC2 credential access.

The V3 API is aiming to clean this up a bit, more for the OpenStack API. V3 API changes does not currently specify any RBAC around EC2 credential authority.

We don't check that the user is still a member of the tenant (project) so the ec2 (or s3) token still works

vishy: heckj: correct, we don't check anywhere that the user is still a member of the tenant
[2:29pm] heckj: vishy: so the core of that bug is that we're not verifying that the user is a member of the tenant?
[2:30pm] vishy: correct
[2:30pm] vishy: i just marked it confirmed
[2:30pm] heckj: heh - I just marked it as triaged
[2:30pm] vishy: i just tested and removed myself from my tenant, I was still able to run commands with my ec2 token
[2:30pm] heckj: Okay - thank you! I'll get on it!

DELETE /tenants/{tenant_id}/users/{user_id}/roles/OS-KSADM/{role_id}
in keystone/contrib/admin_crud/core.py

[2:31pm] vishy: i can still do stuff through the nova api as well!

related bug around removing a user from a tenant:
https://bugs.launchpad.net/python-keystoneclient/+bug/1078497

ok tracked this down and verified. The other bug above is separate, but this one still occurs. I managed to successfuly delete the role and the ec2 commands still work. This is because in the ec2 credentials we are storing the user and tenant. We should fix this by adding a check in the ec2 authenticate to make sure that the user still has a valid role before validating.

Fix proposed to branch: master
Review: https://review.openstack.org/16028

Looks like security is back on track !

So if I get this straight, the issue is that if a user has a role that makes it associated with a tenant, but then that role is removed, the token is still valid. That would make it part of the "extension of validity" class I mentioned above, which we considered in the past as worthy of an advisory.

If confirmed, I'll prepare an impact statement.

Vish's folsom-backport tag probably means that Folsom is also affected. What about Essex at first glance ?

Reviewed: https://review.openstack.org/16028
Committed: http://github.com/openstack/keystone/commit/9d68b40cb9ea818c48152e6c712ff41586ad9653
Submitter: Jenkins
Branch: master

commit 9d68b40cb9ea818c48152e6c712ff41586ad9653
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Nov 13 15:49:19 2012 -0800

    Ensures User is member of tenant in ec2 validation

    It is possible that a user is no longer a member of a tenant when
    they attempt to use an ec2 token. This checks to make sure that
    the user still has at least one valid role in the tenant before
    authenticating them. This should automatically work for the s3
    version as well since it is a subclass.

    Fixes bug 1064914

    Change-Id: Ieb237bae936a7b00ce7ba4d4c59aec6c7a69ec21

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/16304

Reviewed: https://review.openstack.org/16304
Committed: http://github.com/openstack/keystone/commit/37308dd4f3e33f7bd0f71d83fd51734d1870713b
Submitter: Jenkins
Branch: stable/folsom

commit 37308dd4f3e33f7bd0f71d83fd51734d1870713b
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Nov 13 15:49:19 2012 -0800

    Ensures User is member of tenant in ec2 validation

    It is possible that a user is no longer a member of a tenant when
    they attempt to use an ec2 token. This checks to make sure that
    the user still has at least one valid role in the tenant before
    authenticating them. This should automatically work for the s3
    version as well since it is a subclass.

    Fixes bug 1064914

    Change-Id: Ieb237bae936a7b00ce7ba4d4c59aec6c7a69ec21

@Russell: we have a precedent for issuing after-the-fact OSSA on this kind of thing. Do you think we should here ? If yes, maybe we can give a few days warning to the stakeholders while we work on the impact statement ?

Yes, I think based on our precedent, we should issue an OSSA for this.

Would be good to push an Essex backport for this as well -- CCing stable/essex team

Proposed impact description:

Title: EC2-style credentials invalidation issue
Reporter: Vijaya Erukala
Products: Keystone
Affects: All versions

Description:
Vijaya Erukala reported a vulnerability in Keystone EC2-style credentials invalidation: when a user is removed from a tenant, issued EC2-style credentials would continue to be valid for that tenant. An authenticated and authorized user could potentially leverage this vulnerability to extend his access beyond the account owner expectations. Only setups enabling EC2-style credentials (for example enabling EC2 API in Nova) are affected.

Fix proposed to branch: stable/essex
Review: https://review.openstack.org/16759

Fix proposed to branch: stable/essex
Review: https://review.openstack.org/16760

Reviewed: https://review.openstack.org/16760
Committed: http://github.com/openstack/keystone/commit/8735009dc5b895db265a1cd573f39f4acfca2a19
Submitter: Jenkins
Branch: stable/essex

commit 8735009dc5b895db265a1cd573f39f4acfca2a19
Author: Vishvananda Ishaya <email address hidden>
Date: Tue Nov 13 15:49:19 2012 -0800

    Ensures User is member of tenant in ec2 validation

    It is possible that a user is no longer a member of a tenant when
    they attempt to use an ec2 token. This checks to make sure that
    the user still has at least one valid role in the tenant before
    authenticating them. This should automatically work for the s3
    version as well since it is a subclass.

    Fixes bug 1064914

    Change-Id: Ieb237bae936a7b00ce7ba4d4c59aec6c7a69ec21

@ttx: The vulnerability description sounds good to me. Thanks!

Pushed to downstream stakeholders. CVE-2012-5571

OSSA 2012-018

This was fixed in http://www.ubuntu.com/usn/usn-1641-1/.

Hello Vijaya, or anyone else affected,

Accepted keystone into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/keystone/2012.2.1-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I have verified this on grizzly-2 and validation seems to be working fine.

$ keystone user-role-remove --user bd9c28b6e2794574b030ad5c3a7e4818 --tenant 97c5778a10f34b499a5abe1e2ecff24c --role 9fe2ff9ee4384b1894a90878d3e92bab

$ euca-describe-instances
Unauthorized: Failure communicating with keystone

Hello Vijaya, or anyone else affected,

Accepted keystone into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/keystone/2012.1.3+stable-20130423-f48dd0fc-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Please find the attached test log from the Ubuntu Server Team's CI infrastructure. As part of the verification process for this bug, Keystone has been deployed and configured across multiple nodes using precise-proposed as an installation source. After successful bring-up and configuration of the cluster, a number of exercises and smoke tests have be invoked to ensure the updated package did not introduce any regressions. A number of test iterations were carried out to catch any possible transient errors.

Please Note the list of installed packages at the top and bottom of the report.

For records of upstream test coverage of this update, please see the Jenkins links in the comments of the relevant upstream code-review(s):

Trunk review: https://review.openstack.org/16028
Stable review: https://review.openstack.org/16760

As per the provisional Micro Release Exception granted to this package by the Technical Board, we hope this contributes toward verification of this update.

Test coverage log.

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.