Image location can be used to capture user tokens

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

jeeeeeeeeeeeeeeeeeeeeez! Image locations strike again.

Here's what I think about this specific issue. I don't believe we'll find a good solution for it that doesn' t require refactoring image locations. I've proposed a session for the summit and I hope it'll raise enough interest to help start discussing those changes that we need to make there.

For example, we need a better way to validate locations or a way to validate them to begin with.

So, for now, I think the best we can do is to recommend not allowing external users of the cloud to manipulate the image's locations at all. We have ways to prevent that with our config options and policies.

So, this seems like a B2 type of bug (according to https://security.openstack.org/vmt-process.html#incident-report-taxonomy ). I've subscribed ossg-coresec to check in an eventual OSSN.

I'm now curious how this works, why Glance would send a token to a new swift endpoint ?
And in the case of "multi-tenant swift store", what is the strategy to pick the correct token ?

The multi-tenant swift store traditionally works by reusing the original user token.

In multi-tenant mode the image data is stored as a Swift object in the user's own Swift account. You need a valid user token to access the data.

The flow is:

A request, containing an 'X-Auth-Token: token123' hits the Glance server. The server performs token validation using the keystone middleware. If some metadata is needed to perform the operation the token is passed on to a request to the Glance registry. Again the registry will use the keystone middleware to validate the token.

The above is common for both ways of accessing the swift store (single tenant and multi tenant).

In the case of the multi-tenant store the 'location' for the image is now read from the glance database.

This might look like:

 swift+http://192.168.1.103:8080/v1/AUTH_1a93b08616534cfa88af4ecaf5dbb35d/glance_a785957a-1340-498c-9427-3a549fc06842/a785957a-1340-498c-9427-3a549fc06842

The location tells Glance the URL where the data lives. When the location has been set by Glance's code (rather than specified by the user) the URL will point to the user's Swift in the local region. A token is required to access the user's swift account, so the initial token is reused once more. Otherwise Swift would return a 401 when the request to access the data arrived.

This works fine when the location is safely generated by Glance. But when a user can manually specify any string for the location they can potentially do bad things.

Note: If swift is running with https the operator can configure Glance to do a full cert check with the swift server. That would prevent a bogus https location being used. I don't think that saves us here though. You can set swift+http rather than swift+https, so the cert options are probably just ignored.

I think the proper solution here is to sanitize the location that convinces Glance to pass the user token.

@Travis

Can you describe what you mean by sanitizing the location? Thanks.

Looks like this bug is similar to https://bugs.launchpad.net/glance/+bug/1546507.
Looks like the long term solution must be the same as for that bug but not sure about backportable way to fix that.

Hey Stuart,

Sorry, I've re-read and looks like the sanitizing the location bit might have missed the point. Basically what we're saying is that the auth token is passed to a potentially user controlled Swift. Would it be possible to check the location and only pass the token for whitelisted Swift stores or is this un-necessarily complicated?

> Would it be possible to check the location and only pass the token for whitelisted Swift stores or is this un-necessarily complicated?

Remember, the location may or may not be for a Swift store. It could be any (enabled) store, eg an arbitrary http URL.

I think trying to provide a kind of (discoverable) white list would be one approach.
It could be a list of regex's to try to handle different stores etc.

I think really security conscious sites might hesitate to deploy something like that ... it's hard to know if you've caught all the potential bad ones. Using policies to allow setting locations for admin users or openstack services (via service tokens) would be an alternative (possibly complimentary) approach.

One advantage to structuring it as a whitelist is you don't have to catch the bad ones, you just list the sites that tokens should be passed to - everything else doesn't get a token. I like the approach of restricting location setting too.

> you don't have to catch the bad ones

Yes, for this case, agreed.

I think the setting locations stuff is just so hard to pin down for every case. Eg even limiting sending the token to known hosts, is it possible to use:

https://bugs.launchpad.net/glance/+bug/1546507

to delete shared images when using multi-tenant mode? Possibly not, but I'd have to check to be sure.

I think there's a certain amount of bravery required to enable setting locations :-)

Based on above comment and other related issue (bug 1549483 and bug 1546507), I'd like to handle image-location issues as class B2 type of bug according to VMT taxonomy ( http://security.openstack.org/vmt-process.html#incident-report-taxonomy ).

I'm switching the OSSA task status to opinion until a Security Note (OSSN) is defined.
However if a backportable fix can be implemented, then we can revert that decision and issue an advisory.

Marking it as high as it's limited to (or identified to be limited to) multi-tenant store.

Also, I'm adding Nova coresec team to the subscribers' list to help tie the ramifications of this bug with the ongoing Glance v2 work.

I don't see how this is related to Nova. Nova does not allow the creation of images with custom locations so it is not an attack vector here.

For the reason why nova-core sec has been added to this bug, please follow the conversation from comment #46 onward here https://bugs.launchpad.net/glance/+bug/1546507 .

In keeping with recent OpenStack vulnerability management policy changes, no report should remain under private embargo for more than 90 days. Because this report predates the change in policy, the deadline for public disclosure is being set to 90 days from today. If the report is not resolved within the next 90 days, it will revert to our public workflow as of 2020-05-27. Please see http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012721.html for further details.

The embargo for this report has expired and is now lifted, so it's acceptable to discuss further in public.