Swift+Glance stops working after changing service password

This is a pretty major design flaw that we definitely want to rectify.

It's also even more of a hassle if a deployer is encrypting the stored location information (ie: conf.metadata_encryption_key), if we provide a tool as a solution then we will want to keep that in mind.

Agreed. It's a major design flaw that the login and password are even stored in the database at all...

This is something that should be handled by every store implementation. Stores should store the remote path without credentials or host information, IMHO. The thing is that those parameters can vary (as the example used for this issue) and also, having them stored in the database represent a security issue as well, again IMHO.

Perhaps we should create a single bug for every store doing this and make this depend on those.

Fix proposed to branch: master
Review: https://review.openstack.org/22182

While I agree that your experience is the result of a bug in glance, I do want to highlight that there is a valid use case for not having all swift images use the same set of credentials.

In particular, it is quite possible to imagine that you would start out a glance deployment with one default swift store that gradually fills up. Suppose that once that swift store is full, you want to start putting new images in a new swift deployment. It is useful to be able to have all new images going to the new store, while continuing to serve old images out of the old.

However, I think we can and should find a way to support multiple swift storage points for a single glance deployment *without* having to store credentials in the database. And we can probably provide some configuration or tooling around swift store locations that will save the effort the OP was complaining about.

I agree with Mark here.

The review proposed as (WIP) is meant to be read as a POC / start idea of how it could work. We should definitely find a way to support both cases or at least a way to update credentials when / if needed.

I suggested on gerrit (on some review comment) to write a blueprint for these changes and use that as reference for addressing all these bugs.,

The other thing to consider is that this issue has an impact on all store backends.

Mark,

I know that. But if I want to change credentials of some storages (for security reasons, after changing IP of swift-proxy or for some other purposes), now I dont have any tool to do it correctly, without direct access to database.

As for me, glance can store credentials (and paths, IPs, and anything other) in database (or in any other place), but must provide some tool/interface for viewing and editing associated storages, credentials and paths.

Isn't this addressed by the fix to remove creds from location? Of course, the fix itself won't help entirely unless the migrations are run.

 Still reproducible - we hit that on Icehouse code.

Was this ever resolved? I am seeing this in GLance DB swift+http://service%3Aswift:swift_p%40ss@192.168.8.2:5000/v2.0/glance/1568499f-bf25-43dc-a0db-2de7c029ace1

Also having issues with error 500 trying to do basic glance image-list when pointing glance to keystone.
When not pointed to keystone everything works except snapshots that are in private mode not view able by owner.

Bobby, it would appear it wasn't resolved.

You can configure credentials in a different way which avoids the password being stored in the database (even for the single tenant swift store).

This can be done via the glance-swift.conf file, eg:

https://github.com/openstack/glance/blob/master/etc/glance-swift.conf.sample

when you do this, the 'reference' is stored in the database rather than the credentials.
The current username/password are swapped in when required.

I think this means you can do a 'rolling' password change by using two users in the same tenant. While changing password some glance nodes will have tenantX:user1 (valid) and some nodes will have tenantX:user2 (also valid). (Though I haven't tested that).

Should we consider moving to this as the default configuration?

(There is no solution to the previously existing entries with passwords - but I would see that as a different bug.)

Yeah I think the previously existing entries will need a migration at least.

Someone needs to pick this work up https://review.openstack.org/#/c/103981/ . I'm running low on cycles at this point so, any effort here would be appreciated.

Agree cleaning up old entries would be a good thing to do -- not advisory material though

triage bump. This is still a valid issue.

I wonder if we can remove the possibility of storing credentials in the location in favor of just using the new way that doesn't require that.

@Flavio: I think not, not unless we provide a clean db migration for operators that takes care of such locations.

I think we should do something about this in Ocata using Nikhil's spec proposal (for Kilo!) as a start. I agree with Nikhil that it's essential to provide a way to migrate old-style locations to new-style locations. I don't see time for this to happen in Newton, though.

@Brian, perhaps we should do this in Pike instead. ;-)