Dharini and I spent sometime on this and it is indeed breaking the expected behavior around public and shared images like Andrew reported here.
From what we've seen, a couple of things are at play here.
#1. Swift multi-tenant store and swift+config scheme don't work well together. swift+config scheme is primarily meant for single-tenant store. If it is configured with multi-tenant store, the default reference configured is always used to access swift objects, which is orthogonal the idea of a multi-tenant store. There are a couple of ways this could be handled.
A) Add validation to glance-api such that it fails to start up when both multi-tenant store and swift+config are enabled. While this prevents a certain issue, it may mean that an operator using single-tenant store with swift+config now cannot switch to multi-tenant store. Not sure if anyone intends to do such a thing but it is certainly possible. This brings us to option B
B) To address the issue mentioned with option A, it is probably best to teach multi-tenant store to ignore the swift config file unless it comes across an image that was previously created using single-tenant store and swift+config. Any images that are created after the switch to multi-tenant store should remain unaffected whether or not a swift config file is provided.
I think Option A is the easiest fix for now before we figure out how to do option B
#2. The change that Andrew linked [0] doesn't cause the behavior reported in this bug directly. However, it is a similar change that went into connection manager which is responsible. Essentially, pulling the storage_url out of catalog won't work if the image one is trying to access is not created by him/her. So, we should always refer to the location mentioned in the database via location.swift_url.
One may ask, then why did we ever shift to using the storage_url from catalog in the first place?
Well, my theory is that it was an inadvertent behaviour introduced trying to fix [1]. The bug reported in bug [1] is exactly what we discussed above in issue #1. Both multi-tenant store and swift+config were set and it is because of this that the keystone url appears in the image location. To fix this, I think we went ahead and switched to using storage_url from the catalog. But, [1] really didn't need a code fix. It is just a weird behavior resulting from two orthogonal features at play.
To summarize, I think we need two patches here:
One to switch back to using location.swift_url instead of using storage_url from the catalog.(Do this in the connection manager too). And two, maybe add validation around configuration when both multi-tenant and swift+config are enabled. (Or figure out how to make multi-tenant store ignore swift+config)
Hope this made some sense. This was a tricky one to make sense of.
Let me know if my assessment is off here.
Dharini and I spent sometime on this and it is indeed breaking the expected behavior around public and shared images like Andrew reported here.
From what we've seen, a couple of things are at play here.
#1. Swift multi-tenant store and swift+config scheme don't work well together. swift+config scheme is primarily meant for single-tenant store. If it is configured with multi-tenant store, the default reference configured is always used to access swift objects, which is orthogonal the idea of a multi-tenant store. There are a couple of ways this could be handled.
A) Add validation to glance-api such that it fails to start up when both multi-tenant store and swift+config are enabled. While this prevents a certain issue, it may mean that an operator using single-tenant store with swift+config now cannot switch to multi-tenant store. Not sure if anyone intends to do such a thing but it is certainly possible. This brings us to option B
B) To address the issue mentioned with option A, it is probably best to teach multi-tenant store to ignore the swift config file unless it comes across an image that was previously created using single-tenant store and swift+config. Any images that are created after the switch to multi-tenant store should remain unaffected whether or not a swift config file is provided.
I think Option A is the easiest fix for now before we figure out how to do option B
#2. The change that Andrew linked [0] doesn't cause the behavior reported in this bug directly. However, it is a similar change that went into connection manager which is responsible. Essentially, pulling the storage_url out of catalog won't work if the image one is trying to access is not created by him/her. So, we should always refer to the location mentioned in the database via location.swift_url.
One may ask, then why did we ever shift to using the storage_url from catalog in the first place?
Well, my theory is that it was an inadvertent behaviour introduced trying to fix [1]. The bug reported in bug [1] is exactly what we discussed above in issue #1. Both multi-tenant store and swift+config were set and it is because of this that the keystone url appears in the image location. To fix this, I think we went ahead and switched to using storage_url from the catalog. But, [1] really didn't need a code fix. It is just a weird behavior resulting from two orthogonal features at play.
To summarize, I think we need two patches here:
One to switch back to using location.swift_url instead of using storage_url from the catalog.(Do this in the connection manager too). And two, maybe add validation around configuration when both multi-tenant and swift+config are enabled. (Or figure out how to make multi-tenant store ignore swift+config)
Hope this made some sense. This was a tricky one to make sense of.
Let me know if my assessment is off here.
[0] https:/ /git.openstack. org/cgit/ openstack/ glance_ store/commit/ ?id=68762058cc5 d063f3a846b495a f03150e648224f /bugs.launchpad .net/swift/ +bug/1511025
[1] https:/