Comment 0 for bug 1945500

Brief description:
Cinder (including Wallaby release) is not able to upload a volume that was build from an image back to glance, if multistore (glance) is enabled.

Details:
After enabling glance multistore there will be two extra properties with every image. Those are `os_glance_failed_import`and `os_glance_importing_to_stores`.
If cinder creates a volume from an imageRef it will store all image_metadata with that volume, including `os_glance_failed_import`and `os_glance_importing_to_stores`.
Using the cinder action `volume_client.volumes.upload_to_image` cinder will try to upload that volume to glance including those two properties. But they are "reserved attributes" (in glance). The upload will fail with:

INFO cinder.volume.api [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Volume info retrieved successfully.
INFO cinder.volume.api [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Get volume image-metadata completed successfully.
DEBUG glanceclient.common.http [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Request returned failure status 403. _handle_response /var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/http.py:125
ERROR cinder.volume.api [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] Error while doing something: HTTPForbidden: 403 Forbidden: Access was denied to this resource.: Attribute 'os_glance_importing_to_stores' is reserved. (HTTP 403)
ERROR cinder.volume.api Traceback (most recent call last):
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/api.py", line 1322, in copy_volume_to_image
ERROR cinder.volume.api context, self.image_service._translate_to_glance(metadata))
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/image/glance.py", line 378, in create
ERROR cinder.volume.api **sent_service_image_meta)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/image/glance.py", line 225, in call
ERROR cinder.volume.api return getattr(controller, method)(*args, **kwargs)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/utils.py", line 598, in inner
ERROR cinder.volume.api return RequestIdProxy(wrapped(*args, **kwargs))
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/v2/images.py", line 361, in create
ERROR cinder.volume.api resp, body = self.http_client.post(url, headers=headers, data=image)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 334, in post
ERROR cinder.volume.api return self.request(url, 'POST', **kwargs)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/http.py", line 377, in request
ERROR cinder.volume.api return self._handle_response(resp)
ERROR cinder.volume.api File "/var/lib/kolla/venv/lib/python2.7/site-packages/glanceclient/common/http.py", line 126, in _handle_response
ERROR cinder.volume.api raise exc.from_response(resp, resp.content)
ERROR cinder.volume.api HTTPForbidden: 403 Forbidden: Access was denied to this resource.: Attribute 'os_glance_importing_to_stores' is reserved. (HTTP 403)
ERROR cinder.volume.api
INFO cinder.api.openstack.wsgi [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] HTTP exception thrown: 403 Forbidden: Access was denied to this resource.: Attribute 'os_glance_importing_to_stores' is reserved. (HTTP 403)
INFO cinder.api.openstack.wsgi [req-321adb05-189b-49f9-aa54-62432ee25907 917f4ddabf954b92b84babc613f0292d a69fdf3fa6654adcb8de23c803df6712 - f9f86f7344e942a5a66cec801aa60dbe f9f86f7344e942a5a66cec801aa60dbe] http://cinder.service.stage.ewcs.ch/v2/a69fdf3fa6654adcb8de23c803df6712/volumes/16a09728-a1d9-4032-88a4-16a32536f481/action returned with HTTP 400

This issue is known by Nova and was fixed in Nova with:
https://github.com/openstack/nova/commit/bc290840127c3179227a662584404f9c0178d588

It looks like the issue is still unknown in cinder?

Howto reproduce:
1. Install devstack stable/wallaby
2. Change glance to enable multistore:
#####################
*** /etc/glance/glance-api.conf.org 2021-09-29 16:39:41.813610795 +0200
--- /etc/glance/glance-api.conf 2021-09-29 16:40:23.397360914 +0200
*************** image_cache_dir = /opt/stack/data/glance
*** 11,16 ****
--- 11,20 ----
  use_syslog = False
  debug = True

+ enabled_backends = az1:file, az2:file
+ show_multiple_locations = True
+ show_image_direct_url = True
+
  [database]
  connection = mysql+pymysql://xxxxxxxxx:xyzxyz@127.0.0.1/glance?charset=utf8

*************** auth_type = password
*** 35,42 ****
--- 39,58 ----
  [oslo_messaging_notifications]
  driver = messagingv2

+ [os_glance_staging_store]
+ filesystem_store_datadir = /opt/stack/data/glance//os_glance_staging_store
+
+ [os_glance_tasks_store]
+ filesystem_store_datadir = /opt/stack/data/glance/os_glance_tasks_store
+
  [glance_store]
+ default_backend = az1
+
+ [az1]
  filesystem_store_datadir = /opt/stack/data/glance/images/

+ [az2]
+ filesystem_store_datadir = /opt/stack/data/glance/images2/
+
  [cors]
  allowed_origin = http://172.16.0.6
#######################

mkdir /opt/stack/data/glance/images2
mkdir /opt/stack/data/glance/os_glance_tasks_store
mkdir /opt/stack/data/glance/os_glance_staging_store

systemctl restart <email address hidden>

3. Copy image to second store:
glance image-import --stores az2 --import-method copy-image $(openstack image show cirros-0.5.2-x86_64-disk -c id -f value)

4. Create a volume from an image:
openstack volume create --size 1 --image cirros-0.5.2-x86_64-disk testvol

5. Try to upload that volume to glance:
openstack image create --volume testvol --disk-format raw image-from-vol-from-image --debug

This will result in a 403:
HTTP 403 Forbidden: Access was denied to this resource.: Attribute &#x27;os_glance_failed_import&#x27; is reserved. (HTTP 400)

Fix: Best would be to not store os_glance* properties with a volume but we should also remove those properties when a volume is uploaded to glance.