New image is not uploaded if time to upload of image greater than time to life of keystone token.

diagnostic snapshot:
https://drive.google.com/a/mirantis.com/file/d/0BxAwYghQ8JfbM3lUM3dYYkZtRXM/view?usp=sharing

Sadly, but its a known issue. Some cosmetic surgery is possible at the moment by applying this patch: https://review.fuel-infra.org/#/c/6303/ (it turns 500 into 400 BadRequest), but, alas, it won't help uploading the image.

A different mechanism for large file upload has to be implemented, possibly based on keystone trusts (https://bugs.launchpad.net/fuel/6.0.x/+bug/1441156 - see Mike Fedosin's last comment on the subject).

Upstream fix on review: https://review.openstack.org/#/c/199049/

We have a fix for this bug for ceph - https://review.openstack.org/#/c/213757/. Commit for Swift is under review in community (link above) - when it's merged I will backport it to MOS 7.0.

Currently I set this bug a duplicate of https://bugs.launchpad.net/fuel/+bug/1485639. Swift related bug for MOS 8.0 will be added soon.

Reviewed: https://review.fuel-infra.org/14329
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: fedcb444d11eacbfa51521f2f5479c62eb93e093
Author: Mike Fedosin <email address hidden>
Date: Mon Dec 28 14:05:06 2015

Implement trust support for Glance

This commit adds new functionality in Glance,
called 'trusts', that are used to prevent token
expiration during longtime upload.

The algorighm is the following:
1. If galnce uses registry server to communicate with db, create a trust
2. Upload an image
3. Try to update the image status to 'active' in registry
4. If trust has been created succcessfully and 401 occured during
   updating the image status then renew the token and try to update
   the image record in registry again

If keystone server doesn't support trust functionality
then glance automatically returns to the basic workflow
with user token.

Co-Authored-By: Mike Fedosin <email address hidden>
Co-Authored-By: Kairat Kushaev <email address hidden>

Closes-bug: #1456573

Change-Id: Ia3b82782b14f5dfc93457620633c1039c38fc366

QA testing on devstack:
0. Preparation: Set glance backend to RBD (Ceph), set 'expiration' option in /etc/keystone/keystone.conf to '40', set 'token_cache_time' in /etc/glance/glance-registry.conf to '-1', set 'data_api' in /etc/glance/glance-api.conf to 'glance.db.registry.api'.
1. Try to upload big image (when upload takes at least 1 minute). (glance image-create --name 'test' --disk-format raw --container-format bare --progress --file image.img) Make sure that upload was successfully finished.
2. Perform step 1 again with glance v1 api (glance --os-image-api-version 1 image-create --name 'test' --disk-format raw --container-format bare --progress --file image.img) Make sure that upload was successfully finished.

This fix doesn't work with Swift backend (RadosGW works good), because there is another issue in Swift driver. Code that fixes the issue is on review in upstream. When it's merged it will be backported in MOS. https://review.openstack.org/#/c/251851/11

Verified on VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "457"
  build_id: "457"
  fuel-nailgun_sha: "db88977ea5ec8a034379a953f72627c3819dfd24"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "0c1b17537a35d1b56f382fcc48197d0ee0c02d90"
  fuel-nailgun-agent_sha: "92ebd5ade6fab60897761bfa084aefc320bff246"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "42f1059465ccd7e6b501d9bdca0076fb89a64339"
  fuel-ostf_sha: "f66b1208bb9214513a042d52226583473f6117ea"
  fuel-mirror_sha: "b62f3cce5321fd570c6589bc2684eab994c3f3f2"
  fuelmenu_sha: "6c1888a48e2c3244d7fdf6409c22874aac8a7495"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "9f0ba4577915ce1e77f5dc9c639a5ef66ca45896"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "4a028ee035e70d1e0ccf51c5a5588c9b8a6f78c9"

STR:

1. Deploy cluster with Ceph backeng for glance.
2. Set 'expiration' option in /etc/keystone/keystone.conf to '30'
3. Set 'token_cache_time' in /etc/glance/glance-registry.conf to '-1'
4. Set 'data_api' in /etc/glance/glance-api.conf to 'glance.db.registry.api'.
5. Restart glance services and apache
6. Go to controller node and create large file:
# fallocate -l 10G image.img
7. Try to upload this file to glance:
#glance image-create --name 'test' --disk-format raw --container-format bare --progress --file image.img
2. Perform step 7 again with glance v1 api
# glance --os-image-api-version 1 image-create --name 'test' --disk-format raw --container-format bare --progress --file image.img

Sergey, thank you very much!

All,
Is there an approximate time frame that this fix will be released in 7.0 updates?
Regards
Laukik