Bug #1367771 “glance-manage db load_metadefs will fail if DB is ...” : Bugs : Glance

Pawel Koniszewski (pawel-koniszewski) on 2014-09-10

Changed in glance:
status:	New → In Progress
assignee:	nobody → Pawel Koniszewski (pawel-koniszewski)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-09-10: Fix proposed to glance (master)

#1

Fix proposed to branch: master
Review: https://review.openstack.org/120432

Pawel Skowron (pawel-skowron) on 2014-09-16

tags:

added: metadef

Bartosz Fic (bartosz-fic) on 2014-09-30

Changed in glance:
assignee:	Pawel Koniszewski (pawel-koniszewski) → Bartosz Fic (bartosz-fic)

Pawel Koniszewski (pawel-koniszewski) on 2014-10-01

Changed in glance:
assignee:	Bartosz Fic (bartosz-fic) → Pawel Koniszewski (pawel-koniszewski)

Thierry Carrez (ttx) on 2014-10-06

Changed in glance:
milestone:	none → juno-rc2
importance:	Undecided → High

Thierry Carrez (ttx) on 2014-10-07

Changed in glance:
milestone:	juno-rc2 → none
tags:	added: juno-rc-potential

Thierry Carrez (ttx) on 2014-10-08

Changed in glance:
milestone:	none → juno-rc2

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-10-08: Fix merged to glance (master)

#2

Reviewed: https://review.openstack.org/120432
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=89c04904416270d3c306d430f443a7127c5fc206
Submitter: Jenkins
Branch: master

commit 89c04904416270d3c306d430f443a7127c5fc206
Author: Bartosz Fic <email address hidden>
Date: Tue Oct 7 18:13:51 2014 +0200

Use ID for namespace generated by DB

    In current implementation ID that is used in namespace to
    insert data to DB is generated by built-in function - enumerate.
    This causes problems with loading the metadata when there are already
    namespaces in DB.

This patch removes 'enumerate' and asks for namespace ID
generated by database.

    Change-Id: I235c6310077526cafb898ac007c3601b4d66c9fe
    Closes-Bug: #1367771
    Co-Authored-By: Bartosz Fic <email address hidden>
    Co-Authored-By: Pawel Koniszewski <email address hidden>

Changed in glance:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-10-08: Fix proposed to glance (proposed/juno)

#3

Fix proposed to branch: proposed/juno
Review: https://review.openstack.org/126856

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-10-08: Fix merged to glance (proposed/juno)

#4

Reviewed: https://review.openstack.org/126856
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=76c66343a45cba0068c97d1ad21b46c63977e13d
Submitter: Jenkins
Branch: proposed/juno

commit 76c66343a45cba0068c97d1ad21b46c63977e13d
Author: Bartosz Fic <email address hidden>
Date: Tue Oct 7 18:13:51 2014 +0200

Use ID for namespace generated by DB

    In current implementation ID that is used in namespace to
    insert data to DB is generated by built-in function - enumerate.
    This causes problems with loading the metadata when there are already
    namespaces in DB.

This patch removes 'enumerate' and asks for namespace ID
generated by database.

    Closes-Bug: #1367771
    Co-Authored-By: Bartosz Fic <email address hidden>
    Co-Authored-By: Pawel Koniszewski <email address hidden>
    (cherry picked from commit 89c04904416270d3c306d430f443a7127c5fc206)

Conflicts:
glance/db/sqlalchemy/metadata.py

Change-Id: I235c6310077526cafb898ac007c3601b4d66c9fe

Changed in glance:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2014-10-16

Changed in glance:
milestone:	juno-rc2 → 2014.2

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-10-16: Fix proposed to glance (master)

#5

Fix proposed to branch: master
Review: https://review.openstack.org/128928

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-10-29: Fix merged to glance (master)

#6

Download full text (14.4 KiB)

Reviewed: https://review.openstack.org/128928
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=858cd9d4fdf32b4cc52e02ec771d01bfb463aa77
Submitter: Jenkins
Branch: master

commit 96e28428655aa7122ed74b045ff1bda1984255b1
Author: Nikhil Komawar <email address hidden>
Date: Tue Oct 14 13:09:48 2014 -0400

Fix options and their groups - etc/glance-api.conf

As per the docs at [0] , some of the options should have been moved
around in the etc/glance-api.conf. This patch changes the conf file to:

        1. indicate new default values
        2. change the group of some of the configs in order to adhere to
           new groups as expected by the deployer.
        3. deprecated configs have been removed or replaced with new ones.

[0] http://docs.openstack.org/trunk/config-reference/content/glance-conf-changes-master.html

Fixes bug: 1380689

Change-Id: I5b5ab96b050b502007e6660a7a613e252404d4e8

commit 9b176a278116849c8f7b7f4d9a987f37ec52779c
Author: Andy McCrae <email address hidden>
Date: Sat Oct 11 20:56:36 2014 +0100

Adjust authentication.rst doc to reference "identity_uri"

The "auth_port", "auth_host", and "auth_protocol" variables were
deprecated in favour of a single "identity_uri" variable.

* Adjust authentication.rst doc to reference "identity_uri"

Change-Id: I48de53f21b8d767b276858ed274066015d765f0e
Closes-Bug: #1361613

commit 08f83f543bc992ae8f2787fb405e58c33dadba73
Author: Jamie Lennox <email address hidden>
Date: Wed Aug 6 18:24:05 2014 +1000

Use identity_uri instead of older fragments

This has been the default option in middleware for a while now and we
should recommend the default options.

Change-Id: Ief347c897cf15ab4101936a56404e3a378021b15
(cherry picked from commit e7110a9c6e0119b3d0c6f5cdb3b9675a82b76039)

commit c0d90a580f87dbbf71e3a5d5c1b5cf8d7c7245b2
Author: Stuart McLaren <email address hidden>
Date: Wed Jul 16 13:33:32 2014 +0000

Prevent setting swift+config locations

    Forbid setting 'swift+config' locations in a similar
    manner to 'file' for security reasons; knowledge of
    the reference name should not be exploitable.

    Setting swift+config had been prevented when swift
    was the default store, this patch changes to forbid
    setting no matter which store is the default.

As with change id I75af34145521f533dcd6f5fd7690f5a68f3b44b3
this is v1 only for now.

Change-Id: I62c4980bd5c2f3dd77fc40cd007bc1067eca63a4
Closes-bug: 1334196

commit f259cac74d3e988b4012dcc2abd30091df27f5ce
Author: Wayne Okuma <email address hidden>
Date: Wed Oct 8 08:17:20 2014 -0700

Metadef schema column name is a reserved word in MySQL

    The metadef_properties and metadef_objects tables both have
    a column named schema. Unfortunately, schema is a reserved word
    in some relational database products, including MySQL and PostgreSQL.
    The metadef_properties.schema and metadef_objects.schema
    columns should be renamed to a non reserved word.

    Conflicts:
     glance/db/sqlalchemy/metadata.py
     glance/tests/unit/test_m...

Reviewed:  https://review.openstack.org/128928
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=858cd9d4fdf32b4cc52e02ec771d01bfb463aa77
Submitter: Jenkins
Branch:    master

commit 96e28428655aa7122ed74b045ff1bda1984255b1
Author: Nikhil Komawar <nikhilskomawar@gmail.com>
Date:   Tue Oct 14 13:09:48 2014 -0400

Fix options and their groups - etc/glance-api.conf
    
    As per the docs at [0] , some of the options should have been moved
    around in the etc/glance-api.conf. This patch changes the conf file to:
    
        1. indicate new default values
        2. change the group of some of the configs in order to adhere to
           new groups as expected by the deployer.
        3. deprecated configs have been removed or replaced with new ones.
    
    [0] http://docs.openstack.org/trunk/config-reference/content/glance-conf-changes-master.html
    
    Fixes bug: 1380689
    
    Change-Id: I5b5ab96b050b502007e6660a7a613e252404d4e8

commit 9b176a278116849c8f7b7f4d9a987f37ec52779c
Author: Andy McCrae <andy.mccrae@gmail.com>
Date:   Sat Oct 11 20:56:36 2014 +0100

Adjust authentication.rst doc to reference "identity_uri"
    
    The "auth_port", "auth_host", and "auth_protocol" variables were
    deprecated in favour of a single "identity_uri" variable.
    
    * Adjust authentication.rst doc to reference "identity_uri"
    
    Change-Id: I48de53f21b8d767b276858ed274066015d765f0e
    Closes-Bug: #1361613

commit 08f83f543bc992ae8f2787fb405e58c33dadba73
Author: Jamie Lennox <jamielennox@redhat.com>
Date:   Wed Aug 6 18:24:05 2014 +1000

Use identity_uri instead of older fragments
    
    This has been the default option in middleware for a while now and we
    should recommend the default options.
    
    Change-Id: Ief347c897cf15ab4101936a56404e3a378021b15
    (cherry picked from commit e7110a9c6e0119b3d0c6f5cdb3b9675a82b76039)

commit c0d90a580f87dbbf71e3a5d5c1b5cf8d7c7245b2
Author: Stuart McLaren <stuart.mclaren@hp.com>
Date:   Wed Jul 16 13:33:32 2014 +0000

Prevent setting swift+config locations
    
    Forbid setting 'swift+config' locations in a similar
    manner to 'file' for security reasons; knowledge of
    the reference name should not be exploitable.
    
    Setting swift+config had been prevented when swift
    was the default store, this patch changes to forbid
    setting no matter which store is the default.
    
    As with change id I75af34145521f533dcd6f5fd7690f5a68f3b44b3
    this is v1 only for now.
    
    Change-Id: I62c4980bd5c2f3dd77fc40cd007bc1067eca63a4
    Closes-bug: 1334196

commit f259cac74d3e988b4012dcc2abd30091df27f5ce
Author: Wayne Okuma <wayne.okuma@hp.com>
Date:   Wed Oct 8 08:17:20 2014 -0700

Metadef schema column name is a reserved word in MySQL
    
    The metadef_properties and metadef_objects tables both have
    a column named schema. Unfortunately, schema is a reserved word
    in some relational database products, including MySQL and PostgreSQL.
    The metadef_properties.schema and metadef_objects.schema
    columns should be renamed to a non reserved word.
    
    Conflicts:
    	glance/db/sqlalchemy/metadata.py
    	glance/tests/unit/test_migrations.py
    
    Change-Id: I9c1b497d2b09b9282a83bd8c19c32edfa4dd159f
    Closes-Bug: 1378968

commit fdcc5c4519bd71b9912ad6c0c7562ef51e774523
Author: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Date:   Fri Sep 12 20:02:27 2014 +0800

Remove stale chunks when failed to update image to registry
    
    For image v1 api, the operation of uploading image bits to store and
    the operation of updating image metadata are separaed, so when the
    second operation failed, the image stale chunks will be leaked in store.
    This change added a logic to remove those chunks if an exception
    happenned during the step of image metadata updating. And if
    glance-registry could be reached still, the image status will be changed
    to 'killed', this followed standard glance image status transition.
    
    Closes-bug: 1254497
    Change-Id: I01e13066b48a8feb1ead0de64992e7997feafdea
    Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>

commit 6b0c7ed29ec1e716e704e19112d9ee24553faf19
Author: Bartosz Fic <bartosz.fic@intel.com>
Date:   Wed Oct 8 10:47:31 2014 +0200

GET property which name includes resource type prefix
    
    Currently GET call to API to retrieve property details ends with
    404 error when property name includes resource type prefix.
    This patch extends show method to take filters as a parameter.
    If 'resource_type' is included in filters then the prefix of included
    resource type is removed from property name. This enables user to look
    for property name starting with prefix that comes from associated
    resource type.
    
    Change-Id: I3c4d96fbc9ce15016631017bf76089c338ac3cdc
    Closes-Bug: #1367564
    DocImpact
    Co-Authored-By: Bartosz Fic <bartosz.fic@intel.com>
    Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>

commit ace6197bbaaba7d53c05ff6eb2c39a642cfc463d
Author: abhishekkekane <abhishek.kekane@nttdata.com>
Date:   Wed Sep 24 04:36:13 2014 -0700

g-api raises 500 error while uploading image
    
    If both filesystem_store_datadirs and filesystem_store_datadir parameters
    specified in glance-api.conf file then while creating new image 500
    internal server error is raised.
    
    Caught StoreAddDisabled exception for v1 and v2 api while uploading
    image and raised HTTPGone exception to return HTTP 410 response to
    user.
    
    Closes-Bug: #1372888
    Change-Id: Iccbf5b88634d3c956d41e8d8a0126648c64b34eb

commit ad34b836fb6d8e2ba067e8c14512abc4ec0b4969
Author: Rajesh Tailor <rajesh.tailor@nttdata.com>
Date:   Fri Sep 26 07:59:40 2014 -0700

Fix for Adopt glance.store library in Glance
    
    The store module is removed from glance project and new glance_store
    module is created, but the glance project code was not updated
    properly for the required changes.
    
    The exception (UnknownScheme) was moved to glance_store, but in
    glance image-create api (v1), it still refers to
    exception.UnknownScheme in get_store_or_400 method.
    
    Change-Id: I09e78b92d2d44a659412361b4f03b4a988714bf5
    Closes-Bug: 1373454

commit 9eff67b8fff812acdbf787f2dc119761f3cf3153
Author: Travis Tripp <travis.tripp@hp.com>
Date:   Tue Sep 16 15:42:16 2014 -0600

Update Metadefs associated with ImagePropertiesFilter
    
    From: http://docs.openstack.org/trunk/config-reference/content/section_compute-scheduler.html#imagepropertiesfilter
    
    Filters hosts based on properties defined on the instance's image.
    
    Updated descriptions to be provide better information.
    Added baremetal "hypervisor_type" after getting info from
    Nova core about what hypervisors to include.
    
    Left out "architecture" for now, because horizon fails when
    trying to update architecture property.
    
    Change-Id: I9195825a0ab4b89e764983da06827173a2daa23c
    Closes-bug: 1368965
    (cherry picked from commit 236c9db4eb119dfaa6a592097b97e7b3f623a506)

commit 00f030e34608a7311fea9fe8383e55871c13b5e7
Author: Nikhil Komawar <nikhil.komawar@rackspace.com>
Date:   Wed Oct 8 14:21:41 2014 +0000

updated translations
    
    Commands run:-
    $ python setup.py extract_messages
    $ python setup.py update_catalog --no-fuzzy-matching \
      --ignore-obsolete=true
    $ source \
      ../openstack-infra/project-config/jenkins/scripts/common_translation_update.sh
    $ setup_loglevel_vars
    $ cleanup_po_files glance
    
    Change-Id: If1bdff3f7330c3eb58bd56b08299472ae3d4b552

commit 76c66343a45cba0068c97d1ad21b46c63977e13d
Author: Bartosz Fic <bartosz.fic@intel.com>
Date:   Tue Oct 7 18:13:51 2014 +0200

Use ID for namespace generated by DB
    
    In current implementation ID that is used in namespace to
    insert data to DB is generated by built-in function - enumerate.
    This causes problems with loading the metadata when there are already
    namespaces in DB.
    
    This patch removes 'enumerate' and asks for namespace ID
    generated by database.
    
    Closes-Bug: #1367771
    Co-Authored-By: Bartosz Fic <bartosz.fic@intel.com>
    Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
    (cherry picked from commit 89c04904416270d3c306d430f443a7127c5fc206)
    
    Conflicts:
    	glance/db/sqlalchemy/metadata.py
    
    Change-Id: I235c6310077526cafb898ac007c3601b4d66c9fe

commit da93f408dde9652a3f5e2daaa534852576b8f6f2
Author: Wayne Okuma <wayne.okuma@hp.com>
Date:   Thu Sep 11 13:59:29 2014 -0700

Metadef Property and Object schema columns should use JSONEncodedDict
    
    The MetadefProperty and MetadefObject ORM classes currently specify the
    JSON schema columns as type Text. It is preferred to use the
    JSONEncodedDict Type Decorator instead. This fix also includes necessary
    code changes to remove JSON encoding/decoding that was previously done
    in other layers. Fixes for unit tests involving the schema columns are
    also included.
    
    Closes-Bug: 1368479
    
    Conflicts:
    	glance/db/__init__.py
    	glance/db/sqlalchemy/models_metadef.py
    
    Change-Id: I2c574210f8d62c77a438afab83ff80f3e5bd2fe7
    (cherry picked from commit 824d9620b0b90483baf45981d2cb328855943e06)

commit cecc9497c158fbf43c26aa8943a41b4e6fcc5fed
Author: Travis Tripp <travis.tripp@hp.com>
Date:   Fri Sep 12 11:55:52 2014 -0600

Add missing metadefs for shutdown behavior
    
    The following Nova patch adds support for graceful shutdown
    of a guest VM and allows setting timeout properties on images.
    The properties should be updated in the Metadata Definitions catalog.
    
    https://review.openstack.org/#/c/68942/
    
    Change-Id: I58145d9d0114b3932b63263ea123c4662146d14b
    Closes-bug: 1368036
    (cherry picked from commit 5fcb3aa2e35e9af17cb8be9e24c6613626036f2b)

commit eec47084783e931248690a22c143af6f9030b293
Author: Travis Tripp <travis.tripp@hp.com>
Date:   Wed Sep 17 17:30:03 2014 -0600

Update driver metadata definitions to Juno
    
    vmware and libvirt support different hw_vif_model settings.
    This patch updates them so that each namespace can specify
    the models they support.
    
    vmware api is updated with the vmware_disktype
    
    Change-Id: Iec5901097c9621a052a930b99d5cbe7872d4f3ff
    Closes-bug: 1370767
    (cherry picked from commit ebafdbeef6420d0fcc4922f245956096ca9e50b3)

commit 0f3b518028196b5c8c36b378928dae31c2c4a6fa
Author: Kamil Rykowski <kamil.rykowski@intel.com>
Date:   Wed Sep 24 14:15:59 2014 +0200

Mark custom properties in image schema as non-base
    
    Currently it is impossible to determine if given image schema property
    is base or custom one and knowledge of that can be handy in some
    situations.    Proposed change appends to every custom property special
    key which determiness that it is not a base property.
    
    Change-Id: I49255255df311036d516768afc55475c1f9aad47
    Partial-Bug: #1371559
    (cherry picked from commit 94c05cbdbb3a78b3df4df8d522555f34d2f0a166)

commit be38189310e38a18f841f27808d469e01ba7f696
Author: Wayne Okuma <wayne.okuma@hp.com>
Date:   Tue Sep 9 20:46:47 2014 -0700

Specify the MetadefNamespace.namespace column is not nullable
    
    The metadef_namespaces table definition indicates the namespace
    column as not accepting nulls. The related MetadefNamespace ORM class
    should also indicate that the namespace column does not accept nulls
    with "nullable=False" in the column definition.
    
    Change-Id: I681747b6579a2284f23bf154889a61c639b0616d
    Closes-Bug: 1367619
    (cherry picked from commit 10e858d57be5e04615f390b3060c8aeadaeed954)

commit 3c0ff8d94562bf42d4092db0aaad04fb5cc93fe3
Author: Pawel Koniszewski <pawel.koniszewski@intel.com>
Date:   Mon Sep 22 02:55:26 2014 -0400

Make compute-trust.json compatible with TrustFilter
    
    Current properties inside compute-trust.json does not match
    with how TrustFilter in nova works. JSON provides True/False
    boolean values but TrustFilter expects trusted/untrusted/unknown
    string values. This patch repairs compute-trust.json to be
    compatible with TrustFilter.
    
    Change-Id: I26965a549daf9340621b0f18a1b845b39bac4bd8
    Closes-Bug: #1369581
    (cherry picked from commit 39e90f29d93f991d95092de5f93a239be1b3ca3b)

commit 60ff6fd0efb6615b9df3adbd7913f1a3fa615acf
Author: Travis Tripp <travis.tripp@hp.com>
Date:   Tue Sep 9 15:26:13 2014 -0600

Include Metadata Defs Concepts in Dev Docs
    
    The http://docs.openstack.org/developer/glance/
    site currently doesn't include the Juno Metadata
    Definitions concepts.  This patch adds
    an overview of the concepts to this site.
    
    This provides a synopis of the concepts in:
    https://github.com/openstack/glance-specs/blob/master/specs/juno/metadata-schema-catalog.rst
    
    DocImpact
    Closes-Bug: 1367432
    Related-Bug: 1367908
    Related-Bug: 1363615
    Related-Bug: 1366286
    Related-Bug: 1363383
    Change-Id: Iad4d388cbbf2f63fa243d04d35032de0cb0bc1b4
    (cherry picked from commit 0aab5e271667990afb1a2522c7c9c61fa7211e0b)

commit 5601f7fcdc4482ba27908e156e350095be1e0805
Author: Travis Tripp <travis.tripp@hp.com>
Date:   Wed Sep 10 19:05:19 2014 -0600

Nova instance config drive Metadata Definition
    
    A nova Juno FFE landed to support setting the
    img_config_drive property on images to require images
    to be booted with a config drive. The Glance Metadata
    Definitions should include this property.
    
    See Nova Blueprint:
    https://blueprints.launchpad.net/nova/+spec/config-drive-image-property
    
    Change-Id: Iffc4d4f00f3677ed1fa41233ef6cfbc5c46d8602
    Closes-bug: 1367981
    (cherry picked from commit 0dd620c7bd7f07935e1cea5d0bfab2a41f754d1b)

commit 94eac77743635face04ee5bad69d57d2f7e97eb6
Author: Travis Tripp <travis.tripp@hp.com>
Date:   Wed Sep 10 23:24:36 2014 -0600

Add missing metadefs for Aggregate Filters
    
    The below spec implemented in Juno added numerous properties
    that can be added to Host Aggregates. The Metadata
    Definitions catalog should include them.
    
    https://github.com/openstack/nova-specs/blob/master/specs/juno/per-aggregate-filters.rst
    
    Change-Id: Id37028b4b24ab5f6b0c4547a3c1af47bd76e61f6
    Closes-bug: 1368032
    (cherry picked from commit 313d4f272cbeabfdcb39de4bcb441749f56fb4d4)

commit 91439069a73af0c105706c949291272422471247
Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>
Date:   Mon Oct 6 16:09:16 2014 +0000

Updated from global requirements
    
    Change-Id: I32c8167913b9412120d1256ffddf9117f765551f

Glance

glance-manage db load_metadefs will fail if DB is not empty

Bug Description

Other bug subscribers

Remote bug watches