Murano with Glare backend does not work with HOT packages

Fix proposed to branch: master
Review: https://review.openstack.org/301812

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/304144

Reviewed: https://review.openstack.org/301812
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=28586897671e3aa2d9abcfb40e78d345436bc8a7
Submitter: Jenkins
Branch: master

commit 28586897671e3aa2d9abcfb40e78d345436bc8a7
Author: Alexander Tivelkov <email address hidden>
Date: Tue Apr 5 19:04:27 2016 +0300

    'GetPackageUI' API can now be called even if Glare is used

    Since Glare support was introduced the 'catalog/packages' part of
    Murano API (/v1/catalog/packages) became obsolete and was intended to
    be used for "legacy" (i.e. non-glare-based) package storage. The
    murano client was supposed to directly call glare to retrieve the
    packages.

    However, the "GetPackageUI" call (GET
    /v1/catalog/packages/%pkg_id%/ui) is different: not all the packages
    bundle the UI definitions, some require Murano to generate them based
    on the contents of the package. This is true for HOT-based packages
    and others, plugin-based ones. In future this will be true for 100% of
    the packages when the dynamic ui and object model generation is added.

    Because of this that API should be callable even if Glare is
    configured as a package storage backend. It should fetch the artifact
    from glare, unpack it and use whatever plugin-specific logic required
    to generate the UI definitions for this particular package.

    This patch introduces appropriate change while maintaining the
    backwards compatibility: in the legacy mode (i.e. when murano's DB
    acts as package storage) the UI is read directly from an appropriate
    DB table without fetching the whole package.

    This fixes the issue when the HOT-based packages could not be
    deplpoyed properly with murano-dashboard, but the murano-client has to
    be updated as well to properly call the modified API even if Glare is
    enabled.

    Change-Id: Id9327e4015e1c0a1553c0b1a0151a94ee4da2928
    Partial-bug: #1565805

Reviewed: https://review.openstack.org/304144
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=c81fc4b79a5054949b253a375b660c7de33f852f
Submitter: Jenkins
Branch: stable/mitaka

commit c81fc4b79a5054949b253a375b660c7de33f852f
Author: Alexander Tivelkov <email address hidden>
Date: Tue Apr 5 19:04:27 2016 +0300

    'GetPackageUI' API can now be called even if Glare is used

    Since Glare support was introduced the 'catalog/packages' part of
    Murano API (/v1/catalog/packages) became obsolete and was intended to
    be used for "legacy" (i.e. non-glare-based) package storage. The
    murano client was supposed to directly call glare to retrieve the
    packages.

    However, the "GetPackageUI" call (GET
    /v1/catalog/packages/%pkg_id%/ui) is different: not all the packages
    bundle the UI definitions, some require Murano to generate them based
    on the contents of the package. This is true for HOT-based packages
    and others, plugin-based ones. In future this will be true for 100% of
    the packages when the dynamic ui and object model generation is added.

    Because of this that API should be callable even if Glare is
    configured as a package storage backend. It should fetch the artifact
    from glare, unpack it and use whatever plugin-specific logic required
    to generate the UI definitions for this particular package.

    This patch introduces appropriate change while maintaining the
    backwards compatibility: in the legacy mode (i.e. when murano's DB
    acts as package storage) the UI is read directly from an appropriate
    DB table without fetching the whole package.

    This fixes the issue when the HOT-based packages could not be
    deplpoyed properly with murano-dashboard, but the murano-client has to
    be updated as well to properly call the modified API even if Glare is
    enabled.

    Change-Id: Id9327e4015e1c0a1553c0b1a0151a94ee4da2928
    Partial-bug: #1565805

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/320372

Change abandoned by Kirill Zaitsev (<email address hidden>) on branch: stable/liberty
Review: https://review.openstack.org/320372

Fix proposed to branch: master
Review: https://review.openstack.org/320540

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/320543

Fix proposed to branch: master
Review: https://review.openstack.org/320615

Reviewed: https://review.openstack.org/320540
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=623ebce812a3b4ec2f3396b44aaef6c58a140400
Submitter: Jenkins
Branch: master

commit 623ebce812a3b4ec2f3396b44aaef6c58a140400
Author: Alexander Tivelkov <email address hidden>
Date: Tue May 24 18:45:59 2016 +0300

    Forced flush of tmp file to disk on ui retrieval

    UI retrieval routine loads the package's temporary file from disk
    to properly render its ui. This procedure does not close the file
    before reading it, so the write buffers should be flushed first.

    This was causing the bug when the UI could not be properly fetched
    from the API.

    Change-Id: I53cd08945c826dd18a7308f9f007d50241bb9041
    Closes-bug: #1565805

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/321129

Reviewed: https://review.openstack.org/320615
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=afa85b3c2116403505c860ece2b2f0e781aad66c
Submitter: Jenkins
Branch: master

commit afa85b3c2116403505c860ece2b2f0e781aad66c
Author: Alexander Tivelkov <email address hidden>
Date: Tue May 24 21:26:36 2016 +0300

    Added package references to generated UIs

    Generated UIs of the non-muranoPL packages (HOT, cloudify_tosca and
    CSAR) reference the type of object model's object by class FQNs.
    However, when these packages are uploaded to glare, their class
    contents are not indexed since the client which uploads them has no
    knowledge of the generated classes.

    To workaround this issue it is proposed to reference objects not just
    by class FQNs but to include the package FQN and the version, so the
    engine does not have to rely on Glare's indexes.

    Change-Id: I6175e89b68bcdfc29d33ae3616d6ecba662f2509
    Partial-bug: #1565805

Reviewed: https://review.openstack.org/321129
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=1ff47503a1df6a22b576dec1552068e8b6f10edd
Submitter: Jenkins
Branch: stable/mitaka

commit 1ff47503a1df6a22b576dec1552068e8b6f10edd
Author: Alexander Tivelkov <email address hidden>
Date: Tue May 24 21:26:36 2016 +0300

    Added package references to generated UIs

    Generated UIs of the non-muranoPL packages (HOT, cloudify_tosca and
    CSAR) reference the type of object model's object by class FQNs.
    However, when these packages are uploaded to glare, their class
    contents are not indexed since the client which uploads them has no
    knowledge of the generated classes.

    To workaround this issue it is proposed to reference objects not just
    by class FQNs but to include the package FQN and the version, so the
    engine does not have to rely on Glare's indexes.

    Change-Id: I6175e89b68bcdfc29d33ae3616d6ecba662f2509
    Partial-bug: #1565805

Reviewed: https://review.openstack.org/320543
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=ca1f261ee2a12f7c1629c66f4bd03458e1f81bed
Submitter: Jenkins
Branch: stable/mitaka

commit ca1f261ee2a12f7c1629c66f4bd03458e1f81bed
Author: Alexander Tivelkov <email address hidden>
Date: Tue May 24 18:45:59 2016 +0300

    Forced flush of tmp file to disk on ui retrieval

    UI retrieval routine loads the package's temporary file from disk
    to properly render its ui. This procedure does not close the file
    before reading it, so the write buffers should be flushed first.

    This was causing the bug when the UI could not be properly fetched
    from the API.

    Change-Id: I53cd08945c826dd18a7308f9f007d50241bb9041
    Closes-bug: #1565805

This issue was fixed in the openstack/murano 3.0.0.0b1 development milestone.

This issue was fixed in the openstack/murano 2.0.1 release.