OpenStack Heat

Outputs aren't correctly validated

Bug #1599114 reported by Steven Hardy on 2016-07-05

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Heat	Triaged	Low	Unassigned	OpenStack Heat next

Bug Description

We've had bug(s) about this before IIRC, but it's still not working correctly, see reproducer below where an output references a wrong parameter - the stack goes CREATE_COMPLETE, but then resolving the outputs fails.

This is more of a problem than it seems at first glance, because although the stack-show error output is pretty clear, it's not reflected clearly at all when this happens in a nested template - some parent resource that references the nested stack output randomly fails and you have to manually inspect the nested stack to see the cause of the error - it'd be *much* clearer and easier to debug if the nested stack just failed.

If it's not possible to validate this during the validate stage due to dependencies on runtime data (I suspect this may be the case), we should special-case this for TemplateResource such that we resolve all attributes/outputs before declaring the parent resource COMPLETE.

[stack@instack openstack-tripleo-heat-templates]$ cat /tmp/reproducer.yaml
heat_template_version: 2016-04-08

parameters:
  SaharaWorkers:
    default: 0
    description: The number of workers for the sahara-api.
    type: number
outputs:
  role_data:
    description: Role data for the Sahara API role.
    value:
      config_settings:
        sahara::service::api::api_workers: {get_param: SaharaApiWorkers}

[stack@instack openstack-tripleo-heat-templates]$ heat stack-show r1
...
| outputs | [ |
| | { |
| | "output_value": null, |
| | "output_error": "The Parameter (SaharaApiWorkers) was not provided.", |
| | "output_key": "role_data", |
| | "description": "Role data for the Sahara API role." |
| | }

Tags:

Oleksii Chuprykov (ochuprykov) on 2016-07-06

Changed in heat:
assignee:	nobody → Oleksii Chuprykov (ochuprykov)
status:	New → Confirmed

Oleksii Chuprykov (ochuprykov) on 2016-07-13

Changed in heat:
milestone:	none → newton-3
importance:	Undecided → Medium

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-07-14: Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/342124

Changed in heat:
status:	Confirmed → In Progress

Revision history for this message

Lars Kellogg-Stedman (larsks) wrote on 2016-08-12:

There is another problem that is not addressed by the current review.

Start with this in a file named `parent.yaml`:

heat_template_version: 2016-04-08

description: Fluentd logging configuration

    parameters:
      ParamA:
        type: string
        default: "this is a test"

    resources:
      TopLevelResource:
        type: ./child-1.yaml

    outputs:
      role_data:
        value:
          config_settings:
            map_merge:
              - get_attr: [TopLevelResource, role_data, config_settings]

Place this in `child-1.yaml`:

heat_template_version: 2016-04-08

description: Fluentd logging configuration

    parameters:
      ParamA:
        type: string
        default: "this is a test"

    resources:
      IntermediateResource:
        type: ./child-2.yaml

    outputs:
      role_data:
        value:
          config_settings:
            map_merge:
              - get_attr: [IntermediateResource, role_data, config_settings]

Place this in `child-2.yaml`:

heat_template_version: 2016-04-08

description: Fluentd logging configuration

    parameters:
      ParamA:
        type: string
        default: "this is a test"

Note that `child-2.yaml` stack has no outputs, although we ask for one
in the `child-1.yaml` stack.

Deploy a new stack from `parent.yaml`.

The stack will go to state `CREATE_COMPLETE`. Running `stack-show` on
this stack will reveal:

    [
      {
        "output_value": null,
        "output_error": "The Referenced Attribute (TopLevelResource role_data) is incorrect.",
        "output_key": "role_data",
        "description": "No description given"
      }
    ]

Note that the error is reported against `TopLevelResource`, even
though the actual problem is in the `child-2.yaml` stack. This makes
the error extremely hard to diagnose, especially in environments with
many nested stacks.

Revision history for this message

Oleksii Chuprykov (ochuprykov) wrote on 2016-08-13:

That patch is partial fix. It adds "static" validation, i.e. validation before starting creation of the stack. Some cases of invilid outputs can be determined only after creation of the stack. In your case you use 'get_attr' that resolves during creation/update and should be handled differently. I assumed one more patch for adding "dynamic" validation.

Thomas Herve (therve) on 2016-09-01

Changed in heat:
milestone:	newton-3 → next

Revision history for this message

Steven Hardy (shardy) wrote on 2016-09-16:

Oleksii - are you still working on this? We keep getting folks confused due to this issue with TripleO templates, so if you're not planning to pick it up, can I take a look and revive your previous patch?

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-16:

Fix proposed to branch: master
Review: https://review.openstack.org/371704

Revision history for this message

Oleksii Chuprykov (ochuprykov) wrote on 2016-09-16:

Hi, Steven. I'm going to work on this bug. But if you need this bug solved for "yesterday", feel free to take it. I uploaded a WIP patch few minutes ago, though. Welcome to review.

Revision history for this message

Steven Hardy (shardy) wrote on 2016-09-16:

Oleksii - thanks for the update - ideally I'd like to get this fixed for the final Newton release (or at least have a patch up we can backport to stable/newton soon after, I'll check out your patch and see if I can help speed things up with some reviews.

Sergey Kraynev (skraynev) on 2016-09-27

Changed in heat:
milestone:	next → ocata-1

Revision history for this message

Zane Bitter (zaneb) wrote on 2016-10-13:

I think the real problem is that we raise InvalidTemplateAttribute in StackResource regardless of whether the error is the key not existing or an error resolving the output:

http://git.openstack.org/cgit/openstack/heat/tree/heat/engine/resources/stack_resource.py?h=stable%2Fnewton#n593

In the latter case that's clearly the wrong exception. (a) because InvalidTemplateAttribute means that you accessed an attribute that doesn't exist, so it tells the user the wrong thing, and (b) because InvalidTemplateAttribute doesn't allow you to specify a reason, so the error message gets lost. If we raised a useful exception containing the error message there then the user would get the right feedback without any change in behaviour.

Rabi Mishra (rabi) on 2016-11-17

Changed in heat:
milestone:	ocata-1 → ocata-2

Rabi Mishra (rabi) on 2016-12-14

Changed in heat:
milestone:	ocata-2 → ocata-3

Rabi Mishra (rabi) on 2017-01-25

Changed in heat:
milestone:	ocata-3 → ocata-rc1

Rabi Mishra (rabi) on 2017-02-01

Changed in heat:
milestone:	ocata-rc1 → pike-1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-02-20:

Fix proposed to branch: master
Review: https://review.openstack.org/436179

Changed in heat:
assignee:	Oleksii Chuprykov (ochuprykov) → Zane Bitter (zaneb)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-02-24: Fix merged to heat (master)

#10

Reviewed: https://review.openstack.org/436179
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=b90991e00b4e0029748c28c5258607f266f1bf85
Submitter: Jenkins
Branch: master

commit b90991e00b4e0029748c28c5258607f266f1bf85
Author: Zane Bitter <email address hidden>
Date: Mon Feb 20 15:12:49 2017 -0500

Pass on outputs errors to parent stacks

    If getting an output from a child stack fails with an error, we didn't pass
    on the error message to the parent stack that was requesting it but instead
    reported essentially that the given output did not exist.

Change-Id: I5653baf310a29dc4829ad570c769cf67ce12695e
Partial-Bug: #1599114

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-03: Fix proposed to heat (stable/ocata)

#11

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/441445

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-21: Fix merged to heat (stable/ocata)

#12

Reviewed: https://review.openstack.org/441445
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=34d46d90a816f2811d203ddb5233ed3fd3d3a434
Submitter: Jenkins
Branch: stable/ocata

commit 34d46d90a816f2811d203ddb5233ed3fd3d3a434
Author: Zane Bitter <email address hidden>
Date: Mon Feb 20 15:12:49 2017 -0500

Pass on outputs errors to parent stacks

    Change-Id: I5653baf310a29dc4829ad570c769cf67ce12695e
    Partial-Bug: #1599114
    (cherry picked from commit b90991e00b4e0029748c28c5258607f266f1bf85)

tags:

added: in-stable-ocata

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-22: Fix proposed to heat (stable/newton)

#13

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/448589

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-03-30: Fix merged to heat (stable/newton)

#14

Reviewed: https://review.openstack.org/448589
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=e5cd34402572c73c701357deb2c69d72d589b781
Submitter: Jenkins
Branch: stable/newton

commit e5cd34402572c73c701357deb2c69d72d589b781
Author: Zane Bitter <email address hidden>
Date: Mon Feb 20 15:12:49 2017 -0500

Pass on outputs errors to parent stacks

    Change-Id: I5653baf310a29dc4829ad570c769cf67ce12695e
    Partial-Bug: #1599114
    (cherry picked from commit b90991e00b4e0029748c28c5258607f266f1bf85)

tags:

added: in-stable-newton

Rico Lin (rico-lin) on 2017-04-11

Changed in heat:
milestone:	pike-1 → pike-2

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-05-04: Change abandoned on heat (master)

#15

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/342124
Reason: Just a test patch, not going to be final solution

Rico Lin (rico-lin) on 2017-06-07

Changed in heat:
milestone:	pike-2 → pike-3

Rico Lin (rico-lin) on 2017-07-31

Changed in heat:
milestone:	pike-3 → pike-rc1

Zane Bitter (zaneb) on 2017-07-31

Changed in heat:
assignee:	Zane Bitter (zaneb) → nobody
milestone:	pike-rc1 → next
status:	In Progress → Triaged
importance:	Medium → Low

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-03-06:

#16

Change abandoned by Zane Bitter (<email address hidden>) on branch: master
Review: https://review.openstack.org/342124
Reason: Let's not do this.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-10-22:

#17

Change abandoned by Zane Bitter (<email address hidden>) on branch: master
Review: https://review.opendev.org/371704
Reason: Bug was fixed in I5653baf310a29dc4829ad570c769cf67ce12695e. Nobody has shown further interest in this approach, so abandoning.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.