Ad-hoc actions cannot reference env() in YAQL/Jinja expressions

Bug #1690158 reported by Bob Haddleton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mistral
Fix Released
Medium
Bob Haddleton

Bug Description

Ad-hoc actions cannot reference env() in YAQL and Jinja expressions.

The AdHocAction methods _prepare_input() and _prepare_output() call expr.evaluate_recursively() but they don't have access to the environment data that was passed in to the workflow, since the Action does not have access to the wf_ex.context.

Sample workbook:

version: "2.0"

name: TestEnv

actions:
  print_env_yaql_msg:
    input:
      - message: bar
    base: std.echo
    base-input:
      output: "<% $.message %>: <% env().foo %>"

  print_env_jinja_msg:
    input:
      - message: bar
    base: std.echo
    base-input:
      output: '{{ env().foo }}'

workflows:
  dump_environment_broken:
    type: direct
    input:
      - dummy: null
    tasks:
      print_env_yaql_bad:
        action: print_env_yaql_msg

      print_env_jinja_bad:
        action: print_env_jinja_msg

Executing this workflow produces the following exception:

mistral execution-create TestEnv.dump_environment_broken '{"dummy": "foobar"}' '{"env": {"foo": "bar"}}'

+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID | 4a2ddd7c-409d-4162-902a-f17f6f3505a2 |
| Workflow ID | f734a15b-c471-4cfc-b0fb-7623a89f269f |
| Workflow name | TestEnv.dump_environment_broken |
| Description | |
| Task Execution ID | <none> |
| State | ERROR |
| State info | Failed to run task [error=Can not evaluate YAQL expression [expression=env().foo, error=Unknown function "#property#foo", data={u'message': u'foo'}], wf=WorkflowExecution |
| | {'state_info': None, 'accepted': False, 'output': {}, 'name': u'TestEnv.dump_environment_broken', 'workflow_name': u'TestEnv.dump_environment_broken', 'state': |
| | 'RUNNING', 'created_at': '2017-05-15 00:27:17', 'runtime_context': {'index': 0}, 'updated_at': '2017-05-15 00:27:17', 'spec': {u'input': [{u'dummy': None}], u'tasks': |
| | {u'print_env_jinja_bad': {u'action': u'print_env_jinja_msg', u'version': '2.0', u'type': u'direct', u'name': u'print_env_jinja_bad'}, u'print_env_yaql_bad': {u'action': |
| | u'print_env_yaql_msg', u'version': '2.0', u'type': u'direct', u'name': u'print_env_yaql_bad'}}, u'type': u'direct', u'name': u'dump_environment_broken', u'version': |
| | u'2.0'}, 'workflow_id': u'f734a15b-c471-4cfc-b0fb-7623a89f269f', 'params': {u'env': {u'foo': u'bar'}}, 'context': {'__env': {u'foo': u'bar'}, '__execution': {'id': '4a2ddd7c- |
| | 409d-4162-902a-f17f6f3505a2'}}, 'input': {u'dummy': u'foobar'}, 'scope': 'private', 'project_id': '<default-project>', 'task_execution_id': None, 'id': '4a2ddd7c-409d-4162 |
| | -902a-f17f6f3505a2', 'description': u''}, task=print_env_yaql_bad]: |
| | Traceback (most recent call last): |
| | File "/root/git/mistral/mistral/engine/task_handler.py", line 58, in run_task |
| | task.run() |
| | File "/root/git/mistral/.tox/py27/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper |
| | return f(*args, **kwargs) |
| | File "/root/git/mistral/mistral/engine/tasks.py", line 269, in run |
| | self._run_new() |
| | File "/root/git/mistral/.tox/py27/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper |
| | return f(*args, **kwargs) |
| | File "/root/git/mistral/mistral/engine/tasks.py", line 293, in _run_new |
| | self._schedule_actions() |
| | File "/root/git/mistral/mistral/engine/tasks.py", line 347, in _schedule_actions |
| | action.validate_input(input_dict) |
| | File "/root/git/mistral/mistral/engine/actions.py", line 352, in validate_input |
| | self._prepare_input(input_dict) |
| | File "/root/git/mistral/mistral/engine/actions.py", line 364, in _prepare_input |
| | base_input_dict |
| | File "/root/git/mistral/mistral/expressions/__init__.py", line 96, in evaluate_recursively |
| | data[key] = _evaluate_item(data[key], context) |
| | File "/root/git/mistral/mistral/expressions/__init__.py", line 79, in _evaluate_item |
| | return evaluate(item, context) |
| | File "/root/git/mistral/mistral/expressions/__init__.py", line 71, in evaluate |
| | return evaluator.evaluate(expression, context) |
| | File "/root/git/mistral/mistral/expressions/yaql_expression.py", line 110, in evaluate |
| | cls).evaluate(trim_expr, data_context) |
| | File "/root/git/mistral/mistral/expressions/yaql_expression.py", line 61, in evaluate |
| | ", data=%s]" % (expression, str(e), data_context) |
| | YaqlEvaluationException: Can not evaluate YAQL expression [expression=env().foo, error=Unknown function "#property#foo", data={u'message': u'foo'}] |
| | |
| Created at | 2017-05-15 00:27:17 |
| Updated at | 2017-05-15 00:27:17 |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Changed in mistral:
assignee: nobody → Bob Haddleton (bob-haddleton)
status: New → In Progress
description: updated
description: updated
Changed in mistral:
milestone: none → pike-2
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to mistral (master)

Reviewed: https://review.openstack.org/464677
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=a121aacbbcca0ee08832d387eafa2885de61f054
Submitter: Jenkins
Branch: master

commit a121aacbbcca0ee08832d387eafa2885de61f054
Author: Bob Haddleton <email address hidden>
Date: Mon May 15 10:40:59 2017 -0500

    Update AdHoc Actions to support context data references

    AdHoc actions can be defined using YAQL and Jinja2 expressions in the
    same way as Tasks, but they could not access the associated context
    data because the context was not available when the expression is
    evaluated. This patchset passes the task and workflow context into
    the AdHocAction object so that the inputs can be evaluated using the
    available context, and the context data will be available for
    reference.

    Added a test to verify that the env() works in AdHoc Actions.

    Change-Id: Ib95604d3d494a443e852bc7f5eee24f398b1648c
    Closes-Bug: 1690158

Changed in mistral:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/mistral 5.0.0.0b2

This issue was fixed in the openstack/mistral 5.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.