kolla_toolbox module sometimes ignores errors and changes

Fix proposed to branch: master
Review: https://review.opendev.org/682340

I think the situation is not quite as bad as I first thought. Modules sometimes produce JSON output, and we can catch this and parse the 'failed' and 'changed' fields. The problems I'm aware of are:

* modules like 'command' do not output in JSON format, so we can't detect whether they changed. We could detect their status, but currently do not.
* if we have an error like passing invalid module arguments, we do not get JSON format, and do not detect this error.

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/682344

Reviewed: https://review.opendev.org/682340
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=70b515bf1225e56b7df81677043d75be4bbb1ab4
Submitter: Zuul
Branch: master

commit 70b515bf1225e56b7df81677043d75be4bbb1ab4
Author: Mark Goddard <email address hidden>
Date: Mon Sep 16 11:27:48 2019 +0100

    Catch errors and changes in kolla_toolbox module

    The kolla_toolbox Ansible module executes as-hoc ansible commands in the
    kolla_toolbox container, and parses the output to make it look as if
    ansible-playbook executed the command. Currently however, this module
    sometimes fails to catch failures of the underlying command, and also
    sometimes shows tasks as 'ok' when the underlying command was changed.
    This has been tested both before and after the upgrade to ansible 2.8.

    This change fixes this issue by configuring ansible to emit output in
    JSON format, to make parsing simpler. We can now pick up errors and
    changes, and signal them to the caller.

    This change also adds an ansible playbook, tests/test-kolla-toolbox.yml,
    that can be executed to test the module. It's not currently integrated
    with any CI jobs.

    Note that this change cannot be backported as the JSON output callback
    plugin was added in Ansible 2.5.

    Change-Id: I8236dd4165f760c819ca972b75cbebc62015fada
    Closes-Bug: #1844114

Reviewed: https://review.opendev.org/682344
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=28273411a8fd76ff26b0507854efd0017dbb3520
Submitter: Zuul
Branch: stable/stein

commit 28273411a8fd76ff26b0507854efd0017dbb3520
Author: Mark Goddard <email address hidden>
Date: Mon Sep 16 12:30:23 2019 +0100

    Fix kolla_toolbox error handling

    Backport only.

    The kolla_toolbox Ansible module executes as-hoc ansible commands in the
    kolla_toolbox container, and parses the output to make it look as if
    ansible-playbook executed the command. Currently however, this module
    sometimes fails to catch failures of the underlying command, and also
    sometimes shows tasks as 'ok' when the underlying command was changed.

    This change introduces a check on the status field for non-JSON output,
    and assumes the module has changes on success.

    NOTE: The original patch on master
    (https://review.opendev.org/#/c/682340/) cannot be backported as it
    relies on Ansible 2.5+ in the kolla_toolbox container.

    Change-Id: Ib1e33f8e5c60d98b76a4eba9f405af89431d60bc
    Closes-Bug: 1844114

TASK [mariadb : Creating haproxy mysql user] ***************************************************************************************************************************************************
fatal: [10.5.5.108]: FAILED! => {"changed": false, "msg": "Can not parse the inner module output: localhost | SUCCESS => {\n \"changed\": false, \n \"user\": \"haproxy\"\n}\n"}

JK: As mentioned elsewhere, I think you are using the master branch.

This issue was fixed in the openstack/kolla-ansible 9.0.0.0rc1 release candidate.

This issue was fixed in the openstack/kolla-ansible 8.1.0 release.