python-tripleoclient cannot handle multiple lines when conditionals in the upgrade tasks

Bug #1715871 reported by Marius Cornea
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Emilien Macchi

Bug Description

With the following patch applied:

https://review.openstack.org/#/c/501734/5/docker/services/pacemaker/haproxy.yaml

note lines 245-247

openstack --debug overcloud config download --config-dir compute-0/tripleo-JdgDtQ-config/
fails with the following error:

expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer
clean_up DownloadConfig: expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 134, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 279, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 169, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer

Adding a debug line to provide more info we can see that the when conditional containing multiple lines is not handled well by tripleoclient:

whenline=rpm_python_check.changed != false type=<type 'unicode'>
whenline=rpm_python_check.changed != false type=<type 'unicode'>
whenline=httpd_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_api_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_scheduler_enabled.rc == 0 type=<type 'unicode'>
whenline=heat_api_cloudwatch_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_server_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_dhcp_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_l3_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_metadata_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_ovs_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_apache.rc == 0 type=<type 'unicode'>
whenline=cinder_api_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_scheduler_enabled.rc == 0 type=<type 'unicode'>
whenline=heat_api_cloudwatch_apache.rc == 0 type=<type 'unicode'>
whenline=heat_api_cloudwatch_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_server_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_dhcp_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_l3_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_metadata_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_ovs_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=is_bootstrap_node type=<type 'unicode'>
whenline=[u'is_bootstrap_node', u'haproxy_res is defined and haproxy_res|succeeded'] type=<type 'list'>
expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer
clean_up DownloadConfig: expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 134, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 279, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 169, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer

END return value: 1

Revision history for this message
Steven Hardy (shardy) wrote :

I took a look at this, there's actually a comment about it in the code :(

I'm working on a patch

Changed in tripleo:
assignee: nobody → Steven Hardy (shardy)
tags: added: pike-backport-potential tripleoclient
Steven Hardy (shardy)
Changed in tripleo:
milestone: none → queens-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/502076

Changed in tripleo:
status: Confirmed → In Progress
Changed in tripleo:
assignee: Steven Hardy (shardy) → Emilien Macchi (emilienm)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (master)

Reviewed: https://review.openstack.org/502076
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=fa4b8fb00ce199fa767e0b0b0349e6fc3eaf4b95
Submitter: Jenkins
Branch: master

commit fa4b8fb00ce199fa767e0b0b0349e6fc3eaf4b95
Author: Steven Hardy <email address hidden>
Date: Fri Sep 8 15:46:13 2017 +0100

    Handle when lists not only string in config download

    There was a comment about this in the code, we didn't handle
    when lists properly, only strings.

    Also fixed the test to avoid leaking temporary files every tox run.

    Change-Id: I24f80199ba430870af2b4aab9453387304403215
    Closes-Bug: #1715871

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/502291

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (stable/pike)

Reviewed: https://review.openstack.org/502291
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=e786a130fa699f0137bad186942bab47ed220786
Submitter: Jenkins
Branch: stable/pike

commit e786a130fa699f0137bad186942bab47ed220786
Author: Steven Hardy <email address hidden>
Date: Fri Sep 8 15:46:13 2017 +0100

    Handle when lists not only string in config download

    There was a comment about this in the code, we didn't handle
    when lists properly, only strings.

    Also fixed the test to avoid leaking temporary files every tox run.

    Change-Id: I24f80199ba430870af2b4aab9453387304403215
    Closes-Bug: #1715871
    (cherry picked from commit fa4b8fb00ce199fa767e0b0b0349e6fc3eaf4b95)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-tripleoclient 7.3.1

This issue was fixed in the openstack/python-tripleoclient 7.3.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-tripleoclient 8.0.0

This issue was fixed in the openstack/python-tripleoclient 8.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.