In multinode environment with manila component enabled when running "kolla-ansible reconfigure" the command will fail with message:
error while evaluating conditional: config_strategy == "COPY_ONCE" or item[1]['KOLLA_CONFIG_STRATEGY'] == 'COPY_ONCE'
in step:
TASK: [manila | Remove the containers]
The manila-related inventory configuration:
[control]
gs0104
gs0105
gs0106
[storage]
ssd05
ssd06
[manila:children]
control
[manila-api:children]
manila
[manila-scheduler:children]
manila
[manila-share:children]
storage
The important ansible steps log:
TASK: [manila | Check the configs] ********************************************
skipping: [ssd06] => (item={'group': 'manila-api', 'name': 'manila_api'})
skipping: [ssd06] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
skipping: [ssd05] => (item={'group': 'manila-api', 'name': 'manila_api'})
skipping: [ssd05] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
ok: [gs0105] => (item={'group': 'manila-api', 'name': 'manila_api'})
ok: [gs0106] => (item={'group': 'manila-api', 'name': 'manila_api'})
ok: [gs0104] => (item={'group': 'manila-api', 'name': 'manila_api'})
ok: [ssd06] => (item={'group': 'manila-share', 'name': 'manila_share'})
ok: [ssd05] => (item={'group': 'manila-share', 'name': 'manila_share'})
ok: [gs0105] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
ok: [gs0106] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
skipping: [gs0105] => (item={'group': 'manila-share', 'name': 'manila_share'})
ok: [gs0104] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
skipping: [gs0106] => (item={'group': 'manila-share', 'name': 'manila_share'})
skipping: [gs0104] => (item={'group': 'manila-share', 'name': 'manila_share'})
TASK: [manila | Containers config strategy] ***********************************
skipping: [ssd05] => (item={'group': 'manila-api', 'name': 'manila_api'})
skipping: [ssd06] => (item={'group': 'manila-api', 'name': 'manila_api'})
skipping: [ssd05] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
skipping: [ssd06] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
ok: [gs0106] => (item={'group': 'manila-api', 'name': 'manila_api'})
ok: [gs0104] => (item={'group': 'manila-api', 'name': 'manila_api'})
ok: [gs0105] => (item={'group': 'manila-api', 'name': 'manila_api'})
ok: [ssd06] => (item={'group': 'manila-share', 'name': 'manila_share'})
ok: [ssd05] => (item={'group': 'manila-share', 'name': 'manila_share'})
ok: [gs0106] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
ok: [gs0104] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
skipping: [gs0106] => (item={'group': 'manila-share', 'name': 'manila_share'})
ok: [gs0105] => (item={'group': 'manila-scheduler', 'name': 'manila_scheduler'})
skipping: [gs0104] => (item={'group': 'manila-share', 'name': 'manila_share'})
skipping: [gs0105] => (item={'group': 'manila-share', 'name': 'manila_share'})
TASK: [manila | Remove the containers] ****************************************
skipping: [gs0104] => (item=[{'group': 'manila-api', 'name': 'manila_api'}, {'KOLLA_BASE_DISTRO': 'centos', 'PS1': '$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ', 'KOLLA_INSTALL_TYPE': 'source', 'changed': False, 'item': {'group': 'manila-api', 'name': 'manila_api'}, 'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS', 'PATH': '/var/lib/kolla/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'invocation': {'module_name': u'kolla_docker', 'module_complex_args': {'action': 'get_container_env', 'name': u'manila_api'}, 'module_args': ''}, 'KOLLA_SERVICE_NAME': 'manila-api', 'KOLLA_INSTALL_METATYPE': 'mixed'}, {'cmd': ['docker', 'exec', 'manila_api', '/usr/local/bin/kolla_set_configs', '--check'], 'end': '2016-07-15 11:41:41.572993', 'stderr': 'INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:The config files are in the expected state', 'stdout': u'', 'item': {'group': 'manila-api', 'name': 'manila_api'}, 'changed': False, 'rc': 0, 'failed': False, 'warnings': [], 'delta': '0:00:00.072470', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'docker exec manila_api /usr/local/bin/kolla_set_configs --check'}, 'stdout_lines': [], 'failed_when_result': False, 'start': '2016-07-15 11:41:41.500523'}])
skipping: [gs0105] => (item=[{'group': 'manila-api', 'name': 'manila_api'}, {'KOLLA_BASE_DISTRO': 'centos', 'PS1': '$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ', 'KOLLA_INSTALL_TYPE': 'source', 'changed': False, 'item': {'group': 'manila-api', 'name': 'manila_api'}, 'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS', 'PATH': '/var/lib/kolla/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'invocation': {'module_name': u'kolla_docker', 'module_complex_args': {'action': 'get_container_env', 'name': u'manila_api'}, 'module_args': ''}, 'KOLLA_SERVICE_NAME': 'manila-api', 'KOLLA_INSTALL_METATYPE': 'mixed'}, {'cmd': ['docker', 'exec', 'manila_api', '/usr/local/bin/kolla_set_configs', '--check'], 'end': '2016-07-15 11:41:41.574359', 'stderr': 'INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:The config files are in the expected state', 'stdout': u'', 'item': {'group': 'manila-api', 'name': 'manila_api'}, 'changed': False, 'rc': 0, 'failed': False, 'warnings': [], 'delta': '0:00:00.070076', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'docker exec manila_api /usr/local/bin/kolla_set_configs --check'}, 'stdout_lines': [], 'failed_when_result': False, 'start': '2016-07-15 11:41:41.504283'}])
skipping: [gs0106] => (item=[{'group': 'manila-api', 'name': 'manila_api'}, {'KOLLA_BASE_DISTRO': 'centos', 'PS1': '$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ', 'KOLLA_INSTALL_TYPE': 'source', 'changed': False, 'item': {'group': 'manila-api', 'name': 'manila_api'}, 'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS', 'PATH': '/var/lib/kolla/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'invocation': {'module_name': u'kolla_docker', 'module_complex_args': {'action': 'get_container_env', 'name': u'manila_api'}, 'module_args': ''}, 'KOLLA_SERVICE_NAME': 'manila-api', 'KOLLA_INSTALL_METATYPE': 'mixed'}, {'cmd': ['docker', 'exec', 'manila_api', '/usr/local/bin/kolla_set_configs', '--check'], 'end': '2016-07-15 11:41:41.574669', 'stderr': 'INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:The config files are in the expected state', 'stdout': u'', 'item': {'group': 'manila-api', 'name': 'manila_api'}, 'changed': False, 'rc': 0, 'failed': False, 'warnings': [], 'delta': '0:00:00.072411', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'docker exec manila_api /usr/local/bin/kolla_set_configs --check'}, 'stdout_lines': [], 'failed_when_result': False, 'start': '2016-07-15 11:41:41.502258'}])
fatal: [ssd05] => error while evaluating conditional: config_strategy == "COPY_ONCE" or item[1]['KOLLA_CONFIG_STRATEGY'] == 'COPY_ONCE'
fatal: [ssd06] => error while evaluating conditional: config_strategy == "COPY_ONCE" or item[1]['KOLLA_CONFIG_STRATEGY'] == 'COPY_ONCE'
skipping: [gs0104] => (item=[{'group': 'manila-scheduler', 'name': 'manila_scheduler'}, {'KOLLA_BASE_DISTRO': 'centos', 'PS1': '$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ', 'KOLLA_INSTALL_TYPE': 'source', 'changed': False, 'item': {'group': 'manila-scheduler', 'name': 'manila_scheduler'}, 'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS', 'PATH': '/var/lib/kolla/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'invocation': {'module_name': u'kolla_docker', 'module_complex_args': {'action': 'get_container_env', 'name': u'manila_scheduler'}, 'module_args': ''}, 'KOLLA_SERVICE_NAME': 'manila-scheduler', 'KOLLA_INSTALL_METATYPE': 'mixed'}, {'cmd': ['docker', 'exec', 'manila_scheduler', '/usr/local/bin/kolla_set_configs', '--check'], 'end': '2016-07-15 11:41:41.793828', 'stderr': 'INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:The config files are in the expected state', 'stdout': u'', 'item': {'group': 'manila-scheduler', 'name': 'manila_scheduler'}, 'changed': False, 'rc': 0, 'failed': False, 'warnings': [], 'delta': '0:00:00.072669', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'docker exec manila_scheduler /usr/local/bin/kolla_set_configs --check'}, 'stdout_lines': [], 'failed_when_result': False, 'start': '2016-07-15 11:41:41.721159'}])
skipping: [gs0105] => (item=[{'group': 'manila-scheduler', 'name': 'manila_scheduler'}, {'KOLLA_BASE_DISTRO': 'centos', 'PS1': '$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ', 'KOLLA_INSTALL_TYPE': 'source', 'changed': False, 'item': {'group': 'manila-scheduler', 'name': 'manila_scheduler'}, 'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS', 'PATH': '/var/lib/kolla/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'invocation': {'module_name': u'kolla_docker', 'module_complex_args': {'action': 'get_container_env', 'name': u'manila_scheduler'}, 'module_args': ''}, 'KOLLA_SERVICE_NAME': 'manila-scheduler', 'KOLLA_INSTALL_METATYPE': 'mixed'}, {'cmd': ['docker', 'exec', 'manila_scheduler', '/usr/local/bin/kolla_set_configs', '--check'], 'end': '2016-07-15 11:41:41.794886', 'stderr': 'INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:The config files are in the expected state', 'stdout': u'', 'item': {'group': 'manila-scheduler', 'name': 'manila_scheduler'}, 'changed': False, 'rc': 0, 'failed': False, 'warnings': [], 'delta': '0:00:00.073903', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'docker exec manila_scheduler /usr/local/bin/kolla_set_configs --check'}, 'stdout_lines': [], 'failed_when_result': False, 'start': '2016-07-15 11:41:41.720983'}])
skipping: [gs0106] => (item=[{'group': 'manila-scheduler', 'name': 'manila_scheduler'}, {'KOLLA_BASE_DISTRO': 'centos', 'PS1': '$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ', 'KOLLA_INSTALL_TYPE': 'source', 'changed': False, 'item': {'group': 'manila-scheduler', 'name': 'manila_scheduler'}, 'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS', 'PATH': '/var/lib/kolla/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'invocation': {'module_name': u'kolla_docker', 'module_complex_args': {'action': 'get_container_env', 'name': u'manila_scheduler'}, 'module_args': ''}, 'KOLLA_SERVICE_NAME': 'manila-scheduler', 'KOLLA_INSTALL_METATYPE': 'mixed'}, {'cmd': ['docker', 'exec', 'manila_scheduler', '/usr/local/bin/kolla_set_configs', '--check'], 'end': '2016-07-15 11:41:41.801423', 'stderr': 'INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:The config files are in the expected state', 'stdout': u'', 'item': {'group': 'manila-scheduler', 'name': 'manila_scheduler'}, 'changed': False, 'rc': 0, 'failed': False, 'warnings': [], 'delta': '0:00:00.072810', 'invocation': {'module_name': u'command', 'module_complex_args': {}, 'module_args': u'docker exec manila_scheduler /usr/local/bin/kolla_set_configs --check'}, 'stdout_lines': [], 'failed_when_result': False, 'start': '2016-07-15 11:41:41.728613'}])
fatal: [gs0104] => error while evaluating conditional: config_strategy == "COPY_ONCE" or item[1]['KOLLA_CONFIG_STRATEGY'] == 'COPY_ONCE'
fatal: [gs0105] => error while evaluating conditional: config_strategy == "COPY_ONCE" or item[1]['KOLLA_CONFIG_STRATEGY'] == 'COPY_ONCE'
fatal: [gs0106] => error while evaluating conditional: config_strategy == "COPY_ONCE" or item[1]['KOLLA_CONFIG_STRATEGY'] == 'COPY_ONCE'
It looks like it fails because some items are skipped in steps "Check the configs" and "Containers config strategy". Because of this, the item[1] (container_envs.results) cannot have the 'KOLLA_CONFIG_STRATEGY' defined.
Why these items are skipped? Because part of the manila components are deployed on the controller ansible group, and the other part on the storage ansible group, which is ok I guess.
From the code analysis it seems that this is not only manila-related problem. The same problem may occur with other components, when someone will try to customize placement of the component "subelements". For example:
# Murano
[murano-api:children]
host001
[murano-engine:children]
host002
Affected components list:
* ceilometer
* glance
* heat
* ironic
* magnum
* mistral
* murano
Solution: change the order of "when" statements and bring "inventory_hostname in groups[item[0]['group']]" to the top.
Reviewed: https:/ /review. openstack. org/343604 /git.openstack. org/cgit/ openstack/ kolla/commit/ ?id=e61f13d880e fd95de8002854ad 9e64526d34ab4b
Committed: https:/
Submitter: Jenkins
Branch: master
commit e61f13d880efd95 de8002854ad9e64 526d34ab4b
Author: Bartłomiej Daca <email address hidden>
Date: Mon Jul 18 13:51:11 2016 +0200
Fix 'reconfigure' command of kolla
Changed the order of the 'when' statements in "remove/restart
containers" tasks. It will fix the reconfiguration problem when
deploying different components on different hosts.
Change-Id: Ibee9dd56b6128b 664144deb1d9eb7 ec32e39fd5c
Closes-Bug: #1603943