Brief Description
-----------------
stx-openstack apply failed due to CalledProcessError: Command '['/usr/local/bin/puppet-manifest-apply.sh' '/var/run/platform/puppet/20.12/hieradata', '192.168.204.2', 'controller', 'runtime', '/tmp/tmp_3nV4d.yaml']'
Severity
--------
Major
Expected Behavior
------------------
stx-openstack apply success
Actual Behavior
----------------
stx-openstack apply failed
Reproducibility
---------------
reproducible
System Configuration
--------------------
Regular standard 2+2
Branch/Pull Time/Commit
-----------------------
BUILD_ID="20210306T023358Z"
Timestamp/Logs
--------------
sysinv 2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common [-] Failed to execute runtime manifest for host 192.168.204.2: CalledProcessError: Command '['/usr/local/bin/puppet-manifest-apply.sh', '/var/run/platform/puppet/20.12/hieradata', '192.168.204.2', 'controller', 'runtime', '/tmp/tmp_3nV4d.yaml']' returned non-zero exit status 1
2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common Traceback (most recent call last):
2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common File "/usr/lib64/python2.7/site-packages/sysinv/puppet/common.py", line 76, in puppet_apply_manifest
2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common subprocess.check_call(cmd, stdout=fnull, stderr=fnull) # pylint: disable=not-callable
2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common raise CalledProcessError(retcode, cmd)
2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common CalledProcessError: Command '['/usr/local/bin/puppet-manifest-apply.sh', '/var/run/platform/puppet/20.12/hieradata', '192.168.204.2', 'controller', 'runtime', '/tmp/tmp_3nV4d.yaml']' returned non-zero exit status 1
2021-03-11 03:44:31.079 2688 ERROR sysinv.puppet.common
sysinv 2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager [-] failed to apply runtime manifest: SysinvException: Failed to execute runtime manifest for host 192.168.204.2
2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager Traceback (most recent call last):
2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager File "/usr/lib64/python2.7/site-packages/sysinv/agent/manager.py", line 1775, in _apply_runtime_manifest
2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager hieradata_path=hieradata_path)
2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager File "/usr/lib64/python2.7/site-packages/sysinv/puppet/common.py", line 81, in puppet_apply_manifest
2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager raise exception.SysinvException(_(msg))
2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager SysinvException: Failed to execute runtime manifest for host 192.168.204.2
2021-03-11 03:44:31.082 2688 ERROR sysinv.agent.manager
sysinv 2021-03-11 03:44:31.172 2688 INFO sysinv.agent.manager [-] Caught exception. Retrying... Exception: Failed to execute runtime manifest for host 192.168.204.2
sysinv 2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp [-] Exception during message handling: SysinvException: Failed to execute runtime manifest for host 192.168.204.2
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp Traceback (most recent call last):
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/amqp.py", line 437, in _process_data
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp **args)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp return Retrying(*dargs, **dkw).call(f, *args, **kw)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib/python2.7/site-packages/retrying.py", line 223, in call
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp return attempt.get(self._wrap_exception)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp six.reraise(self.value[0], self.value[1], self.value[2])
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 328, in inner
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp return f(*args, **kwargs)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/agent/manager.py", line 1706, in config_apply_runtime_manifest
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp self._apply_runtime_manifest(config_dict, hieradata_path=hieradata_path)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/agent/manager.py", line 1775, in _apply_runtime_manifest
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp hieradata_path=hieradata_path)
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/puppet/common.py", line 81, in puppet_apply_manifest
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp raise exception.SysinvException(_(msg))
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp SysinvException: Failed to execute runtime manifest for host 192.168.204.2
2021-03-11 03:44:31.173 2688 ERROR sysinv.openstack.common.rpc.amqp
sysinv 2021-03-11 03:44:35.204 2688 INFO sysinv.agent.manager [-] Agent config applied 748150cb-d1ac-42e9-895b-1aee23c6ffc1
sysinv 2021-03-11 03:50:19.078 2688 INFO sysinv.agent.manager [-] config_apply_runtime_manifest: 3cf332ae-6ff2-4644-ae16-1e3a3b4a2639 {u'classes': [u'platform::mtce::runtime'], u'force': False, u'personalities': [u'controller']} controller
sysinv 2021-03-11 03:50:19.079 2688 INFO sysinv.agent.manager [-] controller-standby or storage, mount /var/run/platform
sysinv 2021-03-11 03:50:19.109 2688 INFO sysinv.agent.manager [-] _apply_runtime_manifest with hieradata_path = '/var/run/platform/puppet/20.12/hieradata'
sysinv 2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common [-] Failed to execute runtime manifest for host 192.168.204.2: CalledProcessError: Command '['/usr/local/bin/puppet-manifest-apply.sh', '/var/run/platform/puppet/20.12/hieradata', '192.168.204.2', 'controller', 'runtime', '/tmp/tmpEHuxMT.yaml']' returned non-zero exit status 1
2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common Traceback (most recent call last):
2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common File "/usr/lib64/python2.7/site-packages/sysinv/puppet/common.py", line 76, in puppet_apply_manifest
2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common subprocess.check_call(cmd, stdout=fnull, stderr=fnull) # pylint: disable=not-callable
2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common raise CalledProcessError(retcode, cmd)
2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common CalledProcessError: Command '['/usr/local/bin/puppet-manifest-apply.sh', '/var/run/platform/puppet/20.12/hieradata', '192.168.204.2', 'controller', 'runtime', '/tmp/tmpEHuxMT.yaml']' returned non-zero exit status 1
2021-03-11 03:50:26.878 2688 ERROR sysinv.puppet.common
sysinv 2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager [-] failed to apply runtime manifest: SysinvException: Failed to execute runtime manifest for host 192.168.204.2
2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager Traceback (most recent call last):
2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager File "/usr/lib64/python2.7/site-packages/sysinv/agent/manager.py", line 1775, in _apply_runtime_manifest
2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager hieradata_path=hieradata_path)
2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager File "/usr/lib64/python2.7/site-packages/sysinv/puppet/common.py", line 81, in puppet_apply_manifest
2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager raise exception.SysinvException(_(msg))
2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager SysinvException: Failed to execute runtime manifest for host 192.168.204.2
2021-03-11 03:50:26.879 2688 ERROR sysinv.agent.manager
@yvonne : What steps did you execute before applying stx-openstack? Did you enable swift or make any other chart changes?