Description
===========
Queens and rocky jobs are failing with error The WebSocket timed out before the Workflow completed.
Affected jobs:
==============
- periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-queens
- periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens
- periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset021-queens
- periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-queens
Pipeline link:
=============
- https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-wednesday-weekend
Actual Error:
=============
Timed out waiting for messages from Execution (ID: c49f2b5f-5b36-4ebc-8765-fedc40ba9dcd, State: RUNNING). The WebSocket timed out before the Workflow completed.
same error reflecting in different jobs.
Logs:
=====
2020-04-28 23:52:46 | TASK [Write docker-puppet-tasks json files] ************************************
2020-04-28 23:52:46 | task path: /var/lib/mistral/c49f2b5f-5b36-4ebc-8765-fedc40ba9dcd/common_deploy_steps_tasks.yaml:248
2020-04-28 23:52:46 | Tuesday 28 April 2020 23:52:43 +0000 (0:00:00.502) 0:04:48.756 *********
2020-04-29 00:53:36 | changed: [overcloud-controller-0] => (item=step_3) => {"changed": true, "checksum": "de9ae0fa287ee107bf45fe11d5b6fc7bfbbf6ebc", "dest": "/var/lib/docker-puppet/docker-puppet-tasks3.json", "gid": 0, "group": "root", "item": ["step_3", [{"config_image": "192.168.24.1:8787/tripleoqueens/centos-binary-keystone:c9e0fef7c387836d4d3ad46d073f10c312857a4f_5fce6f98", "config_volume": "keystone_init_tasks", "puppet_tags": "keystone_config,keystone_domain_config,keystone_endpoint,keystone_identity_provider,keystone_paste_ini,keystone_role,keystone_service,keystone_tenant,keystone_user,keystone_user_role,keystone_domain", "step_config": "include ::tripleo::profile::base::keystone"}]], "md5sum": "7e5f0188d1467da028173d810e033929", "mode": "0600", "owner": "root", "sTimed out waiting for messages from Execution (ID: c49f2b5f-5b36-4ebc-8765-fedc40ba9dcd, State: RUNNING). The WebSocket timed out before the Workflow completed.
2020-04-29 00:53:36 |
2020-04-29 00:53:36 | END return value: 1
2020-04-29 00:53:36 | econtext": "system_u:object_r:var_lib_t:s0", "size": 444, "src": "/home/tripleo-admin/.ansible/tmp/ansible-tmp-1588117964.24-35593494992396/source", "state": "file", "uid": 0}
2020-04-29 00:53:36 |
2020-04-29 00:53:36 | TASK [Set host puppet debugging fact string] ***********************************
2020-04-29 00:53:36 | task path: /var/lib/mistral/c49f2b5f-5b36-4ebc-8765-fedc40ba9dcd/common_deploy_steps_tasks.yaml:261
2020-04-29 00:53:36 | Tuesday 28 April 2020 23:52:45 +0000 (0:00:01.089) 0:04:49.845 *********
2020-04-29 00:53:36 |
2020-04-29 00:53:36 | TASK [Write the config_step hieradata] *****************************************
2020-04-29 00:53:36 | task path: /var/lib/mistral/c49f2b5f-5b36-4ebc-8765-fedc40ba9dcd/common_deploy_steps_tasks.yaml:268
2020-04-29 00:53:36 | Tuesday 28 April 2020 23:52:45 +0000 (0:00:00.240) 0:04:50.086 *********
2020-04-29 00:53:36 | changed: [overcloud-controller-2] => {"changed": true, "checksum": "dfdcc7695edd230e7a2c06fc7b739bfa56506d8f", "dest": "/etc/puppet/hieradata/config_step.json", "gid": 0, "group": "root", "md5sum": "f0ef53dcc6eb8440334b1ebaa90bfd63", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:puppet_etc_t:s0", "size": 11, "src": "/home/tripleo-admin/.ansible/tmp/ansible-tmp-1588117965.36-137127088675725/source", "state": "file", "uid": 0}
2020-04-29 00:53:36 |
2020-04-29 00:53:36 | changed: [overcloud-controller-1] => {"changed": true, "checksum": "dfdcc7695edd230e7a2c06fc7b739bfa56506d8f", "dest": "/etc/puppet/hieradata/config_step.json", "gid": 0, "group": "root", "md5sum": "f0ef53dcc6eb8440334b1ebaa90bfd63", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:puppet_etc_t:s0", "size": 11, "src": "/home/tripleo-admin/.ansible/tmp/ansible-tmp-1588117965.43-89045811942861/source", "state": "file", "uid": 0}
2020-04-29 00:53:36 | changed: [overcloud-controller-0] => {"changed": true, "checksum": "dfdcc7695edd230e7a2c06fc7b739bfa56506d8f", "dest": "/etc/puppet/hieradata/config_step.json", "gid": 0, "group": "root", "md5sum": "f0ef53dcc6eb8440334b1ebaa90bfd63", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:puppet_etc_t:s0", "size": 11, "src": "/home/tripleo-admin/.ansible/tmp/ansible-tmp-1588117965.5-26684484695720/source", "state": "file", "uid": 0}
2020-04-29 00:53:36 | changed: [overcloud-novacompute-0] => {"changed": true, "checksum": "dfdcc7695edd230e7a2c06fc7b739bfa56506d8f", "dest": "/etc/puppet/hieradata/config_step.json", "gid": 0, "group": "root", "md5sum": "f0ef53dcc6eb8440334b1ebaa90bfd63", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:puppet_etc_t:s0", "size": 11, "src": "/home/tripleo-admin/.ansible/tmp/ansible-tmp-1588117965.62-210910173425716/source", "state": "file", "uid": 0}
2020-04-29 00:53:36 |
2020-04-29 00:53:36 | TASK [Run puppet host configuration for step 1] ********************************
2020-04-29 00:53:36 | task path: /var/lib/mistral/c49f2b5f-5b36-4ebc-8765-fedc40ba9dcd/common_deploy_steps_tasks.yaml:274
2020-04-29 00:53:36 | Tuesday 28 April 2020 23:52:46 +0000 (0:00:01.243) 0:04:51.330 *********
2020-04-29 00:53:36 | changed: [overcloud-novacompute-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}
2020-04-29 00:53:36 |
2020-04-29 00:53:37 | + status_code=1
2020-04-29 00:53:37 | + openstack stack list
2020-04-29 00:53:37 | + grep -q overcloud
2020-04-29 00:53:41 | + openstack stack list
2020-04-29 00:53:41 | + grep -Eq '(CREATE|UPDATE)_COMPLETE'
2020-04-29 00:53:45 | + exit 1
Logs: /logserver. rdoproject. org/openstack- periodic- wednesday- weekend/ opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 7-multinode- 1ctlr-featurese t037-updates- queens/ 9c7889c/ logs/undercloud /home/zuul/ overcloud_ deploy. log.txt. gz
1. https:/
2. https:/ /logserver. rdoproject. org/openstack- periodic- wednesday- weekend/ opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 7-multinode- 1ctlr-featurese t037-updates- queens/ 9c7889c/ logs/undercloud /home/zuul/ overcloud_ deploy. log.txt. gz
3. https:/ /logserver. rdoproject. org/openstack- periodic- wednesday- weekend/ opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 7-ovb-1ctlr_ 2comp-featurese t021-queens/ dc5e08d/ logs/undercloud /home/zuul/ overcloud_ deploy. log.txt. gz
4. https:/ /logserver. rdoproject. org/openstack- periodic- wednesday- weekend/ opendev. org/openstack/ tripleo- ci/master/ periodic- tripleo- ci-centos- 7-ovb-1ctlr_ 2comp-featurese t020-queens/ 3136791/ logs/undercloud /home/zuul/ overcloud_ deploy. log.txt. gz