c8 standalone full tempest scenario wallaby times out consistently

Bug #1963884 reported by Soniya Murlidhar Vyas
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-wallaby is getting 'TIMED_OUT' consistently

In the job output below, most of the tasks have been skipped.
--------------------------------------------------------------------------------------------

2022-03-06 18:21:55.095248 | primary | TASK [os_tempest : Execute tempest tests] **************************************
2022-03-06 18:21:55.095271 | primary | Sunday 06 March 2022 18:21:55 +0000 (0:00:00.043) 1:42:47.500 **********
2022-03-06 20:23:57.596972 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml@master]
2022-03-06 20:23:57.621898 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/tripleo-ci-periodic-base/post.yaml@master]

2022-03-06 18:17:38.405089 | primary | TASK [standalone : Install ansible-lint] ***************************************
2022-03-06 18:17:38.405098 | primary | Sunday 06 March 2022 18:17:38 +0000 (0:00:00.073) 1:38:30.809 **********
2022-03-06 18:17:38.431081 | primary | skipping: [undercloud]
2022-03-06 18:17:38.441112 | primary |
2022-03-06 18:17:38.441136 | primary | TASK [standalone : List playbooks] *********************************************
2022-03-06 18:17:38.441145 | primary | Sunday 06 March 2022 18:17:38 +0000 (0:00:00.036) 1:38:30.846 **********
2022-03-06 18:17:38.465724 | primary | skipping: [undercloud]
2022-03-06 18:17:38.481023 | primary |
2022-03-06 18:17:38.481090 | primary | TASK [standalone : Lint playbooks] *********************************************
2022-03-06 18:17:38.481099 | primary | Sunday 06 March 2022 18:17:38 +0000 (0:00:00.039) 1:38:30.885 **********
2022-03-06 18:17:38.506049 | primary | skipping: [undercloud]
2022-03-06 18:17:38.535095 | primary |
2022-03-06 18:17:38.535149 | primary | PLAY [Execute Validations Framework tests] *************************************
2022-03-06 18:17:38.552022 | primary |
2022-03-06 18:17:38.552104 | primary | TASK [include_role : validations] **********************************************
2022-03-06 18:17:38.552117 | primary | Sunday 06 March 2022 18:17:38 +0000 (0:00:00.070) 1:38:30.956 **********
2022-03-06 18:17:38.588987 | primary | skipping: [undercloud]
2022-03-06 18:17:38.613085 | primary |
2022-03-06 18:17:38.613150 | primary | PLAY [Validate the undercloud] *************************************************
2022-03-06 18:17:38.632081 | primary |
2022-03-06 18:17:38.632129 | primary | TASK [validate-services : include_tasks] ***************************************
2022-03-06 18:17:38.632138 | primary | Sunday 06 March 2022 18:17:38 +0000 (0:00:00.079) 1:38:31.035 **********
2022-03-06 18:17:38.656988 | primary | skipping: [undercloud]
2022-03-06 18:17:38.666974 | primary |
2022-03-06 18:17:38.667008 | primary | TASK [validate-services : include_tasks] ***************************************
2022-03-06 18:17:38.667017 | primary | Sunday 06 March 2022 18:17:38 +0000 (0:00:00.036) 1:38:31.071 **********
2022-03-06 18:17:38.691047 | primary | skipping: [undercloud]
2022-03-06 18:17:40.654035 | primary | TASK [Add skiplist tests to tempest_test_whitelist - run them] *****************
2022-03-06 18:17:40.654044 | primary | Sunday 06 March 2022 18:17:40 +0000 (0:00:00.052) 1:38:33.058 **********
2022-03-06 18:17:40.675014 | primary | skipping: [undercloud]
2022-03-06 18:17:40.697056 | primary |
2022-03-06 18:17:40.697110 | primary | TASK [Load tests to be executed] ***********************************************
2022-03-06 18:17:40.697123 | primary | Sunday 06 March 2022 18:17:40 +0000 (0:00:00.043) 1:38:33.101 **********
2022-03-06 18:17:40.714982 | primary | skipping: [undercloud]
2022-03-06 18:17:40.738018 | primary |
2022-03-06 18:17:40.738083 | primary | TASK [Add tests to tempest_test_includelist] ***********************************
2022-03-06 18:17:40.738101 | primary | Sunday 06 March 2022 18:17:40 +0000 (0:00:00.041) 1:38:33.142 **********
2022-03-06 18:17:40.756168 | primary | skipping: [undercloud]
2022-03-06 18:17:40.765025 | primary |
2022-03-06 18:17:40.765079 | primary | TASK [set python_cmd] **********************************************************
2022-03-06 18:17:40.765092 | primary | Sunday 06 March 2022 18:17:40 +0000 (0:00:00.026) 1:38:33.169 **********
2022-03-06 18:17:40.801078 | primary | ok: [undercloud]
2022-03-06 18:17:40.811983 | primary |
2022-03-06 18:17:40.812023 | primary | TASK [Check if virtualenv is in the system] ************************************
2022-03-06 18:17:40.812031 | primary | Sunday 06 March 2022 18:17:40 +0000 (0:00:00.046) 1:38:33.216 **********
2022-03-06 18:17:42.534649 | primary | ok: [undercloud]
2022-03-06 18:17:42.542743 | primary |
2022-03-06 18:17:42.542785 | primary | TASK [Install virtualenv] ******************************************************
2022-03-06 18:17:42.543018 | primary | Sunday 06 March 2022 18:17:42 +0000 (0:00:01.731) 1:38:34.947 **********
2022-03-06 18:17:42.568137 | primary | skipping: [undercloud]
2022-03-06 18:17:42.578135 | primary |
2022-03-06 18:17:42.578213 | primary | TASK [Set the rc file] *********************************************************
2022-03-06 18:17:42.578230 | primary | Sunday 06 March 2022 18:17:42 +0000 (0:00:00.034) 1:38:34.982 **********
2022-03-06 18:17:42.601038 | primary | skipping: [undercloud]
2022-03-06 18:17:42.613985 | primary |
2022-03-06 18:17:42.614045 | primary | TASK [Disable stackviz for CentOS Stream 9] ************************************
2022-03-06 18:17:42.617186 | primary | Sunday 06 March 2022 18:17:42 +0000 (0:00:00.036) 1:38:35.018 **********
2022-03-06 18:17:42.637004 | primary | skipping: [undercloud]

For more details, please refer the following details:-

1. https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-wallaby/1c1a273/job-output.txt
2. https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-wallaby/1c1a273/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz

Revision history for this message
Ronelle Landy (rlandy) wrote :

This is working now - cloud instability

Changed in tripleo:
importance: Undecided → Critical
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.