Scenario001 master failing on standalone deploy with FATAL | Gather podman infos | standalone | error={"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}

Bug #1931090 reported by Sandeep Yadav
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

Description:-

Scenario001 master failing on standalone deploy with FATAL | Gather podman infos | standalone | error={"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}

Issue started on 06th may:-

https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario001-standalone-master

Logs are not detailed enough, but these snippets are observed:-

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-master/ffb7cc1/job-output.txt
~~~
2021-06-06 18:15:35.409697 | primary |
2021-06-06 18:15:35.409778 | primary | TASK [tripleo.operator.tripleo_deploy : Standalone deploy] *********************
2021-06-06 18:15:35.409836 | primary | Sunday 06 June 2021 18:15:35 +0000 (0:00:00.643) 0:06:03.030 ***********
2021-06-06 18:54:50.653970 | primary | fatal: [undercloud]: FAILED! => {
2021-06-06 18:54:50.654234 | primary | "ansible_job_id": "623894129255.47881",
2021-06-06 18:54:50.654255 | primary | "changed": false,
2021-06-06 18:54:50.654289 | primary | "cmd": "openstack tripleo deploy --templates $DEPLOY_TEMPLATES --yes --output-dir $DEPLOY_OUTPUT_DIR --stack $DEPLOY_STACK --standalone-role $DEPLOY_STANDALONE_ROLE --timeout $DEPLOY_TIMEOUT_ARG -e /usr/share/openstack-tripleo-heat-templates/environments/standalone/standalone-tripleo.yaml -e /home/zuul/containers-prepare-parameters.yaml -e /home/zuul/standalone_parameters.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml -e /usr/share/openstack-tripleo-heat-templates/ci/environments/scenario001-standalone.yaml -r $DEPLOY_ROLES_FILE --deployment-user $DEPLOY_DEPLOYMENT_USER --local-ip $DEPLOY_LOCAL_IP --control-virtual-ip $DEPLOY_CONTROL_VIP >/home/zuul/standalone_deploy.log 2>&1",
2021-06-06 18:54:50.654301 | primary | "delta": "0:39:08.357938",
2021-06-06 18:54:50.654308 | primary | "end": "2021-06-06 18:54:45.523267",
2021-06-06 18:54:50.654325 | primary | "finished": 1,
2021-06-06 18:54:50.654340 | primary | "rc": 1,
2021-06-06 18:54:50.654351 | primary | "start": "2021-06-06 18:15:37.165329"
2021-06-06 18:54:50.654357 | primary | }
~~~

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-master/ffb7cc1/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
~~~
2021-06-06 18:53:15.504335 | fa163ea2-f424-6f3f-0d00-000000003d2b | FATAL | Gather podman infos | standalone | error={"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
~~~

Another example:
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-master/6c43f11/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-ansible (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/795077

Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :
Revision history for this message
Rabi Mishra (rabi) wrote :

https://review.opendev.org/c/openstack/tripleo-ansible/+/789928 was supposed to fix the issue AFAIR. Looks like we still have the issue.

Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :

Testpatch logs:-

https://logserver.rdoproject.org/54/32054/27/check/periodic-tripleo-ci-centos-8-scenario001-standalone-master/600f0b2/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz

~~~
2021-06-07 16:49:32.361146 | fa163e7d-03f0-933b-18bb-00000000610e | FATAL | Gather podman infos | standalone | error={"changed": false, "msg": "Unable to get list of containers: Error: container d7e40ed7d8c0788c5e75268f0daa5808065c752ddc0d99a6637f2ed92c2cde94 does not exist in database: no such container\n"}
~~~

Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote :

After we removed ansible podman module from tripleo-ansible, the workaround has gone as well. I'll release a new RPM of ansible podman collection today with the workaround and it should solve it.

Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote :
Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote :

And updated in RDO: https://review.rdoproject.org/r/c/rdoinfo/+/34033
So let's just wait for update of RPM

Revision history for this message
wes hayutin (weshayutin) wrote :

scenario001 in master intergration is now passing.. thanks Sagi!

Changed in tripleo:
status: Triaged → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-ansible (master)

Change abandoned by "Sandeep Yadav <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/795077
Reason: This was for a test and not needed anymore.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.