tripleo

rhel8 jobs failing for missing containers " Exception: Not found image: docker://..."

Bug #1854062 reported by Marios Andreou on 2019-11-26

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	Fix Released	Critical	Unassigned	tripleo ussuri-1

Bug Description

many examples and affecting different jobs [1][2][3][4] with trace like:

        2019-11-26 04:30:22 | Exception: Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/rhel-binary-aodh-api:c67d7e39ad491e9f1660ce7613c312225b6510e5_6e3b098e
        2019-11-26 04:30:22 | Traceback (most recent call last):
        2019-11-26 04:30:22 | File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1262, in _standalone_deploy
...
        2019-11-26 04:30:22 | image_url.geturl())
        2019-11-26 04:30:22 | tripleo_common.image.exception.ImageNotFoundException: Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/rhel-binary-aodh-api:c67d7e39ad491e9f1660ce7613c312225b6510e5_6e3b098e
        2019-11-26 04:30:22 | None

I suspect this is a bad rhel8 promotion so I think new promotion is our best bet

[1] http://logs.rdoproject.org/21/695921/2/openstack-check/tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001/e2c0ef5/logs/undercloud/home/zuul/undercloud_install.log.txt.gz
[2] http://logs.rdoproject.org/47/695847/1/openstack-check/tripleo-ci-rhel-8-scenario001-standalone-rdo/e1fd1bc/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
[3] http://logs.rdoproject.org/65/694765/6/openstack-check/tripleo-ci-rhel-8-scenario001-standalone-rdo/bda94fd/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz
[4] http://logs.rdoproject.org/92/681092/6/openstack-check/tripleo-ci-rhel-8-scenario001-standalone-rdo/c274a2f/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz

Tags:

Marios Andreou (marios-b) on 2019-11-26

tags:

removed: promotion-blocker

Revision history for this message

Marios Andreou (marios-b) wrote on 2019-11-29:

in case it isn't clear from description, this should clear once we get a new rhel 8 promotion so just waiting on that for now

Revision history for this message

Marios Andreou (marios-b) wrote on 2019-12-03:

1854062_manual_push.txt Edit (42.5 KiB, text/plain)

chandan pushed containers manually to try address that

RHEL-8 master container push manually: http://paste.openstack.org/show/787018/ to rdo registry (attaching contents of the paste here too)

Revision history for this message

Marios Andreou (marios-b) wrote on 2019-12-03:

just went looking for some runs but it might be a bit too early still the ones i found latest fs1 and standalone1 are from last night and have same issue

---

https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-rhel-8-scenario001-standalone-rdo
latest now @ * http://logs.rdoproject.org/30/696330/5/openstack-check/tripleo-ci-rhel-8-scenario001-standalone-rdo/edaf6f0/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz

* 2019-12-02 21:08:44 | tripleo_common.image.exception.ImageNotFoundException: Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/rhel-binary-aodh-api:c67d7e39ad491e9f1660ce7613c312225b6510e5_6e3b098e

---

https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001
latest @
http://logs.rdoproject.org/30/696330/5/openstack-check/tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001/e61c55b/logs/undercloud/home/zuul/undercloud_install.log.txt.gz

* 2019-12-02 21:22:45 | tripleo_common.image.exception.ImageNotFoundException: Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/rhel-binary-cron:c67d7e39ad491e9f1660ce7613c312225b6510e5_6e3b098e