tripleo

registry.access.redhat.com/ubi9/pause: name unknown: Repo not found

Bug #1950279 reported by chandan kumar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Unassigned
tripleo xena-3

Bug Description

On CentOS Stream 9 container multinode job, during overcloud deploy, we are seeing following errors:
https://logserver.rdoproject.org/26/34926/10/check/periodic-tripleo-ci-centos-9-containers-multinode-master/013c609/logs/undercloud/home/zuul-worker/overcloud_deploy.log.txt.gz

```
Error: initializing source docker://registry.access.redhat.com/ubi9/pause:latest: reading manifest latest in registry.access.redhat.com/ubi9/pause: name unknown: Repo not found
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: subprocess.CalledProcessError: Command '['sudo', 'podman', 'play', 'kube', '/home/zuul-worker/overcloud-deploy/overcloud/heat-launcher/heat-pod.yaml']' returned non-zero exit status 125.
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last):
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/command.py", line 34, in run
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, self).run(parsed_args)
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/osc_lib/command/command.py", line 39, in run
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, self).run(parsed_args)
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/cliff/command.py", line 186, in run
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = self.take_action(parsed_args) or 0
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1161, in take_action
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud self.setup_ephemeral_heat(parsed_args)
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/v1/overcloud_deploy.py", line 627, in setup_ephemeral_heat
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud self.orchestration_client = utils.launch_heat(self.heat_launcher)
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/utils.py", line 2764, in launch_heat
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud launcher.launch_heat()
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.9/site-packages/tripleoclient/heat_launcher.py", line 499, in launch_heat
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud subprocess.check_call([
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib64/python3.9/subprocess.py", line 373, in check_call
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise CalledProcessError(retcode, cmd)
2021-11-09 04:02:58 | 2021-11-09 04:02:58.898 98153 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud subprocess.CalledProcessError: Command '['sudo', 'podman', 'play', 'kube', '/home/zuul-worker/overcloud-deploy/overcloud/heat-launcher/heat-pod.yaml']' returned non-zero exit status 125.
```

If I try to pull the above container locally, registry.access.redhat.com/ubi9/pause:latest
We are not able to pull it.

Logging this bug to track it and Opened a bug on centos stream side: https://bugzilla.redhat.com/show_bug.cgi?id=2021980

See original description
Revision history for this message
chandan kumar (chkumar246) wrote :

Similar issue is reported here: https://github.com/containers/podman/issues/11256

Revision history for this message
chandan kumar (chkumar246) wrote :

[root@2300431ac138 containers]# podman --version
podman version 3.4.2-dev
[root@2300431ac138 containers]# cat containers.conf | grep pause
#infra_command = "/pause"
# Infra (pause) container image name for pod infra containers. When running a
# pod, we start a `pause` process in a container to hold open the namespaces
#infra_image = "k8s.gcr.io/pause:3.4.1"
infra_image = "registry.access.redhat.com/ubi9/pause"

I need to propose a change in tripleo-ansible to fix it and use ubi8/pause currently.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/817202

Changed in tripleo:
status: Triaged → In Progress
chandan kumar (chkumar246)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/817202
Committed: https://opendev.org/openstack/tripleo-ansible/commit/46a09438e19650eb33bb6074148b2e802bea3fcd
Submitter: "Zuul (22348)"
Branch: master

commit 46a09438e19650eb33bb6074148b2e802bea3fcd
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Tue Nov 9 18:39:06 2021 +0530

    Explicitly define infra_image in containers.conf

    Currently in /usr/share/containers/containers.conf,
    under engine,
    infra_image = "registry.access.redhat.com/ubi9/pause" is set
    by default but, ubi9/pause container image does not exists.

    This patch sets infra_image and sets ubi8/pause as a
    default for infra_image till it gets available.

    Closes-Bug: #1950279

    Signed-off-by: Chandan Kumar (raukadah) <email address hidden>
    Change-Id: I2048de62700ae46896a77b4fd59f272032ef719c

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ansible 4.2.0

This issue was fixed in the openstack/tripleo-ansible 4.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.