tripleo

periodic-tripleo-ci-centos-8-standalone-upgrade-master failing, tripleo upgrade not identifed as an openstack command.

Bug #1930575 reported by Sandeep Yadav
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned
tripleo xena-1

Bug Description

Description:_

periodic-tripleo-ci-centos-8-standalone-upgrade-master is failing with below error snippet:-

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-upgrade-master/9bd4b10/logs/undercloud/home/zuul/standalone_upgrade.log.txt.gz

~~~
2021-06-01 18:47:06 | Could not load 'tripleo_config_generate_ansible': No module named 'tripleo_common.actions'
2021-06-01 18:47:06 | Could not load 'tripleo_deploy': No module named 'tripleo_common.actions'
2021-06-01 18:47:06 | Could not load 'tripleo_upgrade': No module named 'tripleo_common.actions'
2021-06-01 18:47:08 | Could not load 'tripleo_config_generate_ansible': No module named 'tripleo_common.actions'
2021-06-01 18:47:08 | Could not load 'tripleo_deploy': No module named 'tripleo_common.actions'
2021-06-01 18:47:08 | Could not load 'tripleo_upgrade': No module named 'tripleo_common.actions'
2021-06-01 18:47:09 | openstack: 'tripleo upgrade --templates --local-ip=192.168.24.1/24 --control-virtual-ip=192.168.24.3 -e /usr/share/openstack-tripleo-heat-templates/environments/standalone/standalone-tripleo.yaml -r /usr/share/openstack-tripleo-heat-templates/roles/Standalone.yaml -e /home/zuul/containers-prepare-parameters-upgrade.yaml -e /home/zuul/standalone_parameters_upgrade.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml --output-dir /home/zuul --standalone' is not an openstack command. See 'openstack --help'.
2021-06-01 18:47:09 | Did you mean one of these?
2021-06-01 18:47:09 | tripleo container image build
2021-06-01 18:47:09 | tripleo container image delete
2021-06-01 18:47:09 | tripleo container image hotfix
2021-06-01 18:47:09 | tripleo container image list
2021-06-01 18:47:09 | tripleo container image prepare
2021-06-01 18:47:09 | tripleo container image prepare default
2021-06-01 18:47:09 | tripleo container image push
2021-06-01 18:47:09 | tripleo container image show
2021-06-01 18:47:09 | tripleo launch heat
2021-06-01 18:47:09 | tripleo validator group info
2021-06-01 18:47:09 | tripleo validator list
2021-06-01 18:47:09 | tripleo validator run
2021-06-01 18:47:09 | tripleo validator show
2021-06-01 18:47:09 | tripleo validator show history
2021-06-01 18:47:09 | tripleo validator show parameter
2021-06-01 18:47:09 | tripleo validator show run
2021-06-01 18:47:09 | tsigkey create
2021-06-01 18:47:09 | tsigkey delete
2021-06-01 18:47:09 | tsigkey list
2021-06-01 18:47:09 | tsigkey set
2021-06-01 18:47:09 | tsigkey show
~~~

Job is failing since 29th May, build history:-
https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-upgrade-master&project=openstack/tripleo-ci

Revision history for this message
Rabi Mishra (rabi) wrote :

Using newer tripleo-common (29th May) and python3-tripleoclient(27th May) is missing https://review.opendev.org/c/openstack/python-tripleoclient/+/792137

Should be fixed with newer python3-tripleoclient

python3-tripleo-common-16.1.0-0.20210529122603.9026107.el8.noarch
python3-tripleoclient-16.2.1-0.20210527122904.fea5819.el8.noarch

https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-upgrade-master/9bd4b10/logs/undercloud/var/log/extra/rpm-list.txt.gz

Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :

Hello Rabi,

We are hitting a similar issue in the Master tripleo component, which has the latest python-tripleoclient

https://logserver.rdoproject.org/86/31586/64/check/periodic-tripleo-ci-centos-8-standalone-upgrade-tripleo-master/c08a30d/logs/undercloud/home/zuul/standalone_upgrade.log.txt.gz
~~~
2021-06-08 21:04:42 | openstack: 'tripleo upgrade --templates --local-ip=192.168.24.1/24 --control-virtual-ip=192.168.24.3 -e /usr/share/openstack-tripleo-heat-templates/environments/standalone/standalone-tripleo.yaml -r /usr/share/openstack-tripleo-heat-templates/roles/Standalone.yaml -e /home/zuul/containers-prepare-parameters-upgrade.yaml -e /home/zuul/standalone_parameters_upgrade.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml --output-dir /home/zuul --standalone' is not an openstack command. See 'openstack --help'.
~~~

https://logserver.rdoproject.org/86/31586/64/check/periodic-tripleo-ci-centos-8-standalone-upgrade-tripleo-master/c08a30d/logs/undercloud/var/log/extra/package-list-installed.txt.gz
~~~
openstack-tripleo-common.noarch 16.1.0-0.20210605151902.7e1436b.el8 @tripleo
python3-tripleoclient.noarch 16.2.1-0.20210608151413.c8870f5.el8 @delorean-current
~~~

Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :

Setting promotion blocker flag as this is blocking tripleo component promotion in Master.

tags: added: promotion-blocker
removed: alert
Revision history for this message
Rabi Mishra (rabi) wrote :

It's using master tripleo-common vs wallaby tripleoclient. I think because of the tag 16.2 in wallaby it's not upgrading tripleoclient.

Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :

I agree with Rabi observation:-

job is still using python3-tripleoclient.noarch version from wallaby

https://logserver.rdoproject.org/86/31586/64/check/periodic-tripleo-ci-centos-8-standalone-upgrade-tripleo-master/c08a30d/logs/undercloud/var/log/extra/package-list-installed.txt.gz
~~~
python3-tripleoclient.noarch 16.2.1-0.20210608151413.c8870f5.el8 @delorean-current
~~~

Wallaby:-
https://trunk.rdoproject.org/centos8-wallaby/component/tripleo/component-ci-testing/
~~~
python-tripleoclient-16.2.1-0.20210608151413.c8870f5.el8.src.rpm
~~~
Master:-
https://trunk.rdoproject.org/centos8-master/component/tripleo/component-ci-testing/
~~~
python-tripleoclient-16.1.0-0.20210608041413.c4261c4.el8.src.rpm
~~~

Revision history for this message
Sandeep Yadav (sandeepyadav93) wrote :

https://bugs.launchpad.net/tripleo/+bug/1931223 might be a duplicate

Revision history for this message
Marios Andreou (marios-b) wrote :

also possibly duplicate of https://bugs.launchpad.net/tripleo/+bug/1930416 (and similar to earlier https://bugs.launchpad.net/tripleo/+bug/1921679 )

strangely undercloud-upgrade is not hitting this [1] but standalone-upgrade still is [2]

[1] https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-undercloud-upgrade-master
[2] https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-upgrade-master

Revision history for this message
Marios Andreou (marios-b) wrote :

the (failing) standalone-upgrade is using 16.2.2 tripleoclient but the (passing) undercloud-upgrade is still with 16.1.0

        * https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-upgrade-master/dd2ddd0/logs/undercloud/var/log/extra/package-list-installed.txt.gz
        * python3-tripleoclient.noarch 16.2.2-0.20210609035220.48e5e28.el8 @delorean-current

        * https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-undercloud-upgrade-master/cafe190/logs/undercloud/var/log/extra/package-list-installed.txt.gz
        * python3-tripleoclient.noarch 16.1.0-0.20210609013907.6def2b9.el8 @delorean-current

Revision history for this message
Rabi Mishra (rabi) wrote :

To me it looks like periodic-tripleo-ci-centos-8-undercloud-upgrade-master is deploying standalone and from the package versions it seems to be upgrading from master to master.

Install
-------
2021-06-10T01:32:23+0000 DEBUG ---> Package python3-tripleo-common.noarch 16.1.0-0.20210608215442.e7a4de4.el8 will be installed <<<<< latest master hash
2021-06-10T01:32:23+0000 DEBUG ---> Package python3-tripleoclient.noarch 16.1.0-0.20210609013907.6def2b9.el8 will be installed

Upgrade:
--------
2021-06-10T02:18:04+0000 DDEBUG Command: dnf update -y python*-tripleoclient openstack-tripleo-common tripleo-ansible openstack-tripleo-heat-templates
2021-06-10T02:18:04+0000 DDEBUG Installroot: /
....
2021-06-10T02:18:07+0000 DEBUG --> Starting dependency resolution
2021-06-10T02:18:07+0000 DEBUG --> Finished dependency resolution
2021-06-10T02:18:07+0000 DDEBUG timer: depsolve: 209 ms
2021-06-10T02:18:07+0000 INFO Dependencies resolved.
2021-06-10T02:18:07+0000 INFO Nothing to do. <<<< Not updating anything
2021-06-10T02:18:07+0000 INFO Complete!

Revision history for this message
Marios Andreou (marios-b) wrote :

I suspect this might be solved with https://review.opendev.org/c/openstack/releases/+/795948 via https://bugs.launchpad.net/tripleo/+bug/1931223/comments/12 i.e. we need a tag for master so it will try to install tripleoclient 17.x from/for master

Revision history for this message
Marios Andreou (marios-b) wrote :

We need a promotion of the tripleo component and it should clear the bug.

Now that https://review.opendev.org/c/openstack/releases/+/795948 merged we have a newer tripleoclient tagged in the tripleo component consistent repo there:

https://trunk.rdoproject.org/centos8-master/component/tripleo/consistent/
python-tripleoclient-17.0.1-0.20210615072306.7797524.el8.src.rpm 2021-06-15 07:24 338K

We need the component tripleo promotion as https://trunk.rdoproject.org/centos8-master/component/tripleo/current-tripleo/ - current-tripleo is still on python-tripleoclient-16.0.1-0.20210520134358.e86bccd.el8.src.rpm at time of writing (and we want the latest version from consistent)

Revision history for this message
Marios Andreou (marios-b) wrote :

Confirming comment #11 the tripleo component promotion will help us.

In today's run the standalone-upgrade job [1] went passed this error but failed on tempest and it is using the newer tripleoclient [2]

        * 2021-06-15T20:18:55+0000 DEBUG ---> Package python3-tripleoclient.noarch 17.0.1-0.20210615072306.7797524.el8 will be an upgrade

I am trying to get a re-run of that job with https://review.rdoproject.org/r/c/testproject/+/34130 to force that promotion let's see.

[1] https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-upgrade-tripleo-master/5b497e2/job-output.txt
[2] https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-upgrade-tripleo-master/5b497e2/logs/undercloud/var/log/dnf.log.txt.gz

Revision history for this message
Marios Andreou (marios-b) wrote :

managed to get a promotion on the tripleo component (with https://review.rdoproject.org/r/c/testproject/+/34194 ) so we now have the right version of tripleoclient in the promoted-components:

        * https://trunk.rdoproject.org/centos8-master/component/tripleo/promoted-components/
        * python-tripleoclient-17.0.1-0.20210616113250.15f2539.el8.src.rpm 2021-06-16 11:34 338K

So now we need a master promotion so we can get that ^^^ into current-tripleo and we can close the bug

wes hayutin (weshayutin)
Changed in tripleo:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.