tripleo

wallaby gate blocker tripleo-ci-centos-8-standalone ERROR: Cannot install stackviz

Bug #1976247 reported by Marios Andreou
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned
tripleo zed-1

Bug Description

At [1][2] the tripleo-ci-centos-8-standalone fails for stable/wallaby jobs and blocks the gate. During the os_tempest role execution, the python_venv_build role runs and fails during python package installs with conflict - trace like:

        2022-05-30 08:55:06.326662 | primary | fatal: [undercloud]: FAILED! => {"attempts": 5, "changed": false, "cmd": ["/home/zuul/stackviz_venv/bin/pip", "install", "-U", "--constraint", "/home/zuul/stackviz_venv/global-constraints.txt", "--constraint", "/home/zuul/stackviz_venv/constraints.txt", "--pre", "--log", "/var/log/python_venv_build.log", "--isolated", "https://tarballs.opendev.org/openstack/stackviz/dist/stackviz-latest.tar.gz"], "msg": "stdout: Looking in indexes: https://mirror.mtl01.iweb.opendev.org/pypi/simple, https://mirror.mtl01.iweb.opendev.org/wheel/centos-8-x86_64\nCollecting https://tarballs.opendev.org/openstack/stackviz/dist/stackviz-latest.tar.gz\n Using cached https://tarballs.opendev.org/openstack/stackviz/dist/stackviz-latest.tar.gz (764 kB)\n Preparing metadata (setup.py): started\n Preparing metadata (setup.py): finished with status 'done'\nCollecting python-subunit>=0.0.18\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/88/de/be89fff5c30810afb101bdac2c8706764fafa8e950dc6ffa22092ad81035/python_subunit-1.4.0-py2.py3-none-any.whl (106 kB)\nCollecting testtools>=0.9.30\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/c0/49/b2b4956528cca6954cb3a8016a8283282ccd1a1d66ab1c2d1bbde3f66946/testtools-2.5.0-py3-none-any.whl (181 kB)\nCollecting testrepository>=0.0.18\n Using cached https://mirror.mtl01.iweb.opendev.org/wheel/centos-8-x86_64/testrepository/testrepository-0.0.20-py3-none-any.whl (118 kB)\nCollecting subunit2sql>=0.8.0\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/0a/1b/e1aba9b1ea76c1818d1e629f2071b9604321d8bf85fef1128e6b4c7aaaa6/subunit2sql-1.10.0-py2.py3-none-any.whl (98 kB)\nCollecting six\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl (11 kB)\nCollecting extras\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/03/e9/e915af1f97914cd0bc021e125fd1bfd4106de614a275e4b6866dd9a209ac/extras-1.0.0-py2.py3-none-any.whl (7.3 kB)\nCollecting stevedore>=1.20.0\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/7a/bc/fcce9e50da73ea23af6d236e05e15db8a02da1099a5e0a479451bcea3833/stevedore-3.5.0-py3-none-any.whl (49 kB)\nCollecting python-dateutil>=2.4.2\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/36/7a/87837f39d0296e723bb9b62bbb257d0355c7f6128853c78955f57342a56d/python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)\nCollecting alembic>=0.8.10\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/b3/e2/8d48220731b7279911c43e95cd182961a703b939de6822b00de3ea0d3159/alembic-1.7.7-py3-none-any.whl (210 kB)\nCollecting oslo.db>=4.24.0\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/29/85/bbb2d8d0cc79c45d3d61ccaf257cbac722628ad602e2d9551658b456a06b/oslo.db-11.3.0-py3-none-any.whl (175 kB)\nCollecting SQLAlchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8,>=1.0.10\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/2d/57/b2bf4c7adb045ce05f4f2648e7938aef96bc2909be433b1c650071f6a466/SQLAlchemy-1.4.36-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)\nCollecting oslo.config>=4.0.0\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/c5/4c/6cfd9274f3fb665f276ec41bd7c98fce20d5ec903bac7b4cd2c30f5832fe/oslo.config-8.8.0-py3-none-any.whl (128 kB)\nCollecting pbr!=2.1.0,>=2.0.0\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/1b/ef/0621aa0e422dda72dc89b61972e5a28d5a947c22fdff83a5d69fdbc1949a/pbr-5.9.0-py2.py3-none-any.whl (112 kB)\nCollecting fixtures\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/a8/28/7eed6bf76792f418029a18d5b2ace87ce7562927cdd00f1cefe481cd148f/fixtures-3.0.0-py2.py3-none-any.whl (67 kB)\nINFO: pip is looking at multiple versions of testtools to determine which version is compatible with other requirements. This could take a while.\nINFO: pip is looking at multiple versions of testrepository to determine which version is compatible with other requirements. This could take a while.\nINFO: pip is looking at multiple versions of six to determine which version is compatible with other requirements. This could take a while.\nINFO: pip is looking at multiple versions of subunit2sql to determine which version is compatible with other requirements. This could take a while.\nCollecting subunit2sql>=0.8.0\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/30/ec/ba2964b2040f6700429f640632f16bce1dacd6d2ee254592bc2ba42fa6c3/subunit2sql-1.9.0-py2.py3-none-any.whl (99 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/35/bd/cfb11e9d44af8684c929f1cea930211f38a9339adeb7f34931425d0da938/subunit2sql-1.8.0-py2.py3-none-any.whl (95 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/f5/17/1b5f9e0aa269b66f2f807f1e0d339ec0973637e325ebd82fa49ab34d6e0a/subunit2sql-1.7.0-py2.py3-none-any.whl (92 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/da/06/eb6cfc28025642bf1ebf60c07ee8c49c94d310b4c51f64e924dc036fad68/subunit2sql-1.6.0-py2.py3-none-any.whl (91 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/dd/4c/35d6af4579f3cd45a11f492a2fe437a0d906185425525dbe26c6a1a2487a/subunit2sql-1.5.0-py2.py3-none-any.whl (90 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/06/9d/2323fed7e01d7ebf4dce99939ba9b69ace397b8f2caa4837bc74e5460344/subunit2sql-1.4.0-py2.py3-none-any.whl (89 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/41/a5/bca5b2a165e89790a590f2419baa8b4f59fc56baafcb5de6721adce48b12/subunit2sql-1.3.0-py2.py3-none-any.whl (87 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/90/48/1dee5c9e2aaadaedcd44892e6ba9a34b44fda1808cf492d9a04a73942db9/subunit2sql-1.2.0-py2.py3-none-any.whl (86 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/6d/5f/e8872f7bc5c83b96760101e196c226ebaa385407ac5e4aae2eb8d7f4ac8b/subunit2sql-1.1.1-py2.py3-none-any.whl (80 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/d7/f0/49d1b41e6b2bd0965788b22d44568f41839e12ced09b0a9e9bf2a2eb9107/subunit2sql-1.1.0-py2.py3-none-any.whl (79 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/43/2e/117aaf93d62edccc9504edc9ee1a40cff4d6a353330395d3497de3889336/subunit2sql-1.0.2-py2.py3-none-any.whl (79 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/cc/7e/5b2af1012b0db866b01d4f1d3820f70796e29f85fbec7d839bf28aebe405/subunit2sql-1.0.1-py2.py3-none-any.whl (78 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/dc/51/ec67d653e737d1f599cb9dbb86966d3adbad9ffbf735d093d2e4c390f4d4/subunit2sql-1.0.0-py2.py3-none-any.whl (77 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/ad/a4/bdb40b5ff8195f124db94f22917dd1b5fdbeb8b8f4b77f59df732cb54136/subunit2sql-0.12.0-py2.py3-none-any.whl (69 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/19/a8/dd791dff6915abc0d27fcfdb1b4a41d1ddb2713ed2ef70b9cb6d86567913/subunit2sql-0.11.0-py2.py3-none-any.whl (67 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/b1/fa/8c73ce63c08ec08acee63d9d46ba13f327d661ad9ebe6cf397f1aed82541/subunit2sql-0.10.0-py2.py3-none-any.whl (62 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/1f/6e/8d367e3835bcec033bd326898ead6be6de590670a92fbf7de4ba797df6e4/subunit2sql-0.9.0-py2.py3-none-any.whl (60 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/a2/0d/b312dc04d10331b1ef218f5b1bcd87b22c581bf3e8dbcca628ed567ab9a1/subunit2sql-0.8.1-py2.py3-none-any.whl (59 kB)\n Using cached https://mirror.mtl01.iweb.opendev.org/pypifiles/packages/f6/51/2dd80a612450b9d929d8f6981f0353edf6ae4dc485f6116b615c1eab94c8/subunit2sql-0.8.0-py2.py3-none-any.whl (59 kB)\nINFO: pip is looking at multiple versions of python-subunit to determine which version is compatible with other requirements. This could take a while.\nINFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.\nINFO: pip is looking at multiple versions of stackviz to determine which version is compatible with other requirements. This could take a while.\n\nThe conflict is caused by:\n subunit2sql 1.5.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.4.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.3.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.2.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.1.1 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.1.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.0.2 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.0.1 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 1.0.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 0.12.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 0.11.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 0.10.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 0.9.0 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 0.8.1 depends on oslo.db!=1.12.0 and <2.0.0\n subunit2sql 0.8.0 depends on oslo.db!=1.12.0 and <2.0.0\n The user requested (constraint) oslo-db===11.3.0\n\nTo fix this you could try to:\n1. loosen the range of package versions you've specified\n2. remove package versions to allow pip attempt to solve the dependency conflict\n\n\n:stderr: ERROR: Cannot install stackviz because these package versions have conflicting dependencies.\nERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies\n"}

This issue is also reported at [3].

Thanks @ysandeep for the info indeed this must be related to py36 support.

For master we dropped it with [4]. For stable/wallaby we need it for centos8 jobs, or we need to find another way to overcome that conflict.

[1] https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3af/840477/1/check/tripleo-ci-centos-8-standalone/3af6916/job-output.txt
[2] https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b7b/839184/3/gate/tripleo-ci-centos-8-standalone/b7bc8d9/job-output.txt
[3] https://bugzilla.redhat.com/show_bug.cgi?id=2091502
[4] https://review.opendev.org/q/topic:tripleo-zed-jobs

Revision history for this message
Marios Andreou (marios-b) wrote :

Looks like we're using the *master* upper constraints here [1]

        * oslo.db===11.3.0

But we should be using the wallaby one at [2]

        * oslo.db===8.5.1

[1] https://github.com/openstack/requirements/blob/7ac0e631833ee8c33b42ddbcc23be89c05dcc658/upper-constraints.txt#L478

[2] https://github.com/openstack/requirements/blob/20be78e7b3ef4c3e43ad06745e5ffcb0f1a87576/upper-constraints.txt#L466

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-quickstart-extras (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843841

Revision history for this message
Marios Andreou (marios-b) wrote :

On the ansible-os_tempest role the default at [1] is pulling in master

i am trying to override it for the tripleo side with zuul.branch at [2] and testing with [3]

[1] https://opendev.org/openstack/openstack-ansible-os_tempest/src/commit/1869a9f138bcbbdbb53f2686cd567f71e073472d/defaults/main.yml#L61-L62

[2] https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843841

[3] https://review.opendev.org/c/openstack/tripleo-ansible/+/843842

Revision history for this message
Marios Andreou (marios-b) wrote :

no this is not coming from ansible-os_tempest but from python _venv_build (called by the tempest role):

2022-05-30 08:54:00.578554 | primary | TASK [python_venv_build : Install python packages into the venv] ***************

so still digging...

Revision history for this message
Marios Andreou (marios-b) wrote :

coming from there https://opendev.org/openstack/ansible-role-python_venv_build/src/commit/e7307f5d01db07cc504eb619654a0829c9d4e92c/tasks/python_venv_install.yml#L152

Revision history for this message
Marios Andreou (marios-b) wrote (last edit ):

we may end up disabling stackviz to unblock the gate

the code in ansible-role-python_venv is a bit convoluted wrt the defaults e.g.

- name: Install python packages into the venv
  block:
    - name: Install python packages into the venv
      pip:
        name: "{{ _venv_install_pip_packages }}"
        state: "{{ venv_pip_package_state }}"
        virtualenv: "{{ venv_install_destination_path }}"
        extra_args: >-
          --constraint {{ venv_install_destination_path }}/global-constraints.txt
          --constraint {{ venv_install_destination_path }}/constraints.txt

i've tried to trace how the _venv_install_pip_packages (actually defined in this same file [1]) and constraints.txt and global-constraints.txt are formed and it is not obvious to follow :/

As there are other things that need pressing attention I am contemplating disabling this/stackviz as we already have for c9 already there [2] (thanks chkumar for reminder)

But not even sure if that will work. The earlier fix we have at [2] is for os_tempest itself whereas the failing task i nthis case is from ansible-role-python-venv_build

[1] https://opendev.org/openstack/ansible-role-python_venv_build/src/commit/e7307f5d01db07cc504eb619654a0829c9d4e92c/tasks/python_venv_install.yml#L29
[2] https://opendev.org/openstack/tripleo-quickstart-extras/src/commit/be5e48aeedf5ecd5ee4acdbed7c4fa1be2db0d50/playbooks/tasks/tempest.yml#L87

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843856

Revision history for this message
Marios Andreou (marios-b) wrote :

trying https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843856 & test @ https://review.opendev.org/c/openstack/tripleo-ansible/+/843857 (for disable)

also posted blocker warning to list at http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028750.html [tripleo] stable/wallaby gate blocker for centos-8 standalone - please hold recheck

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-quickstart-extras (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843856
Committed: https://opendev.org/openstack/tripleo-quickstart-extras/commit/8bcf4bfc0f2596776ca6abf284cce6337e097e3d
Submitter: "Zuul (22348)"
Branch: master

commit 8bcf4bfc0f2596776ca6abf284cce6337e097e3d
Author: Marios Andreou <email address hidden>
Date: Mon May 30 16:06:35 2022 +0300

    Disable stackviz for tempest centos 8 to unblock gate

    As commented in related bug (see comments/6) we need to disable
    installation of stackviz as there are issues with the constraints
    used. This is not a proper fix but if it works can unblock for now

    Related-Bug: 1976247

    Change-Id: I8c5e168a9fd18332c073223726ad3a3e338af04d

Revision history for this message
Marios Andreou (marios-b) wrote :

we are unblocked for now with https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843856

moving to fix released for now

there will be more investigation needed if we want to re-enable stackviz

Changed in tripleo:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.