collect-logs: ovb jobs are not collecting logs from all nodes

Bug #1944617 reported by wes hayutin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
Rabi Mishra (rabi) wrote :

STDERR:

Traceback (most recent call last):
  File "/home/zuul/get-overcloud-nodes.py", line 60, in <module>
    servers_list = provisioner.list_instances()
  File "/usr/lib/python3.6/site-packages/metalsmith/_provisioner.py", line 621, in list_instances
    nodes = self.connection.baremetal.nodes(associated=True, details=True)
  File "/usr/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__
    proxy = self._make_proxy(instance)
  File "/usr/lib/python3.6/site-packages/openstack/service_description.py", line 271, in _make_proxy
    region_name=region_name))
openstack.exceptions.NotSupported: The baremetal service for :None exists but does not have any supported versions.

MSG:

non-zero return code
...ignoring

TASK [tripleo-inventory : list the overcloud nodes] ************************

Revision history for this message
Arx Cruz (arxcruz) wrote :

I was debugging the code, and I wasn't able to find any change except for the overcloudrc content, on the no_proxy, the passing one starts with a comma ",":

https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby/cdd50eb/logs/undercloud/home/zuul/overcloudrc.gz

export no_proxy=,10.0.0.5,192.168.24.8

While the failing one doesn't start with comma:

https://logserver.rdoproject.org/56/810456/1/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/e0fa90a/logs/undercloud/home/zuul/overcloudrc.gz

export no_proxy=10.0.0.5,192.168.24.17

Also, the output:

openstack.exceptions.NotSupported: The baremetal service for :None exists but does not have any supported versions.

On the other side, the failing one have this content in the clouds.yaml file:

https://logserver.rdoproject.org/55/810455/1/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/13a0da0/logs/undercloud/etc/openstack/clouds.yaml.txt.gz

clouds:
  overcloud:
    auth:
      auth_url: https://10.0.0.5:13000
      password: NF9Ku9gWIJtJ2cYWRl89FSCT6
      project_domain_name: Default
      project_name: admin
      user_domain_name: Default
      username: admin
    cacert: ''
    identity_api_version: '3'
    region_name: regionOne
    volume_api_version: '3'
  overcloud-system-admin:
    auth:
      auth_url: https://10.0.0.5:13000
      password: NF9Ku9gWIJtJ2cYWRl89FSCT6
      system_scope: all
      user_domain_name: Default
      username: admin
    cacert: ''
    identity_api_version: '3'
    region_name: regionOne
    volume_api_version: '3'
  undercloud:
    auth:
      password: 0nZOcJCUT0xB6IWvIOeAmHfOz
      username: admin
    auth_type: http_basic
    baremetal_api_version: '1'
    baremetal_endpoint_override: https://192.168.24.2:13385
    baremetal_introspection_endpoint_override: https://192.168.24.2:13050
    network_api_version: '2'
    network_endpoint_override: https://192.168.24.2:13696

Where you can see the baremetal_api_version set to 1 and the endpoints as well...

While the passing one have a more simple version:

https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby/cdd50eb/logs/undercloud/etc/openstack/clouds.yaml.txt.gz

clouds:
  overcloud:
    auth:
      auth_url: https://10.0.0.5:13000
      password: sYUl8cWtNtl9VnlCqUHBbWdLB
      project_domain_name: Default
      project_name: admin
      user_domain_name: Default
      username: admin
    cacert: ''
    identity_api_version: '3'
    region_name: regionOne
    volume_api_version: '3'
  undercloud:
    auth:
      auth_url: https://192.168.24.2:13000
      password: aA0O2DrP02qGhoODd6JzFc8rl
      project_domain_name: Default
      project_name: admin
      user_domain_name: Default
      username: admin
    cacert: /etc/pki/ca-trust/source/anchors/cm-local-ca.pem
    identity_api_version: '3'
    region_name: regionOne
    volume_api_version: '3'

Revision history for this message
Arx Cruz (arxcruz) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-quickstart (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart/+/810546
Committed: https://opendev.org/openstack/tripleo-quickstart/commit/b68aefbb695f575e9cecadb02e8a9ccd52f8ce9c
Submitter: "Zuul (22348)"
Branch: master

commit b68aefbb695f575e9cecadb02e8a9ccd52f8ce9c
Author: rabi <email address hidden>
Date: Thu Sep 23 11:54:47 2021 +0530

    Fix get-overcloud-nodes.py

    Now that we don't use keystone in the undercloud, we need to set
    the connection object for the provisioner for it to work

    Closes-Bug: #1944617
    Related-Bug: #1944416
    Change-Id: I0a67f9fb26fda9aec161e4593c75a8110b64b05d

Changed in tripleo:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.