OpenStack Compute (nova)

The placement service for 192.168.24.100:regionOne exists but does not have any supported versions.

Bug #1949991 reported by Jiri Podivin
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
New
Undecided
Unassigned

Bug Description

Nova Conductor container running under featureset 010 fails with:

"The placement service for 192.168.24.100:regionOne exists but does not have any supported versions."

The nova-conductor fails a health check slightly before that, but I'm not certain about connection between the two events.

As of now I don't know what is the cause of the error or how it is related to the featureset and nodeset used.

Release: master

Trace:
------

2021-10-29 16:37:54.218 9 CRITICAL nova [-] Unhandled error: openstack.exceptions.NotSupported: The placement service for 192.168.24.100:regionOne exists but does not have any supported versions.
2021-10-29 16:37:54.218 9 ERROR nova Traceback (most recent call last):
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/bin/nova-conductor", line 10, in <module>
2021-10-29 16:37:54.218 9 ERROR nova sys.exit(main())
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/cmd/conductor.py", line 46, in main
2021-10-29 16:37:54.218 9 ERROR nova topic=rpcapi.RPC_TOPIC)
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/service.py", line 256, in create
2021-10-29 16:37:54.218 9 ERROR nova periodic_interval_max=periodic_interval_max)
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/service.py", line 116, in __init__
2021-10-29 16:37:54.218 9 ERROR nova self.manager = manager_class(host=self.host, *args, **kwargs)
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 119, in __init__
2021-10-29 16:37:54.218 9 ERROR nova self.compute_task_mgr = ComputeTaskManager()
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/conductor/manager.py", line 244, in __init__
2021-10-29 16:37:54.218 9 ERROR nova self.report_client = report.SchedulerReportClient()
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 187, in __init__
2021-10-29 16:37:54.218 9 ERROR nova self._client = self._create_client()
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 230, in _create_client
2021-10-29 16:37:54.218 9 ERROR nova client = self._adapter or utils.get_sdk_adapter('placement')
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/nova/utils.py", line 983, in get_sdk_adapter
2021-10-29 16:37:54.218 9 ERROR nova return getattr(conn, service_type)
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__
2021-10-29 16:37:54.218 9 ERROR nova proxy = self._make_proxy(instance)
2021-10-29 16:37:54.218 9 ERROR nova File "/usr/lib/python3.6/site-packages/openstack/service_description.py", line 271, in _make_proxy
2021-10-29 16:37:54.218 9 ERROR nova region_name=region_name))
2021-10-29 16:37:54.218 9 ERROR nova openstack.exceptions.NotSupported: The placement service for 192.168.24.100:regionOne exists but does not have any supported versions.
2021-10-29 16:37:54.218 9 ERROR nova

Logs:
-----
https://logserver.rdoproject.org/99/36499/6/check/periodic-tripleo-ci-centos-8-containers-multinode-compute-master-validation/91262ed/logs/subnode-1/var/log/containers/nova/nova-conductor.log.txt.gz

https://logserver.rdoproject.org/99/36499/6/check/periodic-tripleo-ci-centos-8-containers-multinode-compute-master-validation/91262ed/logs/subnode-1/var/log/extra/failed_services.txt.gz

Full logs for the job: https://logserver.rdoproject.org/99/36499/6/check/periodic-tripleo-ci-centos-8-containers-multinode-compute-master-validation/91262ed/

Jiri Podivin (jpodivin)
tags: added: conductor placement
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

Is this still happening? I do remember an Apache change that caused a difference in trailing / handling in the proxy configuration [1]. I feel this might be related.

[1] https://review.opendev.org/c/openstack/devstack/+/811303

Changed in nova:
status: New → Incomplete
Revision history for this message
Ronelle Landy (rlandy) wrote :

https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-standalone-on-multinode-ipa&skip=0

does not reproduce on check:

https://ef2ad358dccb56dd5b1d-616db0a32cd5a8455c371f4b45f3133e.ssl.cf5.rackcdn.com/828144/4/check/tripleo-ci-centos-9-standalone-on-multinode-ipa/e5d5640/logs/undercloud/var/log/tempest/stestr_results.html

Revision history for this message
Ronelle Landy (rlandy) wrote :

sorry - posted on wrong bug above

Jiri Podivin (jpodivin)
Changed in nova:
status: Incomplete → Invalid
Revision history for this message
Mario Chirinos (mario-chirinos) wrote :

I have the same problem, did you find a solution?

Revision history for this message
ZhangLong (ankele) wrote (last edit ):

I met the same problem, and I find the placement database is empty, even after "placement-manage db sync". And I find my /etc/placement/placement.conf is empty. Configure it correctly, and restart placement-api wsgi service, I got success.

Revision history for this message
Moises Emilio Benzan Mora (moisesbenzan) wrote :
Download full text (8.0 KiB)

Hello, SQA team found this bug during our testing.

From running the tempest suite, we get the following error:
=================================================================================================================
 tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers
fail 110.351
3b55a32e-4841-48b3-a3aa-050d261129c1

Traceback (most recent call last):
  File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/api/compute/servers/test_attach_interfaces.py", line 337, in test_reassign_port_between_servers
    iface = self.interfaces_client.create_interface(
  File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/services/compute/interfaces_client.py", line 47, in create_interface
    resp, body = self.post('servers/%s/os-interface' % server_id,
  File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/common/rest_client.py", line 300, in post
    return self.request('POST', url, extra_headers, headers, body, chunked)
  File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/services/compute/base_compute_client.py", line 47, in request
    resp, resp_body = super(BaseComputeClient, self).request(
  File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/common/rest_client.py", line 742, in request
    self._error_checker(resp, resp_body)
  File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/common/rest_client.py", line 922, in _error_checker
    raise exceptions.ServerFault(resp_body, resp=resp,
tempest.lib.exceptions.ServerFault: Got server fault
Details: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception_Remote.PortBindingFailed_Remote'>

=====================================================================================================================

Looking into the Nova API logs, we then get:
=====================================================================================================================
2023-05-30 22:27:37.882 395255 ERROR nova
2023-05-30 22:27:49.883 404005 INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs
2023-05-30 22:28:00.147 404005 WARNING nova.conductor.api [req-01768358-b6bc-4503-82e6-2b74b5a8e7be - - - - -] Timed out waiting for nova-conductor. Is it running? Or did this service start before nova-
conductor? Reattempting establishment of nova-conductor connection...: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 4f27a62457e74f2783f79b86415da0f7
=====================================================================================================================

And taking a look at the nova-conductor logs, it shows that around that same time it was failing to initialize the placement client:
=====================================...

Read more...

Changed in nova:
status: Invalid → New
tags: added: cdo-qa cdo-temptest foundations-engine
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.