Hello, SQA team found this bug during our testing. From running the tempest suite, we get the following error: ================================================================================================================= tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers fail 110.351 3b55a32e-4841-48b3-a3aa-050d261129c1 Traceback (most recent call last): File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/api/compute/servers/test_attach_interfaces.py", line 337, in test_reassign_port_between_servers iface = self.interfaces_client.create_interface( File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/services/compute/interfaces_client.py", line 47, in create_interface resp, body = self.post('servers/%s/os-interface' % server_id, File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/common/rest_client.py", line 300, in post return self.request('POST', url, extra_headers, headers, body, chunked) File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/services/compute/base_compute_client.py", line 47, in request resp, resp_body = super(BaseComputeClient, self).request( File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/common/rest_client.py", line 742, in request self._error_checker(resp, resp_body) File "/home/ubuntu/snap/fcbtest/50/.rally/verification/verifier-69ebebc9-78df-4ad7-9740-23b4f54ebb96/repo/tempest/lib/common/rest_client.py", line 922, in _error_checker raise exceptions.ServerFault(resp_body, resp=resp, tempest.lib.exceptions.ServerFault: Got server fault Details: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. ===================================================================================================================== Looking into the Nova API logs, we then get: ===================================================================================================================== 2023-05-30 22:27:37.882 395255 ERROR nova 2023-05-30 22:27:49.883 404005 INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs 2023-05-30 22:28:00.147 404005 WARNING nova.conductor.api [req-01768358-b6bc-4503-82e6-2b74b5a8e7be - - - - -] Timed out waiting for nova-conductor. Is it running? Or did this service start before nova- conductor? Reattempting establishment of nova-conductor connection...: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 4f27a62457e74f2783f79b86415da0f7 ===================================================================================================================== And taking a look at the nova-conductor logs, it shows that around that same time it was failing to initialize the placement client: ===================================================================================================================== 2023-05-30 22:28:32.618 237056 ERROR nova.scheduler.client.report [-] Failed to initialize placement client (is keystone available?): openstack.exceptions.NotSupported: The placement service for keystone -internal.silo5.lab1.solutionsqa:RegionOne exists but does not have any supported versions. 2023-05-30 22:28:32.618 237056 ERROR nova.conductor.manager [-] Fatal error initializing placement client: The placement service for keystone-internal.silo5.lab1.solutionsqa:RegionOne exists but does not have any supported versions.: openstack.exceptions.NotSupported: The placement service for keystone-internal.silo5.lab1.solutionsqa:RegionOne exists but does not have any supported versions. 2023-05-30 22:28:32.620 237056 CRITICAL nova [-] Unhandled error: openstack.exceptions.NotSupported: The placement service for keystone-internal.silo5.lab1.solutionsqa:RegionOne exists but does not have any supported versions. 2023-05-30 22:28:32.620 237056 ERROR nova Traceback (most recent call last): 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/bin/nova-conductor", line 10, in 2023-05-30 22:28:32.620 237056 ERROR nova sys.exit(main()) 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/cmd/conductor.py", line 45, in main 2023-05-30 22:28:32.620 237056 ERROR nova server = service.Service.create(binary='nova-conductor', 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/service.py", line 252, in create 2023-05-30 22:28:32.620 237056 ERROR nova service_obj = cls(host, binary, topic, manager, 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/service.py", line 116, in __init__ 2023-05-30 22:28:32.620 237056 ERROR nova self.manager = manager_class(host=self.host, *args, **kwargs) 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 122, in __init__ 2023-05-30 22:28:32.620 237056 ERROR nova self.compute_task_mgr = ComputeTaskManager() 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 253, in __init__ 2023-05-30 22:28:32.620 237056 ERROR nova self.report_client 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 281, in report_client 2023-05-30 22:28:32.620 237056 ERROR nova return report.report_client_singleton() 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/scheduler/client/report.py", line 91, in report_client_singleton 2023-05-30 22:28:32.620 237056 ERROR nova PLACEMENTCLIENT = SchedulerReportClient() 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/scheduler/client/report.py", line 234, in __init__ 2023-05-30 22:28:32.620 237056 ERROR nova self._client = self._create_client() 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/scheduler/client/report.py", line 277, in _create_client 2023-05-30 22:28:32.620 237056 ERROR nova client = self._adapter or utils.get_sdk_adapter('placement') 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/nova/utils.py", line 985, in get_sdk_adapter 2023-05-30 22:28:32.620 237056 ERROR nova return getattr(conn, service_type) 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/openstack/service_description.py", line 87, in __get__ 2023-05-30 22:28:32.620 237056 ERROR nova proxy = self._make_proxy(instance) 2023-05-30 22:28:32.620 237056 ERROR nova File "/usr/lib/python3/dist-packages/openstack/service_description.py", line 266, in _make_proxy 2023-05-30 22:28:32.620 237056 ERROR nova raise exceptions.NotSupported( 2023-05-30 22:28:32.620 237056 ERROR nova openstack.exceptions.NotSupported: The placement service for keystone-internal.silo5.lab1.solutionsqa:RegionOne exists but does not have any supported versions. 2023-05-30 22:28:32.620 237056 ERROR nova 2023-05-30 22:28:36.286 237411 WARNING keystoneauth.discover [-] Failed to contact the endpoint at http://placement-internal.silo5.lab1.solutionsqa:8778 for discovery. Fallback to using that endpoint as the base url. 2023-05-30 22:28:36.304 237411 WARNING keystoneauth.discover [-] Failed to contact the endpoint at http://placement-internal.silo5.lab1.solutionsqa:8778 for discovery. Fallback to using that endpoint as the base url. ===================================================================================================================== Test Run can be found here: https://solutions.qa.canonical.com/v2/testruns/9ea628e9-d51b-4327-82c6-663988079165/ Artifacts for the test run: https://oil-jenkins.canonical.com/artifacts/9ea628e9-d51b-4327-82c6-663988079165/index.html Further occurrences of that bug in SQA labs: https://solutions.qa.canonical.com/v2/bugs/1949991