[wallaby][multinode][heat-ephemeral] Randomly Failing with Unexpected error occurred serving API: Timed out waiting for a reply to message ID
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tripleo |
Fix Released
|
Medium
|
Unassigned |
Bug Description
Noticed in https:/
2021-06-04 07:14:20 | 2021-06-04 07:14:20.461 172008 INFO tripleoclient.
2021-06-04 07:14:20 | 5888bbe2118be17
2021-06-04 07:14:22 | 5888bbe2118be17
2021-06-04 07:14:22 | 2021-06-04 07:14:22.973 172008 ERROR tripleoclient.
2021-06-04 07:14:22 | Traceback (most recent call last):
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return self._queues[
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return waiter.wait()
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return get_hub().switch()
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return self.greenlet.
2021-06-04 07:14:22 | queue.Empty
2021-06-04 07:14:22 |
2021-06-04 07:14:22 | During handling of the above exception, another exception occurred:
2021-06-04 07:14:22 |
2021-06-04 07:14:22 | Traceback (most recent call last):
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return req.get_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | application, catch_exc_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | app_iter = application(
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return self.app(env, start_response)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | resp = self.call_func(req, *args, **kw)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return self.func(req, *args, **kwargs)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | response = req.get_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | application, catch_exc_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | app_iter = application(
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | resp = self.call_func(req, *args, **kw)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return self.func(req, *args, **kwargs)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | response = req.get_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | application, catch_exc_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | app_iter = application(
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return resp(environ, start_response)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | response = self.app(environ, start_response)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return resp(environ, start_response)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | resp = self.call_func(req, *args, **kw)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return self.func(req, *args, **kwargs)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | raise translate_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | request, **action_args)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return method(*args, **kwargs)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return handler(controller, req, **kwargs)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | ignorable_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | version='1.36')
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | return client.call(ctxt, method, **kwargs)
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | transport_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | transport_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | transport_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | call_monitor_
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | message = self.waiters.
2021-06-04 07:14:22 | File "/usr/lib/
2021-06-04 07:14:22 | 'to message ID %s' % msg_id)
2021-06-04 07:14:22 | oslo_messaging.
Logs:-
https:/
https:/
Changed in tripleo: | |
status: | New → Triaged |
importance: | Undecided → Medium |
milestone: | none → xena-1 |
python3- tripleoclient- 16.2.0- 1.el8.noarch[ 1] does not have https:/ /review. opendev. org/c/openstack /python- tripleoclient/ +/791856
heat-api is processing requests before the queues are created, heat-engines listening on them. When it's trying to put messages in the non-existing queues requests would timeout.
We probably need a new tagged release of tripleoclient and tripleo-common for CBS?
[1] tripleoclient noarch 16.2.0-1.el8 temp-cloud8s- openstack- wallaby- testing 533 k
python3-