"sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 50 reached, connection timed out, timeout 30.00" error raised after repeated calls of Flavor.get_* methods
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
Undecided
|
melanie witt | ||
Antelope |
New
|
Undecided
|
Unassigned | ||
Wallaby |
New
|
Undecided
|
Unassigned | ||
Xena |
New
|
Undecided
|
Unassigned | ||
Yoga |
New
|
Undecided
|
Unassigned | ||
Zed |
New
|
Undecided
|
Unassigned |
Bug Description
This bug was reported downstream by a user who is using some automation to query nova API for periodically for flavor information and occasionally they receive a HTTP 500 error from nova API with a message related to database connection pools.
The error:
sqlalchemy.
is being raised in nova-api, causing a 500 error to be returned.
I think this is happening because of the placement of the @api_db_
The database context manager decorator needs to be on the methods that execute the queries because part of what it does is close connections after the method is run.
Full traceback:
Jul 13 22:06:48 ubuntu-jammy <email address hidden>[270259]: DEBUG nova.api.
Jul 13 22:06:58 ubuntu-jammy <email address hidden>[270259]: DEBUG dbcounter [-] [270259] Writing DB stats nova_api:SELECT=2 {{(pid=270259) stat_writer /usr/local/
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: ERROR nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: INFO nova.api.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: <class 'sqlalchemy.
Jul 13 22:07:18 ubuntu-jammy <email address hidden>[270259]: DEBUG nova.api.
description: | updated |
Patch is proposed at https:/ /review. opendev. org/c/openstack /nova/+ /888513