Hello
My team has a large timescale database, from which we query often, and some of the queries might take up to 5-7 seconds.
We use the python databases package to connect to the db, with “postgresql+asyncpg” schema.
Some of our endpoints use asyncio.gather to perform multiple queries at the same time.
We noticed that while each query run quite fast on it’s own when ran independently (up to 5s), the tasks take up to 40s to complete, which brought us to suspect there might not be a connections_pool in use, and they query the db sequently with the same connection, rather than in parallel.
I queried pg_stat_activity and noticed there’s indeed a single connection there, although there are multiple tasks running.
Experimenting with that, I tried to create a new database object for every task (meaning, do database.connect() and database.disconnect() for every task). I could then see numerous connections in pg_stat_activity, but it seemed like kind of missing the point of connections_pool, as I had to establish the connection for each of the tasks.
My question is - could it be that a single connection in pg_stat_activity encapsulates en entire connections_pool behind it? Or there might be some reason we are not aware of that prevents from the connections_pool to be created?
Notes:
- We use a single python database object (meaning, we only call the database.connect() function once)
- This object has min_pool_size parameter of 20, and max_pool_size of 50
- I verified we have connection pooling enabled in our console (type transaction), and we connect to the tsdb_transaction database.
- We are no where near the max_connections parameter from what I could tell from pg_stat_activity
Thanks for your help,
Tal.
5 posts - 2 participants