QueuePool Limit Reached Under FastAPI Load: pool_size Is Not the Fix

Your FastAPI service is fine at low traffic and falls over under load with a wall of the same exception. The obvious read is "the pool is too small", so you bump pool_size from 5 to 20, add more overflow for good measure, and it holds for a while. Then the same error comes back at a higher concurrency, and now you are burning twenty-plus database connections per worker for a problem that raising the number never actually solved.

Pool size is almost never the real story. In an async FastAPI app the pool usually exhausts because connections are being held far longer than the work needs them, most often across an await to some other service. And even when you size things generously, SQLAlchemy's queue has a known fairness problem under heavy contention that raising the size does not fix. This is one of those errors where the number in the message sends you the wrong way.

Let me separate the two failures that hide behind one exception, because the fix for each is different, and neither is "make the number bigger".

The exact error

sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00

Read it literally, because every part is telling you something. size 5 is pool_size: the connections kept open persistently. overflow 10 is max_overflow: extra connections the pool may open temporarily when the base pool is full. So this pool allows up to 15 concurrent checkouts. timeout 30.00 is pool_timeout: a request that wanted a connection waited 30 seconds, all 15 were still checked out, and it gave up. The message is not "you need more connections". It is "15 connections were all busy for 30 straight seconds", which is a question about what is holding them.

If you are on asyncpg you may also see its sibling, which is the same disease with a different symptom, one connection shared across concurrent coroutines:

asyncpg.exceptions.InterfaceError: cannot perform operation: another operation is in progress

Why the pool exhausts: two different causes

Cause 1: connections held across an await

This is the big one in async apps and it is invisible in testing. A pooled connection is a scarce resource that should be checked out for the microseconds the query runs and returned immediately. But in async code it is easy to write a handler that grabs a session, does a query, then awaits an HTTP call to a payment provider or another microservice while still holding the session, then does another query. For the entire duration of that external call, a database connection sits idle-but-checked-out, doing nothing, unavailable to anyone else.

At low traffic nobody notices. At load, if that external call takes a second and you have 15 connections, you can serve at most 15 concurrent requests through that path no matter how fast your database is, because the connections are blocked on the network, not the query. The 16th request waits pool_timeout seconds and throws. The database is bored; the pool is starved.

Cause 2: a sync engine run through a threadpool

If you built the app with a synchronous SQLAlchemy engine and let FastAPI run your blocking DB calls in its threadpool, every concurrent request needs both a thread and a connection, and the two limits fight each other. You get pool timeouts, thread starvation, or both, and tuning one just moves the bottleneck to the other. The async ecosystem exists precisely so a request awaiting the database does not tie up a thread. Running sync SQLAlchemy under async FastAPI is the architecture that produces this error at modest load.

The fairness limitation you cannot tune away

There is a third, subtler contributor worth naming so you stop chasing it with configuration. SQLAlchemy's QueuePool is built on Python's queue primitives, and the maintainers have acknowledged that under high contention the currently-running thread can effectively jump the wait queue and reacquire a connection ahead of threads that have been waiting longer. The upstream issue about this was closed as a documentation matter, not a queue redesign, which tells you the honest answer: this is a known structural behaviour, not a bug you will get patched, and raising pool_size does not make the queue fair. It just gives the queue-jumpers more connections to fight over. The real mitigation is reducing contention by holding connections for less time, which is the same fix as Cause 1.

The fix, in steps

Step 1: Use a real async engine, not a sync engine in threads

from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker

engine = create_async_engine(
    "postgresql+asyncpg://user:pass@host/db",
    pool_size=10,
    max_overflow=10,
    pool_timeout=30,
    pool_pre_ping=True,   # detect dead connections before handing them out
)
AsyncSessionLocal = async_sessionmaker(engine, expire_on_commit=False)

An async engine uses an async-aware pool and lets a request awaiting the database yield instead of holding a thread. This alone removes the thread-versus-connection fight from Cause 2.

Step 2: Scope the session to the DB work, not the whole request

The single most effective change: open the session, do the queries, close it, and do your external calls outside that scope. Do not span an await to another service while holding a session.

# BAD: session held across the external await
async def handle(payload):
    async with AsyncSessionLocal() as session:
        order = await session.get(Order, payload.id)
        await charge_card(order)        # connection held for the whole HTTP call
        order.status = "paid"
        await session.commit()

# GOOD: connection released before the external call, reacquired after
async def handle(payload):
    async with AsyncSessionLocal() as session:
        order = await session.get(Order, payload.id)
    result = await charge_card(order)   # no DB connection held here
    async with AsyncSessionLocal() as session:
        order = await session.get(Order, payload.id)
        order.status = "paid" if result.ok else "failed"
        await session.commit()

Yes, that is two short checkouts instead of one long one. That is the point: the connection is on loan only while it is doing database work. The external call, which is where the time actually goes, holds nothing.

Step 3: Give each request its own session, and never share a connection across coroutines

The asyncpg another operation is in progress error comes from two coroutines using the same connection or session concurrently, for example a shared module-level session or a gather over tasks that all touch one session. Each concurrent unit of work needs its own session from the sessionmaker. Use a FastAPI dependency that yields a fresh session per request:

async def get_session():
    async with AsyncSessionLocal() as session:
        yield session

@app.post("/orders")
async def create_order(session: AsyncSession = Depends(get_session)):
    ...   # this session belongs to this request only

Step 4: Size the pool to your database, not to your panic

Only after the above should you touch the numbers, and then modestly. Total connections across all workers must stay under your database's max_connections, minus its reserved slots. If you run 4 Uvicorn workers with pool_size=10, max_overflow=10, that is up to 80 connections from this one service before overflow even counts elsewhere. Postgres will start refusing connections long before you think, and the reserved superuser slots mean it is effectively full earlier than the raw number suggests, which I covered in why 97 connections counts as full. Sizing the pool up is a database-capacity decision, not a fix for held connections.

Verification: watch the pool, not just the error rate

You want to see checkouts rise and, more importantly, see them return. Log the pool status under load:

print(engine.pool.status())
# e.g. "Pool size: 10  Connections in pool: 2  Current checked out: 8  Overflow: 0"

1. Under a load test, before the fix, "checked out" pins at the maximum and stays there while requests time out. Connections are not coming back.
2. After scoping sessions, "checked out" spikes during query bursts and drops back between them, and the timeouts stop even though pool_size is unchanged. That drop-back is the proof.
3. Enable pool_pre_ping=True and confirm you no longer get stale-connection errors after the database restarts or idle connections are culled by a proxy.

The signal to trust is the checked-out count returning to near zero between bursts. If it does, the pool is healthy at its current size. If it climbs and stays high, you still have a connection held somewhere it should not be.

What people get wrong

Raising pool_size and max_overflow as the first move. It buys time and hides the real bug, which is hold duration. You will hit the same wall at higher concurrency, now with a tighter connection budget and a database closer to its own max_connections ceiling. The number was never the constraint; the loan duration was.

Rotating one global session for the whole app. A shared session or connection across concurrent coroutines produces another operation is in progress and subtle data corruption. Sessions are cheap and are meant to be short-lived and per-request. Do not treat one as a singleton.

Running sync SQLAlchemy under async FastAPI and tuning the threadpool. You end up balancing threads against connections, and every gain in one starves the other. If you are async at the web layer, be async at the database layer with create_async_engine; do not bridge a sync engine through threads at scale.

Treating pool_timeout errors as transient and adding retries. A retry against an exhausted pool just joins the same queue and makes contention worse. Retries are for genuinely transient faults; a pool that is exhausted because connections are held is a design issue, and the pattern here is closer to Prisma's connection pool timeouts than to a flaky network.

When it is still broken

• Checked-out count returns to zero but you still time out in bursts. Your steady-state is fine and your peak concurrency genuinely exceeds the pool. Now sizing up is legitimate, but do the database-capacity arithmetic first and consider a connection pooler like PgBouncer in front, so many app connections share fewer database ones.
• A long-running query holds a connection for seconds. Then the connection is doing real work, not idling, and the fix is the query, not the pool. Find it the way you would any slow query and index or rewrite it.
• Errors spike right after a deploy or a database restart. Stale pooled connections. pool_pre_ping=True handles most of this by validating a connection before handing it out; also check whether a proxy between you and the database is killing idle connections on its own timer.
• The database itself is refusing connections. If you see the server-side "too many clients" rather than a client-side pool timeout, the ceiling is the database, not your pool, and the answer is fewer total connections plus a pooler, covered in why raising max_connections is wrong.

The line to hold onto: QueuePool limit reached is a statement about time, not quantity. Fifteen connections held for thirty seconds and fifteen connections held for thirty milliseconds are the same pool with wildly different capacity. Fix how long you hold them before you ever touch how many you have.