async def vs def — How FastAPI Runs Your Endpoints
Advanceddef endpoints run in a thread pool (blocking code is safe); async def runs on the event loop (huge concurrency, but one blocking call freezes everyone). The wrong combination is FastAPI's #1 performance bug.
Overview
FastAPI accepts both function styles and executes them completely differently. A plain def endpoint is dispatched to a thread pool (~40 threads by default) — blocking calls like requests.get or psycopg2 queries only occupy their thread, so the app stays responsive but concurrency caps at the pool size. An async def endpoint runs directly on the event loop: awaits overlap by the thousands, but any blocking call — time.sleep, a sync DB driver, heavy CPU work — freezes the loop and every in-flight request with it. This asymmetry produces the misdiagnosed bug seen in real incident reviews: someone "modernises" endpoints to async def without swapping the blocking libraries inside, and the service gets slower under load, not faster. The rule fits on a sticky note: async def only when everything inside is awaitable; blocking code stays in def; CPU-heavy work goes to a process pool either way.
The Two Execution Paths — Proven with Timings
Three versions of a "slow" endpoint under 10 concurrent requests tell the whole story: def scales via threads, async-with-await scales via the loop, async-with-blocking serialises everything.
import asyncio, time
from fastapi import FastAPI
app = FastAPI()
# A) def + blocking → thread pool handles it
@app.get("/report-def")
def report_def():
time.sleep(1) # blocking, but only THIS thread waits
return {"ok": True}
# 10 concurrent requests → ~1s total (10 threads sleep in parallel)
# B) async def + await → event loop handles it
@app.get("/report-async")
async def report_async():
await asyncio.sleep(1) # yields; loop serves others meanwhile
return {"ok": True}
# 10 concurrent → ~1s. 10,000 concurrent → still fine (no threads needed)
# C) async def + BLOCKING — the bug
@app.get("/report-broken")
async def report_broken():
time.sleep(1) # ✗ blocks the event loop itself
return {"ok": True}
# 10 concurrent requests → ~10s. Request #10 waited for all nine others.
# EVERY endpoint of the app is frozen during each sleep — /health too.
# Escape hatch when stuck with blocking code in an async path:
@app.get("/report-rescued")
async def report_rescued():
await asyncio.to_thread(time.sleep, 1) # push to a thread, keep loop free
return {"ok": True}
# CPU-bound (ML inference, PDF parsing, crypto)?
# Threads don't help (GIL — Python track) and the loop must never do it:
# ProcessPoolExecutor via run_in_executor, or a task queue (next chapters).Choosing per Endpoint — the Decision Table
Audit what is INSIDE the function, not what is fashionable. Mixed apps are normal and correct; consistency-for-its-own-sake is how the C-case bug ships.
# What's inside the endpoint? Write it as:
# ──────────────────────────────────────── ─────────────────────────────
# await-able I/O (httpx, asyncpg, async def ← scales via loop
# redis.asyncio, asyncio.sleep)
# blocking I/O (requests, psycopg2, def ← thread pool saves you
# time.sleep, boto3, most SDKs)
# pure CPU (report generation, ML) def + offload to process/queue
# trivial (return a dict) either — def has a hair of
# thread overhead; irrelevant
# The audit that matters before adding "async" to a def:
# 1. every DB call awaitable? (asyncpg / SQLAlchemy async — Databases ch.)
# 2. every HTTP call awaitable? (httpx.AsyncClient, not requests)
# 3. every sleep/backoff asyncio.sleep?
# 4. file I/O? (open()/read() block too — aiofiles or to_thread)
# One "no" → keep it def, or wrap the offender in asyncio.to_thread.
# Sizing note: the def thread pool defaults to 40 (AnyIO). 41 concurrent
# slow def requests → #41 queues. High-concurrency blocking workloads
# need the pool raised, or the code made truly async.
# Dependencies follow the same rule: a def dependency runs in the
# thread pool, an async def dependency runs on the loop — a blocking
# call inside an async dependency freezes the app exactly like C).
# Sanity check for the interview: "async is not faster. await lets
# WAITING overlap. Nothing that computes gets quicker — only the
# waiting gets shared."Key Points to Remember
- 1def → thread pool (blocking safe, ~40 threads); async def → event loop (must never block)
- 2async def + blocking call serialises the whole app — the classic FastAPI bug
- 3asyncio.to_thread wraps unavoidable blocking calls; process pools for CPU work
- 4Choose per endpoint by auditing what is inside; mixed apps are correct
Interview Questions
Sign in to ask AriaA teammate converted all endpoints to async def and p99 latency exploded — diagnose it.
How does FastAPI execute def endpoints without blocking the event loop?
Where should a 3-second ML inference run in a FastAPI service? Thread, loop, or elsewhere — argue it.
Ask Aria about async def vs def — How FastAPI Runs Your Endpoints
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.