Calling External APIs — httpx.AsyncClient Patterns
IntermediateOne shared AsyncClient (from lifespan) with explicit timeouts, raise_for_status handling, asyncio.gather for fan-out, and retries with backoff — the toolkit for every payment, SMS, and partner API you'll ever call.
Overview
Real services spend their lives calling other services — payment gateways, SMS providers, internal microservices — and httpx is the requests-compatible library that speaks async. The cardinal rule is client reuse: creating a client per request throws away connection pooling and pays a fresh TCP+TLS handshake every time, so the AsyncClient is created once in lifespan and shared via app.state. Every call needs an explicit timeout (the default 5s saves you from the hung-upstream incident, but be deliberate), status handling via raise_for_status, and — for independent upstream calls — asyncio.gather to overlap them so three 300ms calls cost 300ms, not 900ms. Failures are normal weather: transient 502s and timeouts get bounded retries with exponential backoff and jitter, but only for idempotent operations — retrying a non-idempotent payment capture is how double charges happen.
Shared Client + Timeouts + Fan-Out with gather
The client from lifespan reuses connections; explicit Timeout covers connect/read separately; gather overlaps independent calls and return_exceptions lets one slow upstream degrade gracefully instead of failing the page.
import asyncio, httpx
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
@asynccontextmanager
async def lifespan(app: FastAPI):
app.state.http = httpx.AsyncClient(
timeout=httpx.Timeout(5.0, connect=2.0), # read 5s, connect 2s — ALWAYS explicit
limits=httpx.Limits(max_connections=100, max_keepalive_connections=20),
)
yield
await app.state.http.aclose()
app = FastAPI(lifespan=lifespan)
@app.get("/students/{sid}/dashboard")
async def dashboard(sid: int, request: Request):
http = request.app.state.http # ONE pool for the whole app
# three independent upstreams — overlap them:
profile_t = http.get(f"https://profile.internal/students/{sid}")
offers_t = http.get(f"https://offers.internal/students/{sid}/offers")
events_t = http.get(f"https://events.internal/upcoming")
profile, offers, events = await asyncio.gather(
profile_t, offers_t, events_t,
return_exceptions=True, # one failure ≠ whole page failure
)
def safe_json(r, fallback):
return r.json() if isinstance(r, httpx.Response) and r.is_success else fallback
return {
"profile": safe_json(profile, {}), # required-ish
"offers": safe_json(offers, []), # degrade to empty
"events": safe_json(events, []), # degrade to empty
}
# Sequential: 300+300+300 = 900ms. gather: max(300,300,300) = 300ms.
# Client-per-request instead of shared: add a TCP+TLS handshake to every call.Errors, Retries & Idempotency
Distinguish transport errors from HTTP errors; retry only what is safe to repeat, with backoff + jitter; and translate upstream failures into YOUR error contract (502/503), never a raw stack trace.
import asyncio, random
import httpx
from fastapi import HTTPException
async def call_with_retry(http: httpx.AsyncClient, url: str,
attempts: int = 3) -> dict:
for attempt in range(1, attempts + 1):
try:
r = await http.get(url)
r.raise_for_status() # 4xx/5xx → HTTPStatusError
return r.json()
except httpx.TimeoutException: # slow upstream — retryable
kind = "timeout"
except httpx.ConnectError: # DNS/refused — retryable
kind = "connect"
except httpx.HTTPStatusError as e:
if e.response.status_code in (500, 502, 503, 504) and attempt < attempts:
kind = f"http {e.response.status_code}" # transient — retry
elif e.response.status_code == 429:
kind = "rate-limited" # honor Retry-After ideally
else:
raise HTTPException(502, "upstream rejected the request") # 4xx: OUR bug
if attempt == attempts:
raise HTTPException(503, "upstream unavailable, try again")
# exponential backoff + jitter: 0.5s, 1s, 2s (±25%) — jitter prevents
# every instance retrying in lockstep (thundering herd)
await asyncio.sleep(0.5 * 2 ** (attempt - 1) * random.uniform(0.75, 1.25))
# ── Idempotency: the retry precondition ──
# GET /status → retry freely
# PUT /orders/42/address → retry freely (same result twice)
# POST /payments/capture → NOT safe: timeout ≠ failure! The capture may
# have succeeded as your read timed out. Blind retry = double charge.
# Fix: idempotency keys — send Idempotency-Key: <uuid> (Razorpay/Stripe
# support this); the gateway returns the SAME result for the same key.
# POSTs without idempotency support: query state before retrying.Key Points to Remember
- 1One AsyncClient from lifespan — per-request clients discard pooling and re-handshake
- 2Explicit Timeout on every client; gather overlaps independent upstream calls
- 3Retry only transient faults (timeouts, 5xx) with exponential backoff + jitter
- 4A timed-out POST may have succeeded — idempotency keys prevent double charges
Interview Questions
Sign in to ask AriaYour payment-capture call timed out — why is an automatic retry dangerous, and what is the fix?
A dashboard aggregates three internal services — design the calls for latency and partial failure.
Why does retry backoff need jitter? What happens to a recovering service without it?
Ask Aria about Calling External APIs — httpx.AsyncClient Patterns
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.