Design a Rate Limiter — Cheat Sheet

A rate limiter enforces a ceiling on how many requests a client can make in a given time window. It protects backend services from overload, prevents abuse and DDoS amplification, and ensures fair resource distribution across tenants. The core challenge is making the check atomic, consistent across multiple API Gateway nodes, and fast enough to add no perceptible latency to the request path.

• ✓Reject requests that exceed a configured limit (e.g. 100 req/min per user)
• ✓Return HTTP 429 Too Many Requests with a Retry-After header on rejection
• ✓Support multiple limit dimensions: per-user, per-IP, per API key, and global
• ✓Configurable limits per endpoint (e.g. /login stricter than /search)
• ✓Clients can query remaining quota via X-RateLimit-Remaining headers
• ✓1M requests/sec across all API Gateway nodes

• ✓Inbound RPS: 1M req/sec across 20 gateway nodes = 50K req/sec/node
• ✓Redis ops per request: 1–2 commands (INCR + EXPIRE, or Lua script)
• ✓Redis throughput needed: 1M–2M commands/sec (Redis handles ~1M/sec single-threaded; use cluster)
• ✓Memory per user (token bucket): ~100 bytes (counter + timestamp in Redis hash)
• ✓Memory for 10M active users: 10M × 100B ≈ 1 GB — fits in a single Redis instance
• ✓Latency budget: Redis round-trip ~0.5ms LAN; Lua script <1ms; well within 5ms budget

• ✓API Gateway — The first layer that receives every inbound request. Extracts the rate limit key (user ID, API key, or IP) from the request, calls the Rate Limit Service before proxying to the backend, and attaches X-RateLimit-* response headers.
• ✓Rate Limit Service — A thin sidecar or library embedded in the gateway. Implements the chosen algorithm (token bucket, sliding window counter) and delegates all state reads/writes to Redis. Stateless itself — can scale horizontally without coordination.
• ✓Redis Cluster (Counter Store) — Single source of truth for all rate limit counters and token state. Uses atomic Lua scripts or MULTI/EXEC transactions to avoid race conditions. TTLs on all keys ensure automatic cleanup of inactive clients.
• ✓Config Service — Stores limit rules (endpoint → limit → window) in a fast read path (Redis hash or etcd). Gateway nodes poll or subscribe for rule changes, allowing runtime limit adjustments without restarts.
• ✓Reject Handler — When a request is rejected, the gateway returns HTTP 429 with Retry-After (seconds until next token refill or window reset) and X-RateLimit-Limit / X-RateLimit-Remaining / X-RateLimit-Reset headers. No backend call is made.

• ✓Token bucket vs sliding window counter → Token bucket for per-user quota; sliding window counter for endpoint-level throughput: Token bucket naturally models "quota" (a user has N credits) and allows bursts. Sliding window counter is better for protecting an endpoint from aggregate overload because it more accurately tracks request rate over time. Use both layers: token bucket per user, sliding window counter per endpoint.
• ✓Local (in-process) vs distributed (Redis) state → Distributed via Redis Cluster: Local state allows each gateway node to enforce limit/N independently, which means a client can send N × limit requests by round-robining across nodes. Distributed state in Redis enforces the global limit accurately. The trade-off is one Redis round-trip (~0.5ms) added to every request. At < 5ms latency budget this is acceptable.
• ✓Strict enforcement vs approximate counting → Approximate (sliding window counter) for high-scale public APIs: Exact enforcement (sliding window log) requires O(limit) memory per client per window and O(limit) ZADD/ZREMRANGEBYSCORE Redis operations. At 10M active clients × 100 req/min this is 16 GB and 1B Redis write ops/min — cost-prohibitive. The sliding window counter approximation has <1% error in practice and uses O(1) memory per client.
• ✓Fail-open vs fail-closed on Redis unavailability → Fail-open with degraded local limiting: Fail-closed blocks all traffic — a Redis outage becomes a full API outage, which is worse than temporary unthrottled traffic. Fail-open maintains availability. Mitigate abuse risk by enabling each gateway node's local counter (enforcing limit/N) as a fallback and alerting on-call immediately.

• ✓Clarify the key dimension first — are limits per user, per IP, per API key, or all three? Each needs a different Redis key namespace and the answer changes the architecture.
• ✓Know all four algorithms cold and be able to state the trade-off in one sentence each: fixed window (simple, boundary burst), token bucket (burst-friendly quota), sliding log (accurate, memory-heavy), sliding window counter (hybrid — Cloudflare's approach).
• ✓Interviewers frequently ask "how do you make this atomic?" — answer with Lua scripts immediately. Explain that Lua executes in Redis's single-threaded event loop so no locking is needed, unlike MULTI/EXEC which has retry overhead under contention.
• ✓Address the distributed consistency problem proactively: a single rate limiter embedded in one gateway node does not scale. Redis is the shared counter store; each stateless gateway node just calls Redis.
• ✓Discuss the failure mode explicitly. "What happens when Redis goes down?" is a favourite follow-up. Articulate the fail-open vs fail-closed trade-off and mention local fallback counters.
• ✓Mention the response headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After). This shows you understand the user-facing contract, not just the backend mechanics — interviewers appreciate the product awareness.