How Rate Limiting Works
IntermediateRate limiting caps how many requests a client can make in a time window, protecting your service from abuse, accidental floods, and runaway costs. Several algorithms trade precision against memory and burst behaviour: fixed window is simple but bursty at boundaries, sliding window is smoother, the token bucket allows controlled bursts, and the leaky bucket smooths output. In distributed systems the counter lives in a shared store like Redis so all servers enforce one limit.
Think of a token bucket as an arcade token dispenser
You have a bucket that refills with tokens at a steady rate — say one per second, up to a maximum of ten. Each ride (request) costs a token. If tokens are available you ride immediately; you can even take several quick rides in a row if the bucket is full (a burst). But once it empties, you must wait for it to refill. The steady refill enforces the average rate, while the bucket size allows short bursts.
Step by Step
Key Concepts
Token Bucket
A bucket refills with tokens at a fixed rate up to a capacity; each request consumes a token. It enforces an average rate while allowing bursts up to the bucket size — the most popular API rate-limit algorithm.
Fixed vs Sliding Window
Fixed window counts per discrete interval (simple but allows double-rate bursts at boundaries). Sliding window tracks a continuous trailing period, giving smoother, more accurate limiting.
Leaky Bucket
Requests enter a queue and drain at a constant rate, smoothing a bursty input into a steady output. Good when downstream systems need a uniform request rate.
Distributed Rate Limiting
When many servers must share one limit, the counter lives in a shared store (Redis) using atomic operations, so no client can exceed the limit by hitting different instances.
Key Facts
- The standard response for an exceeded limit is HTTP 429 with a Retry-After header telling the client when to try again.
- The token bucket is the most widely used because it enforces an average rate while still allowing legitimate short bursts.
- In a cluster, an in-memory per-server counter is wrong — a client can multiply their allowance by the number of servers; use a shared atomic counter.
Real-World Applications
Protecting a public API
A per-API-key token bucket (e.g., 1000 requests/hour) prevents any single client from overwhelming the service or running up costs, while still allowing brief bursts for legitimate spikes.
Login and OTP endpoints
A strict sliding-window limit per IP or account on login and one-time-password endpoints throttles brute-force and abuse attempts without affecting normal users.
Frequently Asked Questions
What is the difference between the token bucket and leaky bucket algorithms?
The token bucket refills tokens at a fixed rate up to a capacity and lets requests burst as long as tokens remain — enforcing an average rate while permitting short bursts. The leaky bucket queues requests and processes them at a constant drain rate, smoothing bursty input into a steady output. Token bucket allows bursts; leaky bucket eliminates them.
Why is a fixed window rate limiter bursty?
Because it resets the count at fixed boundaries, a client can send the full quota just before the window ends and the full quota again right after it resets — effectively double the rate across that boundary. A sliding window tracks a continuous trailing period and avoids this spike.
How do you rate limit across multiple servers?
Keep the counter in a shared store like Redis and update it with atomic operations (INCR with expiry, or a Lua script). This ensures all servers enforce a single combined limit; using per-server in-memory counters lets a client exceed the limit by spreading requests across instances.
What should an API return when a client exceeds the rate limit?
Respond with HTTP 429 Too Many Requests and include a Retry-After header (and often X-RateLimit-* headers) so the client knows how long to wait before retrying. Clear signalling lets well-behaved clients back off automatically.