Rate Limiting

Intermediate

Reliability & Fault Tolerance

Rate limiting restricts the number of requests a client can make in a given time window. It protects services from abuse, ensures fair usage, and prevents cascading overload.

Overview

Rate limiting controls how many requests a client (identified by IP, API key, or user ID) can make within a time period. It protects backend services from being overwhelmed by traffic spikes, DDoS attacks, buggy clients, or abusive users. Common algorithms include Token Bucket (smooth, allows bursts), Leaky Bucket (constant rate), Fixed Window (simple, but has boundary burst issues), and Sliding Window Log/Counter (most accurate). Rate limits are typically enforced at the API gateway or a dedicated rate limiter service. Distributed rate limiting (across multiple servers) uses a shared store like Redis. When a client exceeds the limit, the server returns HTTP 429 (Too Many Requests) with Retry-After and X-RateLimit-* headers.

Rate Limiting Algorithms

Token Bucket is the most popular — tokens are added at a fixed rate, each request consumes a token, and burst is allowed up to the bucket capacity. Fixed Window is simpler but can allow double the rate at window boundaries.

Conceptual — rate limiting algorithms

// Token Bucket algorithm
// - Bucket has capacity of N tokens
// - Tokens added at rate R per second
// - Each request takes 1 token
// - If no token available → reject (429)
// - Allows burst up to N, sustained rate = R

// Example: 100 tokens capacity, 10 tokens/sec refill
// → sustained 10 req/sec, can burst to 100 req at once

// Fixed Window — simple but boundary issue
// Window: 1 minute, limit: 100 requests
// 10:00:00–10:00:59 → 100 allowed
// 10:01:00–10:01:59 → 100 allowed
// Problem: 100 requests at 10:00:50 + 100 at 10:01:10 = 200 in 20 seconds!

// Sliding Window Counter — fixes boundary issue
// Split each window into sub-windows
// Rate = (current window count) + (previous window count * overlap %)
// More accurate than fixed window, less memory than sliding log

// Algorithm comparison:
// Token Bucket:    smooth, burst-friendly, most popular
// Leaky Bucket:    constant rate, no bursts
// Fixed Window:    simple, boundary burst issue
// Sliding Window:  accurate, more memory/computation

Distributed Rate Limiting with Redis

For multi-server deployments, rate limiting state must be shared. Redis provides atomic operations (INCR, EXPIRE) for fast, distributed rate limiting.

Lua + Java — distributed rate limiter with Redis

// Redis-based fixed window rate limiter
// Key: rate_limit:{userId}:{minute}
// Lua script for atomic increment + check

local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])

local current = redis.call('INCR', key)
if current == 1 then
    redis.call('EXPIRE', key, window)
end

if current > limit then
    return 0  -- rate limited
else
    return 1  -- allowed
end

// Java rate limiter using Redis
@Component
public class RateLimiter {
    private final StringRedisTemplate redis;
    private final DefaultRedisScript<Long> script;

    public boolean isAllowed(String userId, int limit, int windowSeconds) {
        String key = "rate:" + userId + ":" + Instant.now().getEpochSecond() / windowSeconds;
        Long result = redis.execute(script, List.of(key), String.valueOf(limit), String.valueOf(windowSeconds));
        return result != null && result == 1L;
    }
}

API Gateway Rate Limiting

API gateways (Kong, NGINX, AWS API Gateway) provide built-in rate limiting — no custom code needed. They return standard HTTP 429 responses with rate limit headers.

HTTP + NGINX + AWS — rate limiting in practice

// Response headers when rate limited
HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 100          // max requests per window
X-RateLimit-Remaining: 0        // requests left in current window
X-RateLimit-Reset: 1711699260   // unix timestamp when window resets
Retry-After: 30                 // seconds to wait before retrying

// NGINX rate limiting
http {
    limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;

    server {
        location /api/ {
            limit_req zone=api burst=20 nodelay;
            # 10 req/sec sustained, burst up to 20, no delay for burst
        }
    }
}

// AWS API Gateway — per-client rate limiting
// Usage plan: 100 requests/second, burst 200
// Throttle per API key for tiered pricing:
//   Free:  10 req/sec
//   Pro:   100 req/sec
//   Enterprise: 1000 req/sec

Key Points to Remember

1Rate limiting protects services from overload, abuse, and ensures fair usage across clients.
2Token Bucket is the most popular algorithm — allows bursts while enforcing average rate.
3Distributed rate limiting requires shared state (Redis) for consistency across servers.
4Return HTTP 429 with X-RateLimit-* and Retry-After headers so clients can back off gracefully.
5API gateways (Kong, NGINX, AWS API Gateway) provide built-in rate limiting.

Interview Questions

What is rate limiting and why is it needed?

EasyWipro

Compare Token Bucket and Fixed Window algorithms.

MediumAmazon

How do you implement distributed rate limiting across multiple servers?

MediumGoogle

Design a rate limiter that supports different tiers (free, pro, enterprise).

HardFlipkart

How would you rate-limit a system handling 1M requests per second?

HardUber

Ask Aria about Rate Limiting

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Discussion

Loading discussion…

Idempotency

Failover & Redundancy