Home/Learn/System Design/Design: Rate Limiter

Design: Rate Limiter

Intermediate

Real-World Designs

A rate limiter service enforces request quotas per client using algorithms like token bucket or sliding window. It must be distributed (shared state via Redis), low-latency, and support tiered limits.

Overview

A dedicated rate limiter sits in front of your services (often in the API gateway) and enforces request quotas per client (by API key, user ID, or IP). Key design decisions: (1) Where — at the API gateway (simplest), as a sidecar, or as a shared service. (2) Algorithm — token bucket for burst-friendly limiting, sliding window for accuracy. (3) Distributed state — Redis for shared counters across multiple gateway instances. (4) Rules engine — configurable rules per endpoint and per client tier (free: 10/min, pro: 100/min). (5) Response — HTTP 429 with Retry-After header. (6) Soft vs hard limits — soft limits log and alert, hard limits reject. The rate limiter itself must be extremely fast (sub-millisecond) to avoid adding latency to every request.

System Design

Rate limiting rules are stored in a config store. The rate limiter checks Redis counters on every request and returns 200 (allowed) or 429 (rejected).

Conceptual + JSON — rate limiter architecture and rules

// Rate limiter architecture
//
//  Client Request
//    │
//    ▼
// ┌──────────────────┐
// │  API Gateway       │
// │  ┌──────────────┐ │     ┌──────────────┐
// │  │ Rate Limiter  │◄────►│    Redis      │ (shared counters)
// │  │ Middleware     │ │     └──────────────┘
// │  └──────┬───────┘ │     ┌──────────────┐
// │         │         │     │ Rules Config  │ (rate limit rules)
// │         │ allowed? │     └──────────────┘
// │         ▼         │
// │  ┌──────────────┐ │
// │  │ Route to      │ │
// │  │ Backend Svc   │ │
// │  └──────────────┘ │
// └──────────────────┘

// Rate limiting rules (config)
{
  "rules": [
    {
      "endpoint": "/api/v1/orders",
      "method": "POST",
      "limits": {
        "free":       { "requests": 10,   "window": "1m" },
        "pro":        { "requests": 100,  "window": "1m" },
        "enterprise": { "requests": 1000, "window": "1m" }
      }
    },
    {
      "endpoint": "/api/v1/auth/login",
      "limits": { "default": { "requests": 5, "window": "1m" } }
    }
  ]
}

Sliding Window Counter in Redis

A sliding window counter avoids the boundary burst problem of fixed windows. It uses Redis sorted sets to track individual request timestamps within the current window.

Lua + HTTP — sliding window rate limiter

// Sliding window log with Redis sorted set
// Key: rate:{userId}:{endpoint}
// Members: request timestamps, scored by timestamp

local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])   -- window in seconds
local limit = tonumber(ARGV[3])

-- Remove entries outside the window
redis.call('ZREMRANGEBYSCORE', key, 0, now - window * 1000)

-- Count requests in current window
local count = redis.call('ZCARD', key)

if count >= limit then
    return 0  -- REJECTED
end

-- Add current request
redis.call('ZADD', key, now, now .. ':' .. math.random(1000000))
redis.call('EXPIRE', key, window)
return 1  -- ALLOWED

// Response headers
// 429 Too Many Requests
// X-RateLimit-Limit: 100
// X-RateLimit-Remaining: 0
// X-RateLimit-Reset: 1711700100
// Retry-After: 45

Key Points to Remember

1Rate limiter sits at the API gateway — checks Redis counters on every request.
2Token bucket allows bursts; sliding window is more accurate but uses more memory.
3Redis Lua scripts provide atomic check-and-increment for distributed rate limiting.
4Support tiered limits per client plan (free, pro, enterprise) with configurable rules.
5Return 429 with X-RateLimit-* and Retry-After headers for graceful client back-off.

Interview Questions

How would you design a distributed rate limiter?

EasyTCS

Why use Redis Lua scripts instead of separate GET/SET commands?

MediumAmazon

How do you support different rate limits for different pricing tiers?

MediumGoogle

Compare token bucket and sliding window counter for a rate limiter.

MediumFlipkart

Design a rate limiter that handles 10M requests/second across 50 gateway instances.

HardUber

Ask Aria about Design: Rate Limiter

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Discussion

Loading discussion…

Design: Distributed Cache

Logging Best Practices