Home/Learn/System Design/Design: URL Shortener

Design: URL Shortener

Intermediate

Real-World Designs

A URL shortener (like bit.ly) maps long URLs to short codes, stores the mapping in a database, and redirects users from the short URL to the original. Key challenges include unique ID generation, high-read throughput, and analytics.

Overview

A URL shortener receives a long URL and returns a short code (e.g. short.ly/abc123). When a user visits the short URL, the service looks up the code, finds the original URL, and returns an HTTP 301/302 redirect. The core design decisions are: (1) How to generate unique short codes — base62 encoding of an auto-increment ID, random generation with collision detection, or a pre-generated ID service. (2) Storage — a key-value store (Redis, DynamoDB) for fast lookups by short code. (3) Read-heavy traffic (100:1 read-to-write ratio) — cache popular URLs in Redis. (4) Analytics — track click counts, referrers, geolocation per short URL. Scale considerations: 100M URLs created/month, 10B redirects/month, data retained for 5 years.

High-Level Design

Two main APIs: createShortURL(longURL) → shortURL, and redirect(shortCode) → 301 redirect to longURL. A cache layer handles the read-heavy redirect traffic.

Conceptual — URL shortener architecture

// URL Shortener architecture
//
//  Client: POST /api/shorten { url: "https://very-long-url.com/..." }
//    │
//    ▼
//  ┌──────────────────┐
//  │   API Service     │ → generate short code → store in DB
//  │   (stateless)     │ → return short.ly/abc123
//  └──────┬───────────┘
//         │
//  Client: GET short.ly/abc123
//    │
//    ▼
//  ┌──────────────────┐     ┌──────────┐
//  │  Redirect Service │────►│  Cache    │ (Redis: shortCode → longURL)
//  │  (stateless)      │     │  (L1)     │
//  └──────────────────┘     └──────┬───┘
//                                  │ cache miss
//                                  ▼
//                           ┌──────────┐
//                           │    DB     │ (DynamoDB / PostgreSQL)
//                           └──────────┘
//
//  Response: HTTP 301 Location: https://very-long-url.com/...

Short Code Generation

Base62 encode an auto-increment ID (simple, no collisions) or use a random 7-character string (check for collisions). A 7-character base62 code supports 62^7 = 3.5 trillion URLs.

Java — short code generation strategies

// Approach 1: Base62 encoding of auto-increment ID
// ID: 123456789 → Base62: "8M0kX" (5-7 chars)
private static final String BASE62 = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";

public String encode(long id) {
    StringBuilder sb = new StringBuilder();
    while (id > 0) {
        sb.append(BASE62.charAt((int) (id % 62)));
        id /= 62;
    }
    return sb.reverse().toString();
}

// Approach 2: Random string + collision check
public String generateCode() {
    while (true) {
        String code = RandomStringUtils.randomAlphanumeric(7);
        if (!codeExists(code)) return code;  // check DB
    }
}

// Approach 3: Pre-generated ID range (distributed)
// ID Service allocates ranges: Server1 gets 1-10000, Server2 gets 10001-20000
// No collision, no coordination during normal operation

// Capacity: 7 chars of base62 = 62^7 = 3.5 trillion unique codes
// At 100M/month, lasts ~3000 years

Scale & Analytics

Cache hot URLs in Redis (99% hit rate). Use async event streaming for click analytics. Shard DB by short code for horizontal scaling.

Conceptual — scale estimation and caching

// Read path optimisation
// 100:1 read:write ratio → cache is critical
// Redis cache: shortCode → longURL (TTL: 24 hours)
// Cache hit rate: ~99% for popular URLs (Zipf distribution)
// Cache miss: read from DB, populate cache

// Analytics (async, non-blocking)
// On each redirect:
// 1. Return 301 immediately (don't block on analytics)
// 2. Emit click event to Kafka: { shortCode, timestamp, ip, userAgent, referrer }
// 3. Analytics consumer: aggregate clicks per URL, per day, per country

// Back-of-envelope:
// 100M new URLs/month ÷ 30 days ÷ 86400 sec = ~40 writes/sec
// 10B redirects/month ÷ 30 ÷ 86400 = ~4000 reads/sec
// Storage: 100M × 12 months × 5 years × 1KB = ~6 TB
// Cache: top 20% URLs = ~1.2 TB (fits in Redis cluster)

// DB choice: DynamoDB (key-value, auto-scaling)
// Partition key: short_code
// Read capacity: 4000 RCU (cache handles 99%)
// Write capacity: 40 WCU

Key Points to Remember

1Core: generate unique short code, store mapping, redirect with 301/302.
2Base62 encoding of auto-increment ID is simplest — no collision, predictable length.
3Read-heavy (100:1) — Redis cache with ~99% hit rate handles most redirects.
4Analytics via async event streaming (Kafka) — never block the redirect path.
5A 7-character base62 code supports 3.5 trillion URLs — more than enough for most systems.

Interview Questions

How would you generate unique short codes for a URL shortener?

EasyTCS

How do you handle the 100:1 read-to-write ratio?

MediumAmazon

Should you use 301 or 302 redirects and why?

MediumGoogle

How would you add click analytics without affecting redirect latency?

MediumFlipkart

Design a URL shortener handling 10 billion redirects per month.

HardUber

Ask Aria about Design: URL Shortener

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Discussion

Loading discussion…

LSM-Tree vs B-Tree

Design: Chat System