Design a Social Media Feed (Twitter/X) — Cheat Sheet

A social media feed must aggregate posts from everyone a user follows and present them in a ranked order — in near real-time. Behind this lies a write-fan-out problem: when a user tweets, their message must appear in the timelines of potentially millions of followers. The core design challenge is choosing between fan-out on write (push) and fan-out on read (pull), and handling the celebrity problem where a single account has tens of millions of followers.

• ✓Users can post tweets (text up to 280 chars, optional media)
• ✓Users can follow/unfollow other users
• ✓A user's home timeline shows tweets from everyone they follow, sorted by recency (and optionally ranked)
• ✓Tweets can include images and videos served via CDN
• ✓Push notifications for mentions and replies (stretch goal)
• ✓Tweet can be liked, retweeted, and replied to

• ✓Tweet writes: 500M / day ≈ 5,800 / sec
• ✓Timeline reads: 1.5B / day ≈ 17,400 / sec
• ✓Storage per tweet: ~300 bytes (text + metadata)
• ✓Tweet storage (5 years): 500M × 365 × 5 × 300B ≈ 274 TB
• ✓Avg followers per user: ~200 (median); celebrities > 1M
• ✓Fan-out events (push): 5,800 writes/sec × 200 followers = 1.16M fan-out ops/sec

• ✓API Gateway / Load Balancer — Terminates TLS, handles authentication (JWT/OAuth), enforces rate limits (300 tweets/3h per user), and routes POST /tweet to the Tweet Service and GET /timeline to the Timeline Service.
• ✓Tweet Service — Validates and persists new tweets to the Tweet Store (Cassandra/DynamoDB). After a successful write, publishes a TweetCreated event to Kafka for async fan-out processing.
• ✓Fan-out Service (Workers) — Consumes TweetCreated events from Kafka. For each tweet, looks up the author's follower list and pushes the tweet ID into each follower's timeline cache (Redis sorted set, scored by timestamp). Scales horizontally by partitioning Kafka by author ID.
• ✓Timeline Service — Serves GET /timeline requests. Reads the pre-built timeline from the user's Redis sorted set. For celebrity accounts (> 1M followers) whose tweets are NOT pre-fanned-out, merges on read. Returns a hydrated list of tweets with author info.
• ✓Tweet Store (Cassandra) — Stores tweet content keyed by tweet_id. Partitioned by author_id for efficient author-timeline queries. Cassandra's wide-row model allows efficient range scans for a given author's recent tweets.
• ✓Timeline Cache (Redis Sorted Sets) — Each user has a Redis sorted set keyed by user_id, where members are tweet IDs and scores are Unix timestamps. The Fan-out Service pushes IDs here; Timeline Service reads them. Capped at ~800 most recent entries per user to bound memory.

• ✓Fan-out on write vs fan-out on read → Hybrid (write for normal users, read-merge for celebrities): Pure write fan-out collapses under celebrity-scale follower counts. Pure read fan-out makes timelines expensive for users with many followees. The hybrid caps write amplification while keeping read latency low for the common case.
• ✓SQL vs Cassandra for tweet storage → Cassandra (wide-column): Tweets are written once and read by tweet_id or (author_id, timestamp). Cassandra's wide rows handle high-volume append-only writes and efficient range scans by author. SQL would require expensive sharding at 500M tweets/day.
• ✓Chronological vs algorithmic ranking → Chronological (sorted set score = timestamp) as default; ranking layer on top: Sorted sets give O(log N) inserts and O(log N + K) range reads without a ranking pass. A lightweight re-ranking model (engagement prediction) can be applied as a post-processing step on the top-N candidates without changing the storage model.
• ✓Timeline cache cap (800 entries) → 800 entries per user timeline: The vast majority of users never scroll past 50-100 tweets. Capping at 800 bounds Redis memory cost. Users who scroll beyond the cap fall back to the Tweet Store with negligible impact on the p99 latency SLO.
• ✓Synchronous vs asynchronous fan-out → Asynchronous via Kafka: Posting a tweet must return in milliseconds. Writing to potentially millions of Redis keys synchronously in the request path would time out. Kafka decouples the write path; fan-out may lag by a few seconds, which is acceptable.

• ✓Always start with the fan-out problem — it's the crux. Ask the interviewer whether to optimise for write latency or read latency before choosing your model.
• ✓The celebrity problem is a well-known follow-up. Introduce the threshold concept proactively and explain both the fan-out skip and the read-merge.
• ✓Redis Sorted Set is the canonical data structure for a timeline. Know: ZADD, ZREVRANGE, ZREMRANGEBYRANK. Explain why the score is a timestamp.
• ✓Kafka is mentioned for a reason — explain WHY async fan-out matters: decouples write latency from fan-out depth, enables retry/backpressure, and allows multiple consumers (analytics, notifications, search indexing) from the same event stream.
• ✓Media is a common extension question. Know pre-signed S3 URLs, HLS/DASH for adaptive streaming, and CDN cache-control strategy.
• ✓If asked about ranking: keep it simple. Fetch the top N by recency, score each with a lightweight model (likes, retweets, relationship strength), re-sort. Avoid over-engineering into a full ML pipeline unless asked.