Design a Distributed Cache — Cheat Sheet

A distributed cache sits between your application servers and your database, absorbing the majority of read traffic and reducing database load by 80–95%. Designing one correctly requires solving distributed key placement (consistent hashing), memory pressure management (eviction policies), thundering-herd protection (cache stampede), and fault tolerance (replication and automatic failover). Redis Cluster is the canonical production reference for all of these problems.

• ✓GET key — return the cached value or a cache miss signal in under 1ms at p99
• ✓SET key value ttl — store a value with an optional time-to-live; overwrites silently on re-set
• ✓DEL key — explicitly evict a key from the cache
• ✓Consistent key distribution — the same key must always map to the same cache node (with minimal remapping when nodes are added or removed)
• ✓Automatic failover — if a primary node fails, a replica is promoted without operator intervention
• ✓1M+ cache operations per second across the cluster

• ✓Target hit ratio: 90% (10% fall through to DB)
• ✓Read throughput: 1M ops/sec → ~333K ops/sec per node (3-node cluster)
• ✓Avg value size: 1 KB (JSON object)
• ✓Working set (hot 20%): 20M keys × 1KB = 20 GB across cluster
• ✓Memory per node: ~8 GB data + 2 GB overhead = 10 GB RAM per node
• ✓Network bandwidth: 333K ops/sec × 1KB = ~333 MB/s per node

• ✓Client Library (Lettuce / Jedis) — The application-side library responsible for hashing keys to the correct cluster node using CRC16 mod 16384, following MOVED redirects when the cluster topology changes, and connection pooling. Lettuce (async, non-blocking) is preferred over Jedis for high-throughput Spring Boot services.
• ✓Redis Cluster (Primary Nodes) — Three or more primary nodes that collectively own all 16,384 hash slots. Each primary handles reads and writes for its portion of the keyspace. Nodes communicate via the Redis Cluster Bus (port + 10000) for gossip, heartbeats, and failure detection.
• ✓Replica Nodes — Each primary has at least one replica that asynchronously replicates all writes. Replicas can serve read traffic (with eventual-consistency trade-off) to scale read throughput. On primary failure, the remaining primaries vote to promote the replica within ~5 seconds.
• ✓Consistent Hash Ring — The logical mechanism for mapping keys to nodes. Physical nodes are represented by many virtual nodes (vnodes) on the ring, ensuring even key distribution. When a node is added or removed, only the keys on the adjacent arc are remapped — not the entire keyspace.
• ✓Eviction Manager — Each Redis node enforces a maxmemory limit. When memory is full, the configured eviction policy (allkeys-lru, volatile-lfu, etc.) determines which keys are evicted. This runs inline with every write command, so it adds no background latency spikes.
• ✓Database (Source of Truth) — PostgreSQL or Cassandra behind the cache layer. Cache misses fall through to the database. The application is responsible for populating the cache after a miss (cache-aside) or the write path populates the cache proactively (write-through).

• ✓Consistency vs availability during network partition → Availability (AP): Redis Cluster continues to serve the nodes that remain reachable during a partition, even if some slots are temporarily unavailable. A cache that becomes unavailable is far worse than one that briefly serves stale data — the DB fallback path handles misses. Choosing CP (stopping writes on partition) would mean cache unavailability ripples into DB overload.
• ✓LRU vs LFU eviction policy → Workload-dependent: LRU for session/real-time, LFU for content catalogues: LRU wins when access patterns have strong temporal locality — recently used keys are most likely to be used again (user sessions, hot tweets). LFU wins when popularity is stable over time (product pages, static content) — it avoids evicting frequently accessed keys that happen to have a brief quiet period. Using the wrong policy can cut hit ratio by 10–20%.
• ✓Replication factor (1 replica vs 2 replicas per primary) → 1 replica per primary for most deployments; 2 for critical caches: A single replica provides failover protection at a 2x memory cost. Two replicas survive two simultaneous node failures and also scale read throughput across three nodes per slot, but triple the memory cost. For caches (where a miss gracefully falls through to the DB), 1 replica is sufficient. Use 2 replicas only when DB fallback latency is unacceptable (e.g. user-facing API latency SLAs under 5ms).
• ✓Synchronous (write-through) vs asynchronous (write-behind) cache population → Synchronous for consistency-critical data; asynchronous for counters and analytics: Write-through guarantees the cache is always consistent with the DB immediately after a write, but doubles write latency (two round-trips). Write-behind minimises write latency by flushing asynchronously, but a cache crash before flushing loses writes. Mixing both in the same system is common: write-through for entity data, write-behind for counters and leaderboards.

• ✓Open by distinguishing cache-aside, write-through, and write-behind before drawing any boxes. Interviewers expect you to justify the choice, not just pick one.
• ✓When explaining Redis Cluster, draw the 16,384 hash slots explicitly. Say "CRC16(key) % 16384" — using the exact formula signals you have worked with it in production, not just read about it.
• ✓Always bring up cache stampede proactively. Describe the mutex-lock mitigation, then mention probabilistic early recomputation (PER) as the low-latency alternative — this almost always surprises interviewers and differentiates strong candidates.
• ✓Distinguish LRU from LFU with a concrete example: "For a news feed I would use LRU because last hour's articles drive 90% of traffic; for a product catalogue I would use LFU because the top-100 bestsellers are always popular." Concrete examples beat abstract definitions.
• ✓For hotspot keys (e.g. a viral tweet), mention two defences: (1) local in-process L1 cache (Caffeine) in front of Redis so the network hop is eliminated entirely; (2) Redis hash tags to explicitly co-locate related keys, and explicit shard-splitting using key suffixes for read-heavy singletons.
• ✓Finish your design by quantifying hit ratio: "At 90% hit ratio and 1M ops/sec, only 100K ops/sec reach the database — a 10x reduction in DB load." Interviewers appreciate candidates who tie architecture to numbers.