Consistent Hashing
IntermediateConsistent hashing maps both keys and nodes onto a virtual ring so that adding or removing a node only moves a small fraction of keys. It is the backbone of distributed caches, databases, and load balancers.
Overview
Traditional hash-based partitioning (hash(key) % N) requires remapping almost every key when N changes. Consistent hashing solves this by placing both nodes and keys on a conceptual ring (0 to 2^32). A key is assigned to the first node encountered clockwise from its position on the ring. When a node is added, only the keys between the new node and its predecessor move. When a node is removed, only its keys shift to the next node. To prevent uneven distribution, each physical node is represented by multiple virtual nodes (vnodes) spread around the ring. Consistent hashing is used by DynamoDB, Cassandra, Memcached (ketama), Akka Cluster, and CDN edge routing.
How the Hash Ring Works
Nodes and keys are hashed to positions on a ring. Each key is stored on the first node clockwise from its position. Adding or removing a node only affects the keys between the changed node and its predecessor.
// Consistent hash ring
//
// 0 / 2^32
// │
// Node C ●──────── ● Node A
// │ /
// │ key1 ● ← maps to Node A (first CW)
// │ /
// key3 ●──────●
// │ Node B
// │
//
// Adding Node D between B and C:
// Only keys between B and D move to D
// All other keys stay put
//
// Without consistent hashing:
// hash(key) % 3 → hash(key) % 4 = ~75% of keys remapped!
// With consistent hashing:
// Only ~1/N keys remapped (where N = number of nodes)Virtual Nodes
A single physical node may control a disproportionate arc of the ring. Virtual nodes (vnodes) solve this by mapping each physical node to many positions on the ring, ensuring even distribution.
// Virtual nodes — even distribution
//
// Without vnodes: 3 nodes might get 60/25/15 split
// With 100 vnodes per node: ~33/33/33 split
//
// Implementation sketch
public class ConsistentHash<T> {
private final TreeMap<Long, T> ring = new TreeMap<>();
private final int vnodes;
public ConsistentHash(Collection<T> nodes, int vnodes) {
this.vnodes = vnodes;
for (T node : nodes) addNode(node);
}
public void addNode(T node) {
for (int i = 0; i < vnodes; i++) {
long hash = hash(node.toString() + "#" + i);
ring.put(hash, node);
}
}
public void removeNode(T node) {
for (int i = 0; i < vnodes; i++) {
long hash = hash(node.toString() + "#" + i);
ring.remove(hash);
}
}
public T getNode(String key) {
long hash = hash(key);
Map.Entry<Long, T> entry = ring.ceilingEntry(hash);
return (entry != null) ? entry.getValue() : ring.firstEntry().getValue();
}
}Real-World Usage
Consistent hashing powers data placement in distributed databases (Cassandra, DynamoDB), cache clusters (Memcached, Redis Cluster), and content routing (CDNs, load balancers).
// Cassandra: partitioner uses consistent hashing (Murmur3)
// Each token range maps to a set of nodes
// vnodes = 256 (default) — each node owns 256 token ranges
// DynamoDB: consistent hashing for partition placement
// Partition key → hash → partition → storage node
// Redis Cluster: 16384 hash slots
// slot = CRC16(key) % 16384
// Each node owns a range of slots
// Resharding: migrate slots between nodes
// Envoy proxy: consistent hashing for session affinity
clusters:
- name: api_service
lb_policy: RING_HASH # consistent hashing
ring_hash_lb_config:
minimum_ring_size: 1024Key Points to Remember
- 1Consistent hashing maps keys and nodes to a ring — only ~1/N keys move when a node is added or removed.
- 2Virtual nodes solve uneven distribution by giving each physical node multiple ring positions.
- 3Used in Cassandra (token ring), DynamoDB (partitions), Redis Cluster (hash slots), and CDNs.
- 4It eliminates the catastrophic key remapping of hash(key) % N when N changes.
- 5Combined with replication, it provides both scalability and fault tolerance.
Interview Questions
Sign in to ask AriaWhat problem does consistent hashing solve?
Explain virtual nodes and why they are necessary.
How does Cassandra use consistent hashing?
What happens when a node crashes in a consistent hash ring with replication?
Implement a consistent hashing ring that supports adding/removing nodes dynamically.
Ask Aria about Consistent Hashing
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.