Design: Chat System
AdvancedA real-time chat system (WhatsApp, Slack) requires persistent WebSocket connections, message ordering, delivery guarantees (sent/delivered/read), offline message storage, and group messaging.
Overview
A chat system has two fundamental challenges: real-time message delivery (low latency, bidirectional) and reliable message storage (persist messages, deliver to offline users when they reconnect). The core components are: (1) WebSocket gateway — maintains persistent connections with all online clients, routes messages in real-time. (2) Chat service — handles message validation, storage, and routing logic. (3) Message store — persists all messages with ordering guarantees (Cassandra for write-heavy, or a relational DB for smaller scale). (4) Presence service — tracks online/offline status using heartbeats. (5) Push notification service — delivers notifications to offline users via APNs/FCM. For group chats, the system must fan-out messages to all group members. For 1:1 chats, the sender writes the message, the chat service stores it and forwards it via WebSocket if the recipient is online, or queues a push notification if offline.
High-Level Architecture
Clients maintain WebSocket connections to the gateway. Messages flow through the chat service to the message store and are delivered via WebSocket (online) or push notification (offline).
// Chat system architecture
//
// User A (sender) User B (receiver)
// │ WebSocket ▲ WebSocket
// ▼ │
// ┌─────────────┐ ┌─────────────┐
// │ WS Gateway │────────────────────────│ WS Gateway │
// │ (stateful) │ │ (stateful) │
// └──────┬──────┘ └──────▲──────┘
// │ │
// ▼ │
// ┌─────────────┐ ┌──────────┐ ┌──────────────┐
// │ Chat Service │───►│ Message │───►│ Message Fan- │
// │ │ │ Store │ │ Out Service │
// └──────┬──────┘ │(Cassandra)│ └──────┬───────┘
// │ └──────────┘ │
// ▼ ▼
// ┌─────────────┐ ┌──────────────┐
// │ Presence Svc │ │ Push Notif │
// │ (Redis) │ │ (FCM / APNs) │
// └─────────────┘ └──────────────┘
// Message flow:
// 1. User A sends message via WebSocket
// 2. Chat service validates, assigns message_id + timestamp
// 3. Store in message DB
// 4. Check if User B is online (presence service)
// Online: forward via WebSocket gateway
// Offline: queue push notificationMessage Storage & Delivery
Messages are stored in a write-optimised database (Cassandra) partitioned by conversation_id for fast retrieval. Delivery receipts (sent/delivered/read) track message status.
// Message storage schema (Cassandra)
CREATE TABLE messages (
conversation_id TEXT,
message_id TIMEUUID, -- time-ordered, unique
sender_id TEXT,
content TEXT,
type TEXT, -- text, image, video
created_at TIMESTAMP,
PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id ASC);
// Delivery receipts:
// sent: server received and stored the message
// delivered: recipient's device received the message
// read: recipient opened the conversation
// Offline message delivery:
// 1. User B comes online → connects WebSocket
// 2. Client sends "last_seen_message_id" per conversation
// 3. Server queries: messages WHERE conversation_id = X AND message_id > last_seen
// 4. Push unread messages over WebSocket
// Group chat fan-out:
// Group has 500 members
// Message stored once per conversation
// Fan-out: push to each member's WebSocket connection
// For large groups: async fan-out via message queueScaling WebSockets
Each server handles thousands of WebSocket connections. A connection registry (Redis) maps user_id to the gateway server holding their connection. Cross-server message routing uses an internal message bus.
// WebSocket scaling architecture
//
// User→Server mapping in Redis:
// user:u-42 → ws-gateway-3:8080
// user:u-99 → ws-gateway-1:8080
//
// Message routing:
// 1. User A (on gateway-1) sends message to User B
// 2. Chat service looks up: user:u-B → ws-gateway-3
// 3. Chat service publishes to internal Redis pub-sub channel: gateway-3
// 4. Gateway-3 receives, pushes to User B's WebSocket
// Scale numbers:
// 1 server: ~50K concurrent WebSocket connections
// 100 servers: ~5M concurrent connections
// Connection is long-lived: ~1 KB memory per connection
// Heartbeat every 30s to detect dead connections
// Load balancing:
// Initial HTTP upgrade → L7 LB with sticky sessions
// Or: client connects to assigned gateway (from connection registry)
// Back-of-envelope for WhatsApp-scale:
// 2B users, 500M daily active
// Peak: 100M concurrent connections
// 2000 gateway servers × 50K connections each
// 40B messages/day = ~460K messages/secondKey Points to Remember
- 1WebSocket connections for real-time bidirectional communication; push notifications for offline users.
- 2Message store (Cassandra) partitioned by conversation_id — write-optimised, time-ordered.
- 3Presence service (Redis) tracks online/offline status via heartbeats.
- 4Cross-server message routing via Redis pub-sub or internal message bus.
- 5Delivery receipts (sent/delivered/read) require acknowledgement from client devices.
Interview Questions
Sign in to ask AriaHow would you design the message delivery flow for a 1:1 chat?
How do you handle message delivery to offline users?
How would you scale to 100 million concurrent WebSocket connections?
How do you implement group chat with 500 members efficiently?
Design a chat system like WhatsApp supporting 2 billion users.
Ask Aria about Design: Chat System
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.