How Load Balancers Work
IntermediateA load balancer sits between clients and your servers and distributes incoming requests across multiple backend instances. Without a load balancer, scaling horizontally is impossible — all traffic would hit a single server. A load balancer also monitors server health and stops sending traffic to unhealthy instances, providing both scalability and high availability. Modern load balancers also handle SSL termination, routing by path/hostname, and connection draining.
Think of it like a supermarket checkout manager
When 50 customers arrive, the checkout manager directs them to different cashiers (servers) to avoid queueing everyone at one register. If a cashier goes on break (server goes down), the manager stops sending customers there. If one register has a shorter queue, the manager might send the next customer there (least-connections routing). The customers don't care which cashier they get — they just want to check out quickly.
Step by Step
Key Concepts
Layer 4 vs Layer 7
L4 (transport layer) load balancers route based on IP address and TCP/UDP port. They are very fast but cannot make decisions based on URL path, headers, or cookies. L7 (application layer) load balancers understand HTTP — they can route /api to one server group and /static to another, inspect cookies for sticky sessions, and add/strip headers. AWS ALB is L7; AWS NLB is L4.
Round Robin
The simplest algorithm. Requests are distributed to servers in rotation: server 1, server 2, server 3, server 1, ... Each server gets an equal share of requests. Works well when requests have similar processing times. Fails when requests vary dramatically in cost — a slow server accumulates a backlog while fast ones sit idle.
Least Connections
Routes each new request to the server with the fewest active connections. Better than round robin for long-lived connections (WebSockets, streaming) or workloads with variable processing times. The load balancer tracks active connections per backend in real time.
Sticky Sessions (Session Affinity)
Routes requests from the same client to the same server, typically using a cookie or IP hash. Required when server-side session state is stored in memory (not in a shared store like Redis). Sticky sessions reduce load balancing effectiveness and make deployments harder — prefer stateless backends with shared session storage.
SSL/TLS Termination
The load balancer handles the TLS handshake and decryption, then sends plain HTTP to backends. Centralises certificate management (renew at the LB, not on every server). Enables the LB to inspect HTTP headers for routing decisions. The connection from LB to backend is over a private network where plain HTTP is acceptable.
Health Check
Periodic probe sent to each backend to verify it can handle traffic. HTTP health checks call a dedicated endpoint (GET /health) and expect a 200 OK. TCP health checks just verify the port accepts connections. Passive health checks detect failures from actual request failures (5xx responses). Combine both for fastest failure detection.
Horizontal Scaling
Adding more server instances (scaling out) rather than making one server more powerful (scaling up). Load balancers are the prerequisite for horizontal scaling — without one, you can't distribute traffic across multiple instances. Stateless backends scale horizontally to near-infinite capacity; stateful backends are harder.
Anycast
A routing technique where the same IP address is announced from multiple geographic locations and the internet routes requests to the nearest one. Used by CDNs and global load balancers (Cloudflare, AWS Global Accelerator) to reduce latency by serving users from the nearest point of presence.
Key Facts
- nginx handles over 1 million concurrent connections on a single server. It uses an event-driven, asynchronous architecture (similar to Node.js) to handle connections without a thread per connection.
- AWS ALB supports over 1 million requests per second and automatically scales its own capacity as traffic increases. It's a fully managed service — you never need to scale the load balancer itself.
- HAProxy is used by GitHub, Twitter, Instagram, and Airbnb for high-performance load balancing. It processes millions of connections per second on commodity hardware.
- A load balancer itself is a single point of failure if not deployed in HA mode. Cloud load balancers are inherently redundant. Self-managed setups use active-passive pairs with a floating IP (VRRP/keepalived).
- The X-Forwarded-For header is added by L7 load balancers to pass the original client IP to backends. Without it, all requests appear to come from the load balancer's IP.
- Blue-green deployments use two identical server groups. Traffic is switched 100% from blue (old) to green (new) via the load balancer — enabling instant rollbacks by switching back.
Real-World Applications
Zero-downtime deployments
To deploy new code: (1) mark one server as draining in the LB, (2) wait for connections to drain, (3) deploy new code, (4) re-add to the pool, (5) repeat for all servers. The LB continuously routes traffic to the remaining healthy servers. Users experience no downtime during the entire deployment.
Path-based routing for microservices
An L7 load balancer routes /api/users to the User Service, /api/orders to the Order Service, and /api/payments to the Payment Service — all behind a single domain. Clients call one URL; the LB handles the routing. This pattern also enables API gateways that add auth, rate limiting, and logging at the edge.
Geographic load balancing
A global load balancer routes European users to EU data centers and US users to US data centers, minimising latency. If an entire region goes down, the global LB reroutes all traffic to the healthy region (failover). AWS Route 53 latency-based routing and Cloudflare Load Balancing implement this pattern.
WebSocket load balancing
WebSocket connections are long-lived (hours or days). Least-connections routing ensures no single backend is overwhelmed with long-running connections while others sit idle. Sticky sessions pin WebSocket connections to a specific server if the server maintains per-connection state in memory.
Frequently Asked Questions
What is the difference between a load balancer and a reverse proxy?
A reverse proxy sits in front of servers and forwards client requests to them — it can serve as a cache, handle SSL, compress responses, and add security headers. A load balancer specifically distributes requests across multiple backend servers. In practice, tools like nginx and HAProxy do both. AWS ALB is a fully managed L7 reverse proxy + load balancer.
How does a load balancer handle WebSockets?
WebSocket connections start as HTTP and upgrade to a persistent TCP connection. L7 load balancers must support HTTP Upgrade headers. Once upgraded, the connection is pinned to a single backend server (sticky) because WebSocket is stateful. Load balancers track the connection and don't re-route it mid-session.
What happens when all backend servers fail?
The load balancer returns a 502 Bad Gateway or 503 Service Unavailable to clients — it has nowhere to send the request. Good LBs return a custom error page rather than timing out silently. To prevent this, set up autoscaling to replace failed instances and configure alerting on backend health check failure rates.
Do I need a load balancer for a single server?
Not for load distribution, but it's still useful: you can upgrade the server by swapping it out with zero downtime, use SSL termination at the LB, and add autoscaling later without DNS changes. Cloud deployments almost always put even single-server apps behind a managed LB for these reasons.