Home/Learn/System Design/Load Balancing

Load Balancing

Beginner
Fundamentals

A load balancer distributes incoming traffic across multiple backend servers using algorithms like round-robin, least connections, or consistent hashing. It improves throughput, reduces latency, and provides fault tolerance.

Overview

A load balancer sits between clients and a pool of backend servers, forwarding each request to a healthy server according to a scheduling algorithm. Layer 4 (L4) load balancers operate on TCP/UDP (fast, connection-level); Layer 7 (L7) load balancers inspect HTTP headers, URLs, and cookies for content-based routing. Common algorithms include round-robin (simple, equal distribution), weighted round-robin (accounts for server capacity), least connections (sends to the server with fewest active connections), and IP hash (sticky sessions). Cloud providers offer managed load balancers (AWS ALB/NLB, GCP Cloud Load Balancing) while software options include NGINX, HAProxy, and Envoy. Health checks continuously probe backend servers; unhealthy servers are removed from the pool until they recover.

L4 vs L7 Load Balancing

L4 operates at the transport layer — fast and protocol-agnostic but cannot route based on HTTP content. L7 inspects application-layer data — supports path-based routing, header-based routing, and SSL termination.

NGINX — L7 path-based routing
// L4 load balancer — TCP level
// Client → LB (picks backend by IP:port) → Backend
// Fast, simple, but no content awareness

// L7 load balancer — HTTP level (NGINX example)
upstream api_servers {
    least_conn;                    # algorithm
    server api1.internal:8080;
    server api2.internal:8080;
    server api3.internal:8080;
}

upstream static_servers {
    server cdn1.internal:80;
    server cdn2.internal:80;
}

server {
    listen 443 ssl;

    # Path-based routing (L7 feature)
    location /api/ {
        proxy_pass http://api_servers;
    }
    location /static/ {
        proxy_pass http://static_servers;
    }
}

Common Algorithms

Round-robin is the simplest; weighted round-robin lets you send more traffic to stronger servers; least connections is best when request processing times vary; IP hash provides sticky sessions without cookies.

Comparison + Terraform — ALB health check
// Algorithm comparison
//
// Algorithm           | Best For                    | Weakness
// ──────────────────────────────────────────────────────────────
// Round Robin         | Equal servers, short reqs   | Ignores server load
// Weighted Round Robin| Mixed-capacity servers      | Static weights
// Least Connections   | Varying request durations   | Slightly more overhead
// IP Hash             | Session affinity needed     | Uneven if few clients
// Consistent Hashing  | Caches, sharded services    | More complex

// AWS ALB target group with health check
resource "aws_lb_target_group" "api" {
  name     = "api-targets"
  port     = 8080
  protocol = "HTTP"
  vpc_id   = var.vpc_id

  health_check {
    path                = "/health"
    interval            = 15
    timeout             = 5
    healthy_threshold   = 2
    unhealthy_threshold = 3
  }
}

Global & DNS Load Balancing

For multi-region deployments, DNS-based load balancing (Route 53, Cloud DNS) directs users to the nearest region. Global L7 load balancers (Cloudflare, AWS Global Accelerator) provide anycast routing with automatic failover.

Architecture + Terraform — multi-region LB
// Multi-tier load balancing
//
//  Users (worldwide)
//    │
//    ▼
// DNS Load Balancer (Route 53 — latency-based routing)
//    │                    │
//    ▼                    ▼
// US-East Region       EU-West Region
//    │                    │
//    ▼                    ▼
// ALB (L7)             ALB (L7)
//  ┌─┼─┐               ┌─┼─┐
//  ▼ ▼ ▼               ▼ ▼ ▼
// App App App         App App App

// Route 53 latency-based record
resource "aws_route53_record" "api" {
  zone_id        = var.zone_id
  name           = "api.example.com"
  type           = "A"
  set_identifier = "us-east-1"
  latency_routing_policy {
    region = "us-east-1"
  }
  alias {
    name    = aws_lb.us_east.dns_name
    zone_id = aws_lb.us_east.zone_id
  }
}

Key Points to Remember

  • 1L4 load balancers are fast (TCP level); L7 load balancers enable content-based routing (HTTP).
  • 2Common algorithms: round-robin, weighted round-robin, least connections, IP hash, consistent hashing.
  • 3Health checks automatically remove unhealthy backends and re-add them when recovered.
  • 4DNS-based load balancing provides global traffic management across regions.
  • 5Always deploy at least two load balancers for high availability (active-passive or active-active).

Interview Questions

Sign in to ask Aria
1

What is the difference between L4 and L7 load balancing?

EasyInfosys
2

How does a least-connections algorithm work?

EasyTCS
3

How would you design load balancing for a multi-region application?

MediumAmazon
4

What happens when a load balancer itself fails?

MediumGoogle
5

Design a load balancing strategy for WebSocket connections.

HardUber

Ask Aria about Load Balancing

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Sign in to save personal notes for this topic.

Discussion

Sign in to join the discussion.

Loading discussion…