Cheat SheetsSystem DesignReliability & Fault Tolerance

Reliability & Fault Tolerance — Cheat Sheet

System Design · 7 topics. Download the PDF or the Instagram carousel and share it.

Cheat Sheet · AiCanCode.org
Reliability & Fault Tolerance
System Design7 topicsQuick revision reference
1

Circuit Breaker Pattern

The circuit breaker pattern prevents cascading failures by stopping calls to a failing downstream service. It transitions through Closed → Open → Half-Open states, giving the failing service time to recover.

  • ✓Circuit breaker prevents cascading failures by fast-failing when a downstream service is unhealthy.
  • ✓Three states: Closed (normal), Open (fast-fail), Half-Open (testing recovery).
  • ✓Always implement fallback logic — return cached data, queue for retry, or return a degraded response.
  • ✓Resilience4j is the standard Java library; Istio/Envoy provide infrastructure-level circuit breaking.
  • ✓Monitor circuit breaker state transitions — an open circuit is a critical signal that something is wrong.
Conceptual — circuit breaker state machine
// Circuit breaker states
//
//  ┌────────────────────────────────────────┐
//  │              CLOSED                     │
//  │  (requests flow normally)               │
//  │  Failure counter incremented on error   │
//  └──────────────┬─────────────────────────┘
//                 │ failure threshold reached
//                 ▼
//  ┌────────────────────────────────────────┐
//  │               OPEN                      │
//  │  (all requests fail immediately)        │
//  │  Return fallback / error                │
//  │  Timer starts (e.g. 30 seconds)         │
//  └──────────────┬─────────────────────────┘
//                 │ timeout expires
//                 ▼
//  ┌────────────────────────────────────────┐
//  │           HALF-OPEN                     │
//  │  (allow N trial requests)               │
//  │  If trials succeed → CLOSED             │
//  │  If trials fail    → OPEN               │
//  └────────────────────────────────────────┘
2

Retry & Exponential Backoff

Retry with exponential backoff re-attempts failed operations with progressively longer delays, preventing thundering herd problems and giving failing services time to recover.

  • ✓Retry only transient errors (500, 503, timeout) — never client errors (400, 401, 404).
  • ✓Exponential backoff prevents overwhelming a recovering service: delay = base * 2^attempt.
  • ✓Add jitter to prevent thundering herd — multiple clients retrying at the same moment.
  • ✓Always set a maximum retry count to prevent infinite loops.
  • ✓Combine retry with circuit breakers and idempotency for robust fault tolerance.
Java — exponential backoff with jitter
// Exponential backoff formula
// delay = min(base * 2^attempt, maxDelay)
//
// Attempt 1: 1s
// Attempt 2: 2s
// Attempt 3: 4s
// Attempt 4: 8s (capped at maxDelay)
//
// With full jitter:
// delay = random(0, min(base * 2^attempt, maxDelay))
//
// With equal jitter:
// temp = min(base * 2^attempt, maxDelay)
// delay = temp/2 + random(0, temp/2)

// Java implementation
public <T> T retryWithBackoff(Supplier<T> operation, int maxRetries) {
    int attempt = 0;
    while (true) {
        try {
            return operation.get();
        } catch (RetriableException e) {
            if (++attempt > maxRetries) throw e;
            long delay = Math.min(1000L * (1L << attempt), 30_000L); // exp backoff
            long jitter = ThreadLocalRandom.current().nextLong(delay); // full jitter
            Thread.sleep(jitter);
            log.warn("Retry attempt {} after {}ms: {}", attempt, jitter, e.getMessage());
        }
    }
}
3

Idempotency

An operation is idempotent if performing it multiple times produces the same result as performing it once. Idempotency is essential in distributed systems where retries and duplicate messages are inevitable.

  • ✓Idempotency ensures performing an operation N times has the same effect as performing it once.
  • ✓Idempotency keys prevent duplicate side effects for non-idempotent operations (POST).
  • ✓Store idempotency keys with cached responses; return cached response on duplicates.
  • ✓Message consumers must be idempotent — track processed event IDs with unique constraints.
  • ✓Use absolute values instead of deltas when possible (SET balance = 1100 vs balance += 100).
Conceptual — idempotency key prevents double charge
// Without idempotency — double charge!
// Client → POST /payments { amount: 100 }  → 201 (charged)
// Network timeout — client retries
// Client → POST /payments { amount: 100 }  → 201 (charged AGAIN!)
// User charged $200 instead of $100

// With idempotency key — safe retry
// Client → POST /payments
//   Headers: Idempotency-Key: "pay-abc-123"
//   Body: { amount: 100 }
//   → 201 (charged, response cached with key)
//
// Client retries (same key)
// Client → POST /payments
//   Headers: Idempotency-Key: "pay-abc-123"
//   → 200 (return cached response, no double charge)

// HTTP methods and idempotency:
// GET    → idempotent (read-only)
// PUT    → idempotent (full replace)
// DELETE → idempotent (delete same resource = still gone)
// POST   → NOT idempotent (needs idempotency key)
// PATCH  → depends on implementation
4

Rate Limiting

Rate limiting restricts the number of requests a client can make in a given time window. It protects services from abuse, ensures fair usage, and prevents cascading overload.

  • ✓Rate limiting protects services from overload, abuse, and ensures fair usage across clients.
  • ✓Token Bucket is the most popular algorithm — allows bursts while enforcing average rate.
  • ✓Distributed rate limiting requires shared state (Redis) for consistency across servers.
  • ✓Return HTTP 429 with X-RateLimit-* and Retry-After headers so clients can back off gracefully.
  • ✓API gateways (Kong, NGINX, AWS API Gateway) provide built-in rate limiting.
Conceptual — rate limiting algorithms
// Token Bucket algorithm
// - Bucket has capacity of N tokens
// - Tokens added at rate R per second
// - Each request takes 1 token
// - If no token available → reject (429)
// - Allows burst up to N, sustained rate = R

// Example: 100 tokens capacity, 10 tokens/sec refill
// → sustained 10 req/sec, can burst to 100 req at once

// Fixed Window — simple but boundary issue
// Window: 1 minute, limit: 100 requests
// 10:00:00–10:00:59 → 100 allowed
// 10:01:00–10:01:59 → 100 allowed
// Problem: 100 requests at 10:00:50 + 100 at 10:01:10 = 200 in 20 seconds!

// Sliding Window Counter — fixes boundary issue
// Split each window into sub-windows
// Rate = (current window count) + (previous window count * overlap %)
// More accurate than fixed window, less memory than sliding log

// Algorithm comparison:
// Token Bucket:    smooth, burst-friendly, most popular
// Leaky Bucket:    constant rate, no bursts
// Fixed Window:    simple, boundary burst issue
// Sliding Window:  accurate, more memory/computation
5

Failover & Redundancy

Redundancy duplicates critical components so that if one fails, another takes over. Failover is the process of switching to a standby component. Together, they achieve high availability.

  • ✓Redundancy = multiple copies of components; failover = automatic switchover on failure.
  • ✓Active-passive: standby ready, simple; active-active: all serve traffic, more efficient.
  • ✓RDS Multi-AZ provides automatic database failover in ~60-120s; Aurora in ~30s.
  • ✓Multi-region redundancy protects against datacenter failures but requires cross-region replication.
  • ✓Design for specific availability targets: 99.9% (8.76 hours/year downtime) vs 99.99% (52.6 minutes).
Conceptual — active-passive vs active-active
// Active-Passive failover
//
// ┌───────────┐     health check     ┌───────────┐
// │  Primary   │ ◄──────────────────►│  Standby   │
// │  (Active)  │   replication        │ (Passive)  │
// └─────┬─────┘                      └─────┬─────┘
//       │ all traffic                       │ no traffic (idle)
//       ▼                                   │
//    Clients                                │
//                                           │
// Primary fails → health check detects → Standby promoted
//                                     → DNS/LB updated → traffic flows to new primary

// Active-Active failover
//
// ┌───────────┐  ◄─── sync ───►  ┌───────────┐
// │  Node A    │                   │  Node B    │
// │  (Active)  │                   │  (Active)  │
// └─────┬─────┘                   └─────┬─────┘
//       │ 50% traffic                    │ 50% traffic
//       └──────────┬─────────────────────┘
//                  ▼
//               Clients (load balanced)
//
// Node A fails → 100% traffic goes to Node B (seamless)
6

Health Checks & Heartbeats

Health checks probe services to determine if they are alive and ready to handle traffic. Heartbeats are periodic signals nodes send to indicate they are operational. Both are essential for automated failure detection.

  • ✓Liveness = "is the process alive?" (restart if dead); readiness = "can it handle traffic?" (remove from LB if not).
  • ✓Health check endpoints should verify downstream dependencies (DB, cache, queues).
  • ✓Kubernetes probes: livenessProbe (restart), readinessProbe (traffic), startupProbe (slow startup).
  • ✓Heartbeats are push-based — the service periodically signals a coordinator that it is alive.
  • ✓Set appropriate timeouts and thresholds to avoid false positives (premature restarts).
Java + YAML — Spring Boot health checks
// Spring Boot Actuator health endpoint
// GET /actuator/health → { "status": "UP" }
// GET /actuator/health/liveness → { "status": "UP" }
// GET /actuator/health/readiness → { "status": "UP" }

// application.yml
management:
  endpoint:
    health:
      show-details: when_authorized
  health:
    livenessstate:
      enabled: true
    readinessstate:
      enabled: true

// Custom health indicator
@Component
public class DatabaseHealthIndicator implements HealthIndicator {
    private final DataSource dataSource;

    @Override
    public Health health() {
        try (Connection conn = dataSource.getConnection()) {
            conn.createStatement().execute("SELECT 1");
            return Health.up().withDetail("database", "reachable").build();
        } catch (Exception e) {
            return Health.down().withException(e).build();
        }
    }
}
7

Graceful Degradation

Graceful degradation allows a system to continue operating with reduced functionality when a component fails, rather than failing completely. Users get a degraded experience instead of an error page.

  • ✓Graceful degradation provides reduced functionality instead of complete failure.
  • ✓Classify dependencies as critical (must work) and non-critical (fallback acceptable).
  • ✓Fallback strategies: cached data, popular/default content, empty responses, static pages.
  • ✓Load shedding drops low-priority requests to protect high-priority operations under overload.
  • ✓Feature flags enable rapid toggling of non-critical features during incidents.
Java — fallback for non-critical dependencies
// E-commerce product page — graceful degradation
public ProductPageResponse getProductPage(String productId) {
    // Critical — must succeed or fail the request
    Product product = productService.getProduct(productId);

    // Non-critical — fallback on failure
    List<Product> recommendations;
    try {
        recommendations = recommendationService.getFor(productId);
    } catch (Exception e) {
        recommendations = popularProductsCache.getTopSelling(); // fallback
        log.warn("Recommendation service unavailable, using popular products");
    }

    // Non-critical — return empty on failure
    List<Review> reviews;
    try {
        reviews = reviewService.getReviews(productId);
    } catch (Exception e) {
        reviews = List.of(); // empty list as fallback
    }

    return new ProductPageResponse(product, recommendations, reviews);
}
Learn this free with Aria, your AI tutor → AiCanCode.org/learn/system-design