API Gateway

Intermediate
Infrastructure

An API Gateway is a single entry point for all client requests that handles cross-cutting concerns — authentication, rate limiting, routing, protocol translation, and observability — before forwarding to backend microservices.

Overview

As a system grows from a monolith to microservices, each service should not independently implement auth, rate limiting, logging, and SSL. An API Gateway centralises these cross-cutting concerns. It sits between clients and the internal service mesh, acting as a reverse proxy that can authenticate tokens (JWT/OAuth), enforce quotas, translate protocols (REST↔gRPC), aggregate responses from multiple services, and emit unified access logs. AWS API Gateway, Kong, and Nginx are common choices. Understanding the API Gateway pattern is central to microservice and system design interviews.

API Gateway Responsibilities

An API gateway handles everything a client needs before their request reaches a business logic service. This separation of concerns lets microservices focus on their domain without reimplementing infrastructure logic.

API Gateway responsibility matrix
// API Gateway cross-cutting concerns:
//
// Client Request ──▶ API Gateway ──▶ Backend Service
//
// What the gateway handles:
// ┌──────────────────────────────────────────────────────────┐
// │ 1. SSL Termination     — decrypt HTTPS, forward HTTP     │
// │ 2. Authentication      — validate JWT / API key / OAuth  │
// │ 3. Authorisation       — check scopes / roles            │
// │ 4. Rate Limiting       — 100 req/min per API key         │
// │ 5. Request Routing     — /api/courses → course-service   │
// │                          /api/users   → user-service     │
// │ 6. Protocol Translation— REST → gRPC (grpc-gateway)      │
// │ 7. Request Transform   — add headers, rewrite paths      │
// │ 8. Response Aggregation— fan out to N services, merge    │
// │ 9. Caching             — cache GET responses at edge     │
// │ 10. Observability      — unified access logs, metrics    │
// └──────────────────────────────────────────────────────────┘

JWT Validation at the Gateway

The gateway validates JWTs before forwarding requests to backend services. This means backend services receive only authenticated requests and never need to handle token parsing. The gateway can forward the decoded user identity (user ID, roles) as headers to the backend.

JWT validation at API gateway
// Kong JWT plugin (declarative config):
plugins:
  - name: jwt
    config:
      key_claim_name: kid
      claims_to_verify: [exp, nbf]
      maximum_expiration: 3600

// After validation, Kong forwards to backend:
// X-Consumer-ID: uuid
// X-Consumer-Username: user@aicancode.org
// X-Consumer-Custom-ID: 12345

// AWS API Gateway — JWT authorizer (Terraform):
resource "aws_api_gateway_authorizer" "jwt" {
  name                   = "jwt-authorizer"
  rest_api_id            = aws_api_gateway_rest_api.main.id
  authorizer_uri         = aws_lambda_function.jwt_validator.invoke_arn
  authorizer_result_ttl_in_seconds = 300   // cache auth result for 5min
  type                   = "TOKEN"
  identity_source        = "method.request.header.Authorization"
}

// Backend (Spring Boot) — trusts forwarded identity, no JWT parsing needed:
@GetMapping("/profile")
public ResponseEntity<User> getProfile(
    @RequestHeader("X-User-ID") String userId) {   // set by gateway
  return ResponseEntity.ok(userService.findById(userId));
}

Rate Limiting & Quota Management

Rate limiting at the gateway protects backend services from overload and enforces fair usage. Limits can be per-IP (unauthenticated traffic), per-user, or per-subscription tier. The gateway uses token bucket or sliding window algorithms stored in a shared Redis instance.

Rate limiting configuration at the API gateway
// Kong rate limiting plugin:
plugins:
  - name: rate-limiting
    config:
      minute: 100          // 100 req/min per consumer
      hour: 5000
      policy: redis        // shared across gateway instances
      redis_host: redis.internal
      redis_port: 6379
      error_code: 429
      error_message: "Rate limit exceeded. Retry after 60 seconds."

// AWS API Gateway usage plan:
resource "aws_api_gateway_usage_plan" "free_tier" {
  name = "free-tier"
  throttle_settings {
    burst_limit = 50     // max concurrent request burst
    rate_limit  = 10     // steady-state req/sec
  }
  quota_settings {
    limit  = 1000        // 1000 req/month
    period = "MONTH"
  }
}

// Rate limit headers returned to client:
// X-RateLimit-Limit: 100
// X-RateLimit-Remaining: 87
// X-RateLimit-Reset: 1744567200

API Gateway vs Service Mesh

An API Gateway handles north-south traffic (client → service). A service mesh handles east-west traffic (service → service). They are complementary, not alternatives. The gateway is the external entry point; the mesh provides mTLS, observability, and load balancing between internal services.

API Gateway vs Service Mesh responsibilities
// Traffic flow with both gateway and mesh:
//
//              Internet
//                 │
//         [API Gateway]        ← North-South: auth, rate limit, routing
//           /        \
//    [Service A]  [Service B]
//       │    \       │
//      [sidecar] [sidecar]     ← East-West: Istio/Linkerd sidecars
//                              ← handles: mTLS, retries, circuit breaking,
//                              ←          distributed tracing, traffic shifting
//
// API Gateway concerns:     Service Mesh concerns:
// ✓ External auth (JWT)     ✓ Internal mTLS (service identity)
// ✓ Rate limiting           ✓ Retries + timeouts
// ✓ SSL termination         ✓ Circuit breaking
// ✓ Public API versioning   ✓ Load balancing (L7)
// ✓ Client-facing docs      ✓ Distributed tracing (Jaeger)

Key Points to Remember

  • 1An API Gateway centralises cross-cutting concerns (auth, rate limiting, routing, logging) so backend services focus on business logic.
  • 2JWT validation at the gateway means backend services receive authenticated requests and decoded identity as trusted headers.
  • 3Rate limiting is enforced at the gateway using shared Redis state — limits can be per-IP, per-user, or per-subscription tier.
  • 4API Gateway handles north-south traffic (client↔service); service mesh handles east-west traffic (service↔service) — they are complementary.
  • 5AWS API Gateway authorizer caches auth results (configurable TTL) to avoid hitting the Lambda authorizer on every request.

Interview Questions

Sign in to ask Aria
1

What is an API Gateway and why do microservices need one?

EasyInfosys
2

How does JWT validation work at the API Gateway layer?

MediumThoughtWorks
3

What is the difference between an API Gateway and a reverse proxy?

MediumAmazon
4

What is the difference between an API Gateway and a Service Mesh?

HardRazorpay
5

How would you implement per-user rate limiting in a distributed API gateway?

HardEqual Experts

Ask Aria about API Gateway

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Sign in to save personal notes for this topic.

Discussion

Sign in to join the discussion.

Loading discussion…