How API Gateways Work
IntermediateAn API Gateway is a server that acts as the single entry point for all client requests to a backend system. Instead of clients calling dozens of microservices directly, every request flows through the gateway. It handles the cross-cutting concerns that every API needs: authentication, rate limiting, SSL termination, request routing, and logging — in one place, so individual services don't have to. The gateway abstracts the internal service topology from clients and provides a stable, unified API surface.
Think of it like a hotel concierge
When you arrive at a large hotel, you don't navigate to each department yourself — room service, housekeeping, maintenance, spa. The concierge (API Gateway) is your single point of contact. You tell them what you need; they handle authentication (verify you're a registered guest), route your request to the right department, and return the answer. If a department is busy, they manage that. You never need to know the hotel's internal structure — just the concierge's desk.
Step by Step
Key Concepts
Reverse Proxy vs API Gateway
A reverse proxy forwards requests to backend servers (nginx, HAProxy). An API gateway is a reverse proxy with additional capabilities: JWT auth, API key management, rate limiting, request transformation, response aggregation, developer portal, analytics. Nginx can be an API gateway with plugins. Kong is nginx-based with a rich plugin system. AWS API Gateway is fully managed.
Rate Limiting
Controlling how many requests a client can make in a time window. Algorithms: Fixed window (count resets every minute — can allow 2x limit at window boundary). Sliding window (count over the past 60 seconds — more accurate). Token bucket (refills at a fixed rate — allows controlled bursts). Leaky bucket (processes at a fixed rate — smooths bursts). Most gateways use token bucket or sliding window.
BFF (Backend For Frontend)
A pattern where each client type (mobile, web, partner) has its own dedicated gateway/API layer. The mobile BFF aggregates data optimally for mobile (fewer, larger payloads). The web BFF optimises for the SPA. Clients don't share an API — each gets exactly the data it needs in the right shape. Solves over-fetching and under-fetching without a general-purpose API layer serving all clients.
Request Aggregation
The gateway calls multiple backend services, assembles their responses into one, and returns a single response to the client. Instead of the client making 5 API calls (user profile + recent orders + recommendations + loyalty points + notifications), it makes one call to the gateway which fans out to 5 services in parallel and merges the responses. Reduces round trips and latency for clients.
Circuit Breaking at the Gateway
If a backend service consistently returns errors or times out, the gateway opens the circuit: it stops forwarding requests to that service and returns a cached response or a graceful error immediately. This protects clients from cascading failures and protects the struggling backend from additional load. The circuit resets after the service recovers.
API Versioning via Gateway
Route /api/v1/* to v1 services and /api/v2/* to v2 services. Old clients continue using v1 endpoints while new clients use v2. When all clients have migrated, decommission v1 at the gateway — backend services never need to handle versioning themselves. The gateway owns the public contract.
Service Mesh vs API Gateway
Both handle cross-cutting concerns, but at different layers. API Gateway: north-south traffic (external clients to your services). Service Mesh: east-west traffic (service to service within the cluster). They are complementary. Gateway handles client-facing concerns (auth, rate limiting, public API contracts). Service mesh handles internal concerns (mTLS, service retries, internal observability).
Developer Portal
Many API gateways (Kong, AWS API Gateway, Apigee) include a developer portal — a website where external developers can: browse API documentation (auto-generated from OpenAPI specs), create accounts, get API keys, test endpoints interactively, monitor their usage and quotas. Essential for public APIs and partner integrations.
Key Facts
- Netflix's Zuul gateway processes over 2 billion API calls per day. It handles authentication, routing, rate limiting, and request transformation for all Netflix client traffic across web, mobile, and TV apps.
- Kong (open-source API gateway, built on nginx) has over 300 million downloads. Enterprises use it to manage thousands of APIs across multiple teams and environments.
- AWS API Gateway supports over 1 million API calls per second per region, scales automatically, and charges per request ($3.50 per million requests). Fully managed — no infrastructure to operate.
- GraphQL APIs are increasingly implemented as a layer at the gateway (or just behind it), where a single /graphql endpoint aggregates data from multiple microservices — acting as a powerful BFF for all web/mobile clients.
- OpenAPI (Swagger) specifications can be imported directly into AWS API Gateway, Kong, and Azure API Management — automatically creating routing rules and documentation from the spec. Codegen and API design align.
- The average API call to a modern web app passes through 3–7 network hops: browser → CDN → API Gateway → Load Balancer → Service → Database. Each hop adds latency. Gateway co-location with services (same data center, private network) minimises gateway-to-service latency.
Real-World Applications
Unified authentication across services
Ten microservices each required auth. Instead of each one validating JWTs (duplicated logic, inconsistent behaviour), all requests flow through the gateway. The gateway validates the JWT in <1ms, extracts user ID and roles, and adds X-User-Id and X-User-Role headers. Backend services trust these headers — no JWT library needed. Rotating signing keys only requires updating the gateway.
API monetisation and usage tracking
A public API platform assigns each developer an API key. The gateway tracks requests per key, enforces monthly quotas (free: 10k requests/month, paid: 1M/month), blocks over-quota keys with 429, and exports usage data to a billing system. Developers see their usage in the developer portal in real time.
Mobile app API aggregation
A mobile app's home screen needs data from 5 services. Instead of 5 network requests (high latency on mobile), the gateway aggregates them in one call. The mobile BFF calls all 5 services in parallel (internal network — sub-millisecond), merges the responses, and returns one optimised payload. The app makes one request; the gateway does the fan-out.
Zero-downtime service migration
Migrating the Order Service to a new implementation. The gateway is configured to route 5% of /api/orders traffic to the new service (canary). Monitor error rates in the gateway's metrics. If the new service behaves correctly, gradually shift to 50%, then 100%. Rollback is instant — change the routing rule. Clients never see any of this.
Frequently Asked Questions
What is the difference between an API gateway and a load balancer?
A load balancer (Layer 4 or 7) distributes traffic across instances of the same service for scalability and availability. An API gateway routes traffic to different services based on URL path, performs authentication, rate limiting, transformation, and aggregation. Both are reverse proxies, but they solve different problems. In production, both are used: the API gateway routes to services, each service is behind its own load balancer. AWS: ALB is a load balancer; API Gateway is the application-layer gateway.
Does an API gateway become a performance bottleneck?
It can, but modern gateways are designed for high throughput. nginx processes millions of requests per second; Kong handles hundreds of thousands. The gateway adds ~1–5ms overhead. To prevent the gateway from becoming a bottleneck: use horizontal scaling (multiple gateway instances behind a load balancer), avoid complex transformations in the request path, cache JWT public keys locally, and keep rate limiting state in Redis (shared across instances). Monitor gateway latency separately from service latency.
Should I build my own API gateway?
Almost never. Building a production-grade API gateway (auth, rate limiting, circuit breaking, observability, developer portal, graceful degradation) is a major engineering project. Use Kong (open source, self-hosted), AWS API Gateway (managed), Apigee (Google), Azure API Management, or Traefik. Reserve engineering effort for features that differentiate your product. The exception: some companies build thin custom gateways for very specific performance or control requirements (Netflix Zuul, Uber's Flipt).
How do you handle gateway failures?
The API gateway is a critical single point of failure — if it goes down, all API traffic stops. Mitigations: deploy multiple gateway instances behind a load balancer, use cloud-managed gateways with built-in redundancy (AWS API Gateway has 99.9% SLA), deploy across multiple availability zones, and have a runbook for emergency bypass. Monitor gateway health and latency independently. Some architectures use a primary gateway and a fallback that does minimal processing (auth only) for emergencies.