Bandwidth, Latency & Throughput
BeginnerBandwidth is maximum capacity, latency is delay, and throughput is actual data transferred per second. Understanding the relationship between them is essential for designing and debugging performant distributed systems.
Overview
Three terms define network performance. Bandwidth is the theoretical maximum data rate of a link (e.g., 1 Gbps Ethernet) — analogous to a highway's number of lanes. Latency is the time for a single packet to travel from source to destination (round-trip time, RTT) — analogous to the speed limit on that highway. Throughput is the actual amount of data successfully transferred per second — always ≤ bandwidth, reduced by latency, packet loss, and protocol overhead. For backend engineers, latency dominates small-payload APIs (a 1 KB response takes 50ms on a 150ms-latency WAN link regardless of bandwidth), while bandwidth matters for bulk transfers. TCP's congestion window ties throughput directly to RTT: throughput ≈ window_size / RTT.
Definitions and Relationship
High bandwidth does not mean low latency — a satellite link can be 100 Mbps but have 600ms RTT. Throughput is bounded by both bandwidth and latency (the Bandwidth-Delay Product). Applications must be designed with realistic network characteristics in mind.
// Bandwidth-Delay Product (BDP):
// BDP = Bandwidth × RTT
// = the amount of data "in flight" at any moment
// Example: 1 Gbps link, 100ms RTT
// BDP = 1,000,000,000 bits/s × 0.1s = 100,000,000 bits = 12.5 MB
// To fully utilise this link, the TCP window must be ≥ 12.5 MB
// Latency components:
// 1. Propagation delay = distance / speed of light in medium
// London → New York ≈ 5,570 km, ~28ms one-way
// 2. Transmission delay = packet size / bandwidth
// 1 KB packet on 1 Gbps link = 8,000 bits / 1e9 bps = 0.008ms
// 3. Queuing delay = time waiting in router buffers (variable)
// 4. Processing delay = time for router to inspect headers
// Rule of thumb:
// < 1ms — same datacenter (LAN)
// 1–10ms — same region / CDN edge
// 50–100ms — same continent
// 100–300ms — cross-continentImpact on Backend Applications
Latency is the dominant factor for typical API payloads (< 100 KB). Bandwidth matters for streaming, file uploads, and bulk data transfers. Connection pooling, pipelining, and caching all exist to hide or reduce the latency penalty.
// Latency math for a microservice call:
// Payload: 1 KB, Bandwidth: 100 Mbps, RTT: 50ms
// Transmission time = 1 KB / 100 Mbps = 0.08ms (negligible)
// Propagation RTT = 50ms (dominates)
// A chain of 5 synchronous microservice calls:
// Total latency = 5 × 50ms = 250ms (without parallelism)
// Solution — run independent calls in parallel:
CompletableFuture<User> userF = CompletableFuture.supplyAsync(() -> userService.get(id));
CompletableFuture<Orders> ordersF = CompletableFuture.supplyAsync(() -> orderService.get(id));
CompletableFuture<Profile> profileF = CompletableFuture.supplyAsync(() -> profileService.get(id));
CompletableFuture.allOf(userF, ordersF, profileF).join();
// Total latency ≈ 50ms (max of parallel calls), not 150msKey Points to Remember
- 1Bandwidth = max capacity of a link (Gbps). Latency = delay for a packet to travel (ms). Throughput = actual data rate achieved.
- 2Throughput ≤ Bandwidth; reduced by latency, packet loss, and overhead.
- 3Bandwidth-Delay Product = Bandwidth × RTT — the amount of data in-flight to saturate the link.
- 4For small payloads (APIs), latency dominates; for large payloads (file transfers), bandwidth dominates.
- 5RTT ranges: ~0.1ms (LAN), ~1–10ms (same region), ~50–150ms (cross-continent).
- 6Parallel async calls (CompletableFuture) reduce perceived latency in microservice fan-outs.
Interview Questions
Sign in to ask AriaWhat is the difference between bandwidth, latency, and throughput?
Why can high bandwidth not compensate for high latency in an API call chain?
What is the Bandwidth-Delay Product and why does it matter for TCP performance?
How would you reduce perceived latency in a microservice that makes 5 downstream calls?
Ask Aria about Bandwidth, Latency & Throughput
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.