How Architecture & Design Works

Scaling is how a system handles more load. Vertical scaling (scale up) means giving one machine more CPU, RAM, or disk — simple, but capped by the biggest machine you can buy and a single point of failure. Horizontal scaling (scale out) means adding more machines behind a load balancer — near-limitless and fault-tolerant, but it requires stateless services and coordination. Modern cloud systems favour horizontal scaling, often automatically.

Microservices is an architectural style where a system is built as a collection of small, independently deployable services — each owning its own data, running in its own process, and communicating over a network. Each service does one thing well: User Service, Order Service, Payment Service. Teams can deploy services independently, choose their own tech stacks, and scale each service separately. The trade-off: distributed systems are fundamentally harder to build, test, and operate than monoliths.

Event-Driven Architecture (EDA) is a design approach where services communicate by producing and consuming events rather than making direct API calls. When something happens — "OrderPlaced", "PaymentProcessed", "UserRegistered" — the producing service publishes an event to a broker. Any number of consumers react to it independently, without the producer knowing or caring who they are. EDA enables loose coupling, high scalability, and natural audit trails — at the cost of eventual consistency and increased operational complexity.

An API Gateway is a server that acts as the single entry point for all client requests to a backend system. Instead of clients calling dozens of microservices directly, every request flows through the gateway. It handles the cross-cutting concerns that every API needs: authentication, rate limiting, SSL termination, request routing, and logging — in one place, so individual services don't have to. The gateway abstracts the internal service topology from clients and provides a stable, unified API surface.

A cache stores a copy of data somewhere faster to read than the original source, so repeated requests skip expensive work. Caching is one of the highest-leverage tools in system design: it cuts latency and offloads databases. The hard parts are not storing data but deciding when to update or remove it (invalidation), how to evict when full (LRU/LFU), and how to avoid everyone rebuilding the same entry at once (a stampede).

The CAP theorem says a distributed data store can guarantee at most two of three properties: Consistency (every read sees the latest write), Availability (every request gets a response), and Partition tolerance (the system keeps working despite dropped messages between nodes). Since network partitions are unavoidable in real systems, the real choice is between consistency and availability when a partition happens — making systems either CP or AP.

Rate limiting caps how many requests a client can make in a time window, protecting your service from abuse, accidental floods, and runaway costs. Several algorithms trade precision against memory and burst behaviour: fixed window is simple but bursty at boundaries, sliding window is smoother, the token bucket allows controlled bursts, and the leaky bucket smooths output. In distributed systems the counter lives in a shared store like Redis so all servers enforce one limit.

An operation is idempotent if doing it multiple times has the same effect as doing it once. This matters because networks are unreliable: a client may time out and retry a request that actually succeeded, causing a duplicate charge or order. Idempotency makes retries safe. Some HTTP methods (GET, PUT, DELETE) are naturally idempotent; POST is not, so APIs use an idempotency key to detect and collapse duplicate requests.

Partitioning splits one large table into smaller pieces (partitions) that the database manages as a single logical table. Queries touch only the relevant partitions (partition pruning), maintenance like archiving old data becomes cheap, and indexes stay smaller. Partitioning happens within one database server, which distinguishes it from sharding (across servers). Common strategies are range, hash, and list partitioning, chosen to match your query and retention patterns.

A URL shortener turns a long URL into a short code (bit.ly/abc123) and redirects visitors back to the original. It is a favourite system design interview question because it is simple to state but touches key ideas: generating unique short codes, a read-heavy redirect path, caching, database choice, and scaling to billions of links. The core is a key-value mapping from short code to long URL, optimised for very fast reads.

A monolith packages an entire application as one deployable unit; microservices split it into many small, independently deployable services. Monoliths are simpler to build, test, and deploy, and are the right default for most new projects. Microservices offer independent scaling and team autonomy but add heavy operational and distributed-systems complexity. The choice is really about organisational scale and operational maturity, not just technology — and there is a strong middle ground: the modular monolith.

In a microservices system, instances start, stop, scale, and move constantly, so their network addresses are always changing. Service discovery is how a service finds the current address of another without hard-coded IPs. Instances register themselves in a service registry, health checks remove dead ones, and callers look up healthy instances — either directly (client-side) or through a router (server-side). Kubernetes builds this in via DNS and its internal service abstraction.

A circuit breaker protects a system from cascading failures. When a service keeps calling a dependency that is failing or slow, threads pile up waiting and the caller itself can go down — the failure spreads. A circuit breaker watches the failure rate and, once it crosses a threshold, "trips": it stops calling the failing dependency and fails fast (often returning a fallback) instead. After a cool-down it cautiously tests whether the dependency has recovered.

When one user request flows through a dozen microservices, a single log file cannot tell you where the time went or where it failed. Distributed tracing stitches the whole journey together. Each request gets a unique trace ID, and every service records a span — a timed unit of work — tagged with that trace ID and its parent. A tracing backend assembles the spans into a tree, so you can see the end-to-end path, latency of each hop, and exactly where errors occur.

Eventual consistency is a guarantee used by distributed systems: if no new updates are made, all replicas will converge to the same value — eventually. Reads may briefly return stale data while updates propagate. This relaxation is the price of high availability and low latency at scale. It is perfectly acceptable for many workloads (feeds, counters, catalogs) but wrong for others (bank balances), so systems increasingly let you tune consistency per operation.

The system design interview tests whether you can design a large-scale system and reason about trade-offs — not whether you memorised one right answer. Interviewers watch your process: how you clarify an open-ended problem, structure a solution, and justify decisions. A reliable framework keeps you on track: clarify requirements, estimate scale, define the API and data model, draw a high-level design, deep-dive into a component, and then identify and resolve bottlenecks. Communicating clearly throughout matters as much as the design itself.

The twelve-factor app is a methodology of twelve principles for building software-as-a-service applications that are portable, scalable, and easy to deploy on modern cloud platforms. It emerged from lessons running apps at scale and codifies practices like storing config in the environment, keeping processes stateless, treating backing services as attached resources, and strictly separating build, release, and run. Following it makes an app cloud-ready: it scales horizontally, deploys continuously, and runs the same everywhere.

Consistent hashing is a technique for distributing keys across a changing set of servers so that adding or removing a server moves as few keys as possible. Naive modulo hashing (hash(key) % N) reshuffles almost everything when N changes — catastrophic for a cache. Consistent hashing places both servers and keys on a ring; each key belongs to the next server clockwise. When a server joins or leaves, only its neighbouring keys move. It underpins distributed caches, Cassandra, and DynamoDB.

A distributed lock lets multiple servers coordinate exclusive access to a shared resource — only one worker runs a job, only one process writes a file. Unlike an in-process lock, it must work across machines that can crash or lose network connectivity, which makes it surprisingly hard. The common approach is Redis SET NX with a TTL, but expiry and clock issues can let two holders think they own the lock, so critical systems add fencing tokens.

A notification system delivers messages to users across channels — email, push, SMS, in-app — reliably and at scale. The design decouples the event that triggers a notification from its delivery using message queues, fans out to per-channel workers, respects user preferences and rate limits, and handles the reality that external providers fail. It is a classic system design question because it combines queues, retries, idempotency, and third-party integration.

A service mesh moves cross-cutting networking concerns — encryption, retries, timeouts, load balancing, and observability — out of your application code and into the infrastructure. It does this by placing a lightweight proxy (a sidecar) next to each service instance to intercept all its traffic. A central control plane configures every proxy. The result: consistent security, traffic control, and telemetry across all services, in any language, with no library in your code.

The saga pattern manages a transaction that spans multiple microservices, each with its own database, where a single ACID transaction is impossible. A saga breaks the operation into a sequence of local transactions; each step publishes an event that triggers the next. If a step fails, the saga runs compensating transactions to undo the previous steps. It trades the strong guarantees of a distributed transaction for availability and eventual consistency.

Two-phase commit (2PC) is a protocol for making a transaction atomic across multiple databases or services — either all commit or all abort. A coordinator runs two rounds: first it asks every participant to prepare (can you commit?), and only if all say yes does it tell them all to commit. It guarantees atomicity, but it is blocking: if the coordinator crashes at the wrong moment, participants can be stuck holding locks. This fragility is why microservices usually prefer sagas.

CQRS — Command Query Responsibility Segregation — splits a system into two models: one for writes (commands that change state) and one for reads (queries that return data). In a traditional CRUD app, one model does both, which forces awkward compromises when reads and writes have very different needs. CQRS lets each side be optimised independently: a normalised write model for correctness, and denormalised read models tuned for fast, specific queries. It pairs naturally with event sourcing.

Event sourcing stores the full history of what happened as an immutable, append-only sequence of events, rather than just the current state. Instead of updating a row to balance = 90, you append events like Deposited(100) and Withdrew(10); current state is derived by replaying them. This gives a perfect audit trail, the ability to reconstruct past states, and a natural fit with CQRS — at the cost of more complexity and eventual consistency.