How Kubernetes Works
IntermediateKubernetes is a container orchestration system: you tell it what you want (run 10 replicas of this container, keep them alive, expose them on port 80) and it continuously works to make reality match your desired state. It handles scheduling containers onto nodes, replacing crashed containers, scaling up on load, rolling out updates without downtime, and load balancing traffic. Kubernetes turned the operational complexity of running thousands of containers into a declarative configuration problem.
Think of it like an air traffic control system
Air traffic control doesn't fly the planes — it coordinates them. It assigns runways (nodes), manages take-off and landing schedules (scheduling), monitors for problems (health checks), and reroutes flights if a runway closes (self-healing). Kubernetes is ATC for containers: it doesn't run your code, but it decides where your containers run, keeps them healthy, and manages their communication.
Step by Step
Key Concepts
Pod
The smallest deployable unit in Kubernetes. A pod wraps one or more containers that share a network namespace (same IP address) and can share volumes. Containers in a pod are always co-located on the same node. Pods are ephemeral — they can be killed and replaced at any time. Never rely on a pod's IP address or local storage.
Deployment
A higher-level resource that manages a ReplicaSet (which manages pods). You define a desired number of replicas and a pod template. The Deployment controller ensures that many pods always run. Rolling updates replace pods one at a time; rollbacks revert to the previous ReplicaSet. This is how you deploy new application versions with zero downtime.
Service
A stable virtual IP and DNS name that load balances traffic across pods. Types: ClusterIP (only reachable within the cluster), NodePort (exposed on every node's IP), LoadBalancer (provisions a cloud load balancer), ExternalName (maps to a DNS name). Services use label selectors to find target pods — no manual configuration when pods change.
etcd
The distributed key-value store that serves as Kubernetes' backing store. All cluster state is stored here. etcd uses the Raft consensus algorithm to maintain consistency across multiple replicas. Production clusters run 3 or 5 etcd instances for high availability. etcd is the most critical component — back it up regularly.
Namespace
A virtual cluster within a Kubernetes cluster. Used to isolate resources between teams or environments (dev/staging/prod). Resources in different namespaces are isolated from each other by default. RBAC policies can restrict which users can access which namespaces.
ConfigMap and Secret
ConfigMaps store non-sensitive configuration (environment variables, config files) separate from the container image. Secrets store sensitive data (passwords, API keys, TLS certs) base64-encoded (not encrypted at rest by default — use Sealed Secrets or external secret managers for real security). Both can be mounted as files or environment variables.
Horizontal Pod Autoscaler (HPA)
Automatically scales the number of pod replicas based on CPU utilisation, memory, or custom metrics. When average CPU > 70%, HPA adds pods. When CPU drops, it removes them. Requires the metrics-server add-on. Scaling decisions have a 15-second minimum cooldown. Pair with cluster autoscaler to also add/remove nodes as needed.
Ingress
An API object that manages external HTTP/HTTPS routing to Services. An Ingress resource defines rules (host: api.example.com → service: api, path: /static → service: cdn). An Ingress controller (nginx, Traefik, AWS ALB) watches Ingress resources and configures the actual load balancer. One controller can route traffic for dozens of services.
Key Facts
- Kubernetes was originally designed by Google engineers based on lessons from Borg, Google's internal cluster manager that has run Google's workloads for over 15 years.
- A Kubernetes cluster at Google might run millions of containers. The scheduler makes placement decisions in under 100ms by using a two-phase filter-then-score algorithm rather than evaluating every pod-node combination.
- The control plane components (API server, scheduler, controller manager, etcd) are themselves run as containers on "master" nodes. Kubernetes is self-hosting.
- etcd is the only stateful component of Kubernetes. Losing etcd means losing knowledge of what should be running — nodes continue running their current containers but can no longer be managed. Regular etcd backups are essential.
- Kubernetes uses a watch mechanism (long-lived HTTP connection) rather than polling. When you kubectl apply, controllers are notified in milliseconds — not via periodic polling.
- The CNCF (Cloud Native Computing Foundation) graduated Kubernetes in 2018. Over 100,000 organisations use it in production, making it the dominant container orchestration platform.
Real-World Applications
Zero-downtime deployments
A Deployment rolling update replaces pods one at a time. With maxSurge: 1 and maxUnavailable: 0, Kubernetes always keeps the old version running while adding new pods. readinessProbes ensure new pods only receive traffic when healthy. If a new version has a high error rate, kubectl rollout undo reverts in seconds.
Autoscaling for traffic spikes
An HPA watching CPU utilisation scales the API service from 5 to 50 pods during peak hours, then back down overnight. A Cluster Autoscaler adds EC2 instances when pods can't be scheduled (insufficient capacity) and removes underutilised nodes to reduce costs. Together they handle 10x traffic spikes with no manual intervention.
Multi-tenant SaaS
Each customer's workload runs in a separate Kubernetes Namespace with RBAC policies and NetworkPolicies preventing cross-namespace communication. Resource quotas limit how much CPU/memory each tenant can consume. This provides logical isolation without the cost of separate clusters.
Database operations with StatefulSets
StatefulSets are like Deployments but for stateful workloads. Each pod gets a stable hostname (postgres-0, postgres-1), a persistent volume that follows it, and pods are created/deleted in order. Used for running databases (Postgres, Cassandra, Kafka) in Kubernetes with predictable identities.
Frequently Asked Questions
What is the difference between a Pod and a Deployment?
A Pod is a running container instance. A Deployment is a controller that manages pods: it ensures the right number are running, handles rolling updates, and restarts crashed pods. You almost never create bare Pods directly — if a pod dies without a controller, nothing recreates it. Always use Deployments (or StatefulSets for stateful apps) to manage pods.
How does Kubernetes handle a node failure?
The node controller detects that a node has stopped reporting (heartbeat timeout: 40s by default, eviction after 5 minutes). It marks all pods on that node as "Unknown" and evicts them. The Deployment controller then creates replacement pods and the scheduler places them on healthy nodes. No manual intervention needed — self-healing is automatic.
What is the difference between liveness and readiness probes?
A liveness probe detects if a container is stuck (deadlock, infinite loop) and kills/restarts it. A readiness probe detects if a container is temporarily unable to serve traffic (warming up, overloaded) and removes it from the Service load balancer without restarting it. Use both: liveness for permanent failures, readiness for temporary ones.
Should I run my database in Kubernetes?
It's possible with StatefulSets and persistent volumes, but complex. Managed cloud databases (RDS, Cloud SQL) handle backups, failover, and upgrades automatically. Run stateless services in Kubernetes; use managed databases for persistent state unless you have strong reasons otherwise (cost, latency, multi-cloud requirements). Operators (like the CloudNativePG operator) reduce the operational burden if you must run databases in Kubernetes.