How Docker Works
IntermediateDocker packages your application and all its dependencies into a container — an isolated process that runs the same way everywhere. Under the hood, containers are not virtual machines. They are regular Linux processes isolated using three kernel features: namespaces (what a process can see), cgroups (how much CPU/memory it can use), and union filesystems (layered, copy-on-write storage). Docker adds a developer-friendly layer of tooling on top of these primitives.
Think of it like a standardised shipping container
Before shipping containers, every port needed different equipment and every ship had a different layout. A standardised container can be loaded onto any ship, truck, or train without modification. Docker does the same for software: package your app once in a container, and it runs identically on your laptop, a CI server, or a production Kubernetes cluster. The "ship" (host OS) doesn't matter — the container always looks the same from the inside.
Step by Step
Key Concepts
Image vs Container
An image is a read-only template (a blueprint). A container is a running instance of an image. Multiple containers can run from the same image simultaneously. Analogy: an image is a class definition; a container is an object instance.
Layer Caching
Each Dockerfile instruction creates a cached layer identified by a hash of its content and all previous layers. If the hash matches a cached layer, Docker reuses it. The first changed instruction busts the cache for all subsequent instructions. This is why Dockerfile instruction order matters enormously for build performance.
Linux Namespaces
The kernel feature that provides container isolation. Six namespace types: PID (process IDs), Net (network interfaces), Mount (filesystem mounts), UTS (hostname), IPC (inter-process communication), User (user and group IDs). Docker creates one of each for every container. The host OS can see into all namespaces; containers cannot see each other.
cgroups (Control Groups)
Linux kernel mechanism for limiting, accounting, and isolating CPU, memory, disk I/O, and network bandwidth for groups of processes. Docker uses cgroups to enforce --memory, --cpus, --blkio-weight limits. Without cgroups, containers are just isolated processes with no resource guarantees.
overlay2
The default Docker storage driver on Linux. Uses Linux's OverlayFS to stack multiple read-only image layers and a single read-write container layer. Writes go to the container layer using copy-on-write: the first write to a file copies it from the image layer to the container layer, then modifies it in place.
Docker Compose
A tool for defining and running multi-container applications. A docker-compose.yml file declares services (containers), networks, and volumes. docker compose up starts all services, creates the network, and wires them together. Essential for local development environments (app + database + cache + queue all at once).
Volume
A persistent filesystem mount that survives container restarts. Without volumes, all data written inside a container is lost when it stops. docker run -v mydata:/app/data mounts a Docker-managed volume at /app/data. Bind mounts (-v $(pwd):/app) mount a host directory — used in development to see code changes without rebuilding.
Multi-stage Build
A Dockerfile technique that uses multiple FROM instructions to create a smaller final image. Stage 1 (builder): install build tools, compile the app. Stage 2 (runtime): copy only the compiled binary from stage 1. The final image contains no compiler, source code, or build dependencies — only what the app needs to run. Typical size reduction: 10x.
Key Facts
- Docker containers start in milliseconds. A virtual machine takes minutes to boot because it must initialise a full OS kernel. Containers share the host kernel — there is nothing to boot.
- The overlay2 storage driver means 100 containers running the same nginx image share the same read-only image layers on disk. Only the thin read-write layer per container is unique.
- Docker does NOT provide security isolation as strong as a VM. Containers share the host kernel — a kernel exploit in one container can potentially affect others. For untrusted workloads, use gVisor or Firecracker (lightweight VMs).
- The Docker daemon (dockerd) runs as root by default. Rootless Docker mode (introduced in Docker 19.03) allows running the daemon as a non-root user for improved security.
- Alpine Linux is a 5MB base image commonly used in Docker containers. Ubuntu is ~80MB. Most production images are built FROM alpine or FROM scratch (a completely empty image) to minimise attack surface.
- docker build --no-cache forces a full rebuild by ignoring all cached layers. Useful when you need to fetch updated packages (RUN apt-get update) without a stale cache.
Real-World Applications
Consistent development environments
Instead of "works on my machine", a docker-compose.yml in the repository starts the exact same Postgres version, Redis version, and app configuration for every developer. No more manual setup or version conflicts. New team members are productive on day one.
CI/CD pipelines
Every CI run builds a Docker image, runs tests inside the container, and pushes to a registry if tests pass. The image tag (e.g., git commit SHA) is then deployed to staging/production. Because the image is immutable, what you tested in CI is exactly what runs in production.
Microservices isolation
Each microservice runs in its own container with its own dependencies. The Python ML service uses Python 3.11; the Node.js API uses Node 20; the Go service uses Go 1.22 — all on the same host without dependency conflicts. Resource limits (--memory, --cpus) prevent one service from starving others.
Production image optimisation
Use multi-stage builds to keep images small. Scan images with docker scout or trivy for CVEs. Use distroless base images for the runtime stage — they contain only the language runtime and your app, no shell or package manager for attackers to exploit.
Frequently Asked Questions
What is the difference between Docker and a virtual machine?
A VM virtualises an entire computer including hardware and a full OS kernel. Each VM has its own kernel and boots in minutes. A Docker container shares the host's Linux kernel and is just an isolated process — it starts in milliseconds and uses less RAM. The trade-off: VMs provide stronger isolation (separate kernels) while containers are lighter and faster. For most workloads, containers are sufficient.
How do containers communicate with each other?
Docker creates a virtual bridge network (docker0) by default. Containers on the same network can reach each other by container name (Docker provides DNS resolution). In Docker Compose, all services share a network by default, so your app container can reach the database at the hostname "db" (the service name in docker-compose.yml). Expose ports to the host with -p 8080:80.
What should NOT go in a Docker image?
Never bake secrets (API keys, passwords, certificates) into an image — they become visible to anyone who pulls the image. Pass secrets via environment variables, Docker secrets, or mounted files at runtime. Also avoid dev dependencies and build tools in production images (use multi-stage builds). Keep images as small as possible.
Does Docker work on Mac and Windows?
Docker requires Linux kernel features (namespaces, cgroups). On Mac and Windows, Docker Desktop runs a lightweight Linux VM (using Apple Hypervisor or Hyper-V) and runs containers inside it. Your Docker commands interact with this VM transparently. On Linux, Docker runs containers natively with no VM overhead.