Home/Learn/Docker/Production Patterns — Health Checks, Restart Policies & Logging

Production Patterns — Health Checks, Restart Policies & Logging

Advanced

Production

Running containers in production requires health checks, restart policies, log management, and resource limits to ensure reliability, observability, and efficient resource use.

Overview

A container that passes `docker ps` may still be running a hung application. HEALTHCHECK lets Docker and orchestrators probe whether the application inside is actually healthy, enabling automated restarts and zero-downtime deployments. Restart policies (always, on-failure, unless-stopped) control what happens when a container exits. Log management requires forwarding container logs to a centralised system (CloudWatch, Loki, ELK) — the default json-file driver loses logs after container removal. In orchestrated environments (Docker Swarm, Kubernetes), these patterns are enforced by the platform.

Health Checks

HEALTHCHECK defines a command Docker runs periodically inside the container. The result (exit 0 = healthy, exit 1 = unhealthy) determines the container's health state and triggers Compose depends_on: service_healthy conditions.

Dockerfile + Compose — health checks

# Dockerfile HEALTHCHECK

HEALTHCHECK --interval=30s \     # check every 30s

            --timeout=5s \       # mark unhealthy if check takes >5s

            --start-period=10s \ # grace period after container starts

            --retries=3 \        # 3 consecutive failures = unhealthy

  CMD curl -f http://localhost:8000/health || exit 1



# Or use wget (smaller than curl in Alpine)

HEALTHCHECK CMD wget -qO- http://localhost:8000/health || exit 1



# Check health status

docker ps

# STATUS

# Up 5 minutes (healthy)

# Up 2 minutes (unhealthy)

# Up 10 seconds (starting)   â† during start-period



# Inspect health history

docker inspect --format='{{json .State.Health}}' my-container | jq



# In docker-compose.yml (inline healthcheck):

services:

  api:

    image: my-api

    healthcheck:

      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]

      interval: 30s

      timeout: 10s

      retries: 3

      start_period: 40s

Restart Policies & Graceful Shutdown

Restart policies control container recovery after exit. Graceful shutdown (handling SIGTERM in your app) ensures in-flight requests complete before the container stops.

bash/Node.js — restart policies and graceful shutdown

# Restart policies:

# no          â†’ never restart (default)

# always      â†’ always restart (even on docker daemon restart)

# on-failure  â†’ restart only on non-zero exit code

# unless-stopped â†’ restart always, except when manually stopped



docker run -d --restart=unless-stopped my-app



# In Compose:

services:

  api:

    restart: unless-stopped



# Graceful shutdown — handle SIGTERM in your app:



// Node.js — handle SIGTERM

const server = app.listen(3000)



process.on('SIGTERM', () => {

  console.log('SIGTERM received — closing HTTP server gracefully')

  server.close(() => {

    console.log('HTTP server closed')

    process.exit(0)

  })

  // Force exit after 30s if connections don't drain

  setTimeout(() => process.exit(1), 30_000)

})



# docker stop sends SIGTERM, waits 10s (default), then sends SIGKILL

# Increase grace period if your app needs more time:

docker stop --time=30 my-container

Log Management

Docker containers write stdout/stderr to the Docker daemon's log driver. The default json-file driver stores logs locally (lost after docker rm). Production needs a centralised logging driver.

bash — log drivers and rotation

# Default logging driver (local file, lost on docker rm)

docker logs my-container                    # view logs

docker logs -f --tail=100 my-container      # follow, last 100 lines



# Configure log driver:

# 1. Per-container

docker run --log-driver=awslogs \

  --log-opt awslogs-group=/my-app/prod \

  --log-opt awslogs-region=us-east-1 \

  my-app



# 2. Docker daemon default (daemon.json)

{

  "log-driver": "json-file",

  "log-opts": {

    "max-size": "10m",    # rotate at 10 MB

    "max-file": "3"       # keep 3 rotated files

  }

}



# 3. Sidecar pattern (recommended for Kubernetes)

# App container writes to stdout â†’ log collector sidecar (Fluentd/Promtail)

# ships logs to centralised store (Loki, CloudWatch, Elasticsearch)



# Available log drivers: json-file, syslog, journald, awslogs, gcplogs, splunk, none

Key Points to Remember

1HEALTHCHECK defines how Docker probes application health (not just process alive).
2Unhealthy containers are automatically restarted based on the restart policy.
3Handle SIGTERM in your application for graceful shutdown — drain in-flight requests before exiting.
4Default json-file log driver loses logs when a container is removed — use centralised logging in production.
5Set max-size and max-file on the json-file driver to prevent disk exhaustion.
6The start-period in HEALTHCHECK prevents premature unhealthy marking during slow startup (e.g., JVM warmup).

Interview Questions

What is the difference between a container being "running" and "healthy"?

How does graceful shutdown work in Docker?

Ask Aria about Production Patterns — Health Checks, Restart Policies & Logging

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Discussion

Loading discussion…

Docker Internals — Namespaces, cgroups & OverlayFS

From Docker to Kubernetes