How CI/CD Works
IntermediateCI/CD (Continuous Integration / Continuous Delivery) is the practice of automating the steps between writing code and running it in production. Every code push triggers a pipeline: run tests, build a container image, push it to a registry, deploy to staging, run integration tests, and deploy to production — all without manual steps. The goal is to ship code changes in minutes rather than weeks, catch bugs early when they're cheap to fix, and make deployments a boring, routine event rather than a stressful ceremony.
Think of it like a car assembly line
In a car factory, each station on the assembly line performs one task (weld chassis, install engine, paint, quality check) and passes the car to the next station automatically. If a quality check fails, the line stops and the problem is fixed immediately, before hundreds of defective cars are produced. CI/CD pipelines are assembly lines for software: each stage does one job, problems are caught early, and the final product is automatically shipped if every stage passes.
Step by Step
Key Concepts
Continuous Integration (CI)
The practice of merging all developer work to a shared branch multiple times per day and running automated tests on every push. The goal: catch integration bugs early (while they're cheap to fix) rather than discovering them during a big quarterly release. Requires a fast, reliable automated test suite.
Continuous Delivery vs Deployment
Continuous Delivery: every passing build is deployable to production and deployment is automated, but a human approves the final production push. Continuous Deployment: the pipeline deploys to production automatically on every passing build — no human approval. Most teams use Continuous Delivery for production-critical systems; Continuous Deployment for internal tools or fast-moving products.
Pipeline as Code
CI/CD pipeline configuration stored as code (YAML, Groovy) in the same repository as the application. GitHub Actions uses .github/workflows/*.yml, GitLab CI uses .gitlab-ci.yml, CircleCI uses .circleci/config.yml. Pipeline changes go through code review, are version-controlled, and can be rolled back like any other code change.
Blue-Green Deployment
Running two identical production environments: blue (current live) and green (new version). Deploy to green and run tests. When confident, switch the load balancer from blue to green — zero downtime switch. Blue stays running as an instant rollback target. Switch back to blue if problems emerge. After confidence, decommission blue.
Canary Deployment
Gradually roll out a new version to a small percentage of users (1%, 5%, 10%) before releasing to all. Monitor error rates and latency for the canary group. If metrics look good, increase the percentage incrementally. If problems appear, route all traffic back to the stable version. Named after canaries in coal mines — the canary catches problems before they affect everyone.
Artifact Registry
A versioned store for build outputs. Docker registries (AWS ECR, Google Artifact Registry, Docker Hub) store container images. Maven repositories (Nexus, Artifactory) store JAR files. Every build produces a uniquely tagged artifact stored here. The registry is the source of truth for what versions exist and enables rollbacks to any previous version.
Feature Flags
Configuration that enables/disables features at runtime without a deployment. New code is deployed but the feature is disabled by default. It can be enabled for specific users, percentages, or environments via a flag service (LaunchDarkly, Unleash). Separates deployment (shipping code) from release (enabling features) — reducing deployment risk.
Shift Left
Moving quality checks earlier in the development process (to the "left" of a timeline). Static analysis, security scanning, and tests run in the IDE and on every commit instead of before a quarterly release. Finding a bug in CI costs 10x less to fix than finding it in production. Shift-left testing is the core philosophy behind CI.
Key Facts
- Google deploys to production over 5,000 times per day across all services. Amazon deploys once every 11.6 seconds. This scale is only possible with fully automated CI/CD pipelines.
- The DORA (DevOps Research and Assessment) report identifies deployment frequency, lead time for changes, time to restore service, and change failure rate as the four key metrics of high-performing software teams. All four improve with CI/CD.
- GitHub Actions, GitLab CI, CircleCI, Jenkins, and Buildkite are the most popular CI platforms. GitHub Actions is now the most widely used for open-source projects due to its free tier and tight GitHub integration.
- Studies show that teams practicing CI have 46% higher deployment frequency, 440x faster lead time, and 170x faster mean time to recover from failures compared to teams without CI/CD (2023 DORA report).
- Trunk-based development (everyone pushes to main frequently) is the branching strategy that maximises CI benefits. Long-lived feature branches contradict the "integrate continuously" principle.
- The average cost of a production incident caused by a bad deployment at a large company is over $500,000. Automated testing and incremental deployment strategies (canary, blue-green) directly reduce this risk.
Real-World Applications
Pull request validation
Every PR triggers a CI workflow: install deps (cached for speed), run unit tests in parallel across 4 workers, run lint and type checking, build the Docker image, push to a staging registry, deploy to a preview environment, and post the preview URL as a PR comment. All of this completes in 5–8 minutes on a typical Node.js service.
Automated security scanning
Shift-left security: run Snyk or Trivy on every build to scan Docker images for CVEs. Run SAST (Static Application Security Testing) tools on source code. Fail the build if critical vulnerabilities are found. This catches security issues in the PR before code reaches production — far cheaper than fixing them post-deployment.
Database migrations in CI/CD
Run database migrations as part of the deployment pipeline (not manually). Use backward-compatible migrations: add new columns without removing old ones, deploy the app, then drop old columns in a separate migration. This enables zero-downtime schema changes. Flyway and Liquibase version-control SQL migrations like code.
Multi-environment promotion
A common pipeline: merge to main → deploy to dev (automatic) → run integration tests → deploy to staging (automatic) → run smoke tests → deploy to production (manual approval). Each stage uses the same immutable Docker image tag. Promoting an image between environments means updating a tag reference, not rebuilding.
Frequently Asked Questions
What is the difference between CI and CD?
CI (Continuous Integration) is about frequently merging code and running automated tests to catch integration bugs early. CD (Continuous Delivery/Deployment) is about automating the release process so that code that passes CI can be deployed to production reliably and frequently. CI is a prerequisite for CD — you need confidence from automated tests before automating deployments.
How do I store secrets in a CI/CD pipeline?
Never hardcode secrets in pipeline files (they go into git history and logs). Use your CI platform's secret management: GitHub Encrypted Secrets, GitLab CI Variables, CircleCI Contexts, or HashiCorp Vault. Inject secrets as environment variables at runtime. Audit who has access to production secrets and rotate them regularly. Use OIDC to exchange short-lived tokens instead of long-lived credentials where possible.
How do you speed up slow CI pipelines?
Cache dependencies between runs (node_modules, Maven .m2, pip cache). Parallelise test suites across multiple workers. Run lint/type-check jobs in parallel with tests. Use faster test environments (in-memory SQLite for unit tests instead of real Postgres). Only run affected tests on PR changes (test impact analysis tools like Jest --changedSince). Set a target: CI should complete in under 10 minutes.
What is GitOps?
GitOps is a CD pattern where the desired state of infrastructure and deployments is declared in a Git repository, and an automated agent (Argo CD, Flux) continuously reconciles the actual state to match. Deploying means merging a PR that updates the image tag in a deployment YAML. The Git history is a full audit log. Argo CD is the most popular GitOps tool for Kubernetes.