Service Mesh
AdvancedA service mesh is an infrastructure layer that manages service-to-service communication inside a cluster — providing automatic mTLS, traffic management, retries, circuit breaking, and distributed tracing via a sidecar proxy deployed alongside every service.
Overview
As microservice counts grow, managing cross-service concerns (mTLS, retries, circuit breaking, tracing) in application code leads to duplication and drift. A service mesh externalises these to a dedicated data plane. Each service gets a sidecar proxy (Envoy in Istio, linkerd-proxy in Linkerd) that intercepts all inbound and outbound traffic transparently. A control plane (Istio's istiod) distributes configuration to all sidecars via xDS APIs. The application code is unmodified — it still makes plain HTTP calls, and the sidecar handles encryption, retry, and telemetry. Istio and Linkerd are the most widely deployed service meshes.
Sidecar Proxy Pattern
Every service pod runs a sidecar container (Envoy) that is automatically injected by the mesh's admission webhook. The sidecar intercepts all TCP traffic via iptables rules — the application is unaware. This allows the mesh to enforce mTLS, collect metrics, and apply traffic policies without any application code changes.
// Kubernetes pod with Istio sidecar injection:
// (injection is automatic when namespace is labelled)
kubectl label namespace production istio-injection=enabled
// Resulting pod has 2 containers:
// ┌─────────────────────────────────────────────────────┐
// │ Pod: course-service-7f9b4d5-xk2p8 │
// │ ┌──────────────────┐ ┌──────────────────────────┐ │
// │ │ course-service │ │ istio-proxy (Envoy) │ │
// │ │ :8080 │ │ :15001 (outbound) │ │
// │ │ (app code) │ │ :15006 (inbound) │ │
// │ └──────────────────┘ └──────────────────────────┘ │
// └─────────────────────────────────────────────────────┘
// iptables rules redirect all traffic through Envoy transparently:
// Outbound: app → Envoy :15001 → (encrypt mTLS) → destination Envoy
// Inbound: source Envoy → (decrypt mTLS) → Envoy :15006 → app :8080Automatic mTLS
Istio's control plane (istiod) issues SPIFFE X.509 certificates to every workload, rotated every 24 hours. All service-to-service communication is automatically mTLS — no certificate management in application code. The mesh can enforce that every request must be authenticated (STRICT mode) or allow plaintext from outside the mesh (PERMISSIVE mode).
// Istio PeerAuthentication — enforce mTLS in namespace:
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: production
spec:
mtls:
mode: STRICT # all service-to-service traffic must be mTLS
# PERMISSIVE: allow plaintext (migration phase)
// Workload certificate (SPIFFE SVID):
// spiffe://cluster.local/ns/production/sa/course-service
// Issued by istiod CA, rotated every 24h, embedded in Envoy SDS
// AuthorizationPolicy — fine-grained access control:
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: course-service-policy
namespace: production
spec:
selector:
matchLabels:
app: course-service
rules:
- from:
- source:
principals: ["cluster.local/ns/production/sa/api-gateway"]
to:
- operation:
methods: ["GET"]
paths: ["/api/v1/courses*"]Traffic Management: Retries, Timeouts, Circuit Breaking
Istio VirtualService and DestinationRule resources configure traffic behaviour in the data plane — no application code changes needed. This allows uniform retry/timeout policies, and circuit breakers that open when a service becomes unhealthy.
// VirtualService — retries and timeouts:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata: { name: course-service }
spec:
hosts: [course-service]
http:
- timeout: 3s # overall request timeout
retries:
attempts: 3
perTryTimeout: 1s
retryOn: "gateway-error,connect-failure,retriable-4xx"
route:
- destination: { host: course-service, port: { number: 8080 } }
// DestinationRule — circuit breaker (outlier detection):
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata: { name: course-service }
spec:
host: course-service
trafficPolicy:
outlierDetection:
consecutive5xxErrors: 5 # open circuit after 5 errors
interval: 10s # evaluation window
baseEjectionTime: 30s # eject unhealthy instance for 30s
maxEjectionPercent: 50 # never eject more than 50% of pool
connectionPool:
tcp: { maxConnections: 100 }
http: { http1MaxPendingRequests: 50, http2MaxRequests: 1000 }Canary Deployments with Traffic Splitting
The service mesh enables fine-grained traffic splitting — route 5% of traffic to v2 of a service while 95% stays on v1, without any changes to client code or load balancer configuration. Gradually increase the percentage as confidence grows.
// Deploy v2 alongside v1:
// course-service-v1 pods: 10 replicas
// course-service-v2 pods: 1 replica (canary)
// DestinationRule — define subsets:
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata: { name: course-service }
spec:
host: course-service
subsets:
- name: v1
labels: { version: v1 }
- name: v2
labels: { version: v2 }
// VirtualService — 5% canary traffic split:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata: { name: course-service }
spec:
hosts: [course-service]
http:
- route:
- destination:
host: course-service
subset: v1
weight: 95
- destination:
host: course-service
subset: v2
weight: 5 # canary receives 5%
// Gradually increase: 5% → 25% → 50% → 100% → decommission v1Key Points to Remember
- 1A service mesh injects a sidecar proxy (Envoy) into every pod — all network traffic flows through it transparently, enabling mesh-wide policies.
- 2Istio issues SPIFFE X.509 certificates to every workload automatically — mTLS between all services with zero application code changes.
- 3Traffic management (retries, timeouts, circuit breakers, canary splits) is configured via Kubernetes CRDs — no code changes in services.
- 4Outlier detection ejects repeatedly-failing instances from the load-balancing pool, acting as a distributed circuit breaker.
- 5Service mesh is not a replacement for an API Gateway — the gateway handles north-south (external) traffic; the mesh handles east-west (internal) traffic.
Interview Questions
Sign in to ask AriaWhat is a service mesh and why would you use one instead of handling these concerns in application code?
How does Istio implement mTLS without any changes to application code?
What is the difference between a service mesh and an API Gateway?
How would you implement a canary deployment using Istio?
What is outlier detection and how does it relate to the circuit breaker pattern?
Ask Aria about Service Mesh
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.