Secrets Management
IntermediateInject secrets via HashiCorp Vault, AWS Secrets Manager, or Kubernetes Secrets mounted as env variables; never commit credentials to source control.
Overview
Secrets management addresses how microservices receive credentials (database passwords, API keys, TLS certificates) without embedding them in code or config files committed to version control. The principal options are: Kubernetes Secrets (base64-encoded, not encrypted at rest by default — require etcd encryption), HashiCorp Vault (centralised secret store with dynamic secrets, audit logs, and fine-grained policies), and cloud-native services (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault). The best practice hierarchy: never hardcode → avoid env vars in container images → prefer mounted files/Vault agent injection → rotate secrets regularly and audit access.
Kubernetes Secrets — injection as env vars and volumes
Kubernetes Secrets store base64-encoded data. Inject as environment variables (risk: exposed in ps aux and logs) or as mounted files (safer — only visible to processes that read the file). Enable etcd encryption at rest in production clusters to prevent secret extraction from etcd backups. Use RBAC to restrict which pods can access which Secrets.
# Create a Secret
kubectl create secret generic db-credentials \
--from-literal=username=app_user \
--from-literal=password=supersecret
# Pod spec — mount as files (preferred over env vars)
apiVersion: v1
kind: Pod
spec:
containers:
- name: order-service
image: order-service:1.0.0
# Option 1: env var injection (simpler but less secure)
env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-credentials
key: password
# Option 2: volume mount (safer — not in process env)
volumeMounts:
- name: db-secret
mountPath: /etc/secrets/db
readOnly: true
volumes:
- name: db-secret
secret:
secretName: db-credentials
defaultMode: 0400 # owner read-only
# Spring Boot reads file-based secrets:
# spring.datasource.password=${file:/etc/secrets/db/password}HashiCorp Vault — dynamic secrets and Vault Agent injection
Vault generates short-lived dynamic credentials on demand — the database secret engine creates a temporary MySQL user with a 1-hour TTL. The Vault Agent Injector injects secrets into pods as files via annotations, removing the need for application code to call Vault directly. Dynamic secrets dramatically reduce the blast radius of leaked credentials.
# Vault Agent Injector — annotate pod to auto-inject secrets
apiVersion: apps/v1
kind: Deployment
metadata:
name: order-service
spec:
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "order-service"
# Inject database credentials from Vault dynamic secret engine
vault.hashicorp.com/agent-inject-secret-db: "database/creds/order-service-role"
vault.hashicorp.com/agent-inject-template-db: |
{{- with secret "database/creds/order-service-role" -}}
spring.datasource.username={{ .Data.username }}
spring.datasource.password={{ .Data.password }}
{{- end }}
spec:
serviceAccountName: order-service-sa # Vault uses K8s SA for auth
# Vault policy — order-service can only read its own DB creds
# path "database/creds/order-service-role" {
# capabilities = ["read"]
# }
# Spring Boot reads injected file
# spring.config.import=optional:file:/vault/secrets/dbAWS Secrets Manager with Spring Cloud AWS
Spring Cloud AWS auto-loads secrets from AWS Secrets Manager into the Spring environment at startup. Secrets are stored as JSON strings; individual keys map to Spring properties. IAM roles for service accounts (IRSA) in EKS eliminates static AWS credentials entirely — the pod assumes a role with access to specific secrets.
<!-- pom.xml -->
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-starter-secrets-manager</artifactId>
</dependency>
# application.yml — load secret from Secrets Manager at startup
spring:
config:
import: "aws-secretsmanager:/prod/order-service/db"
# Secret value in AWS: {"username":"app_user","password":"abc123"}
# Maps to: spring.datasource.username=app_user, spring.datasource.password=abc123
# OR explicit property mapping
spring:
datasource:
password: "${/prod/order-service/db:password}"
# IAM policy — EKS IRSA (pod assumes this role, no static keys needed)
# {
# "Effect": "Allow",
# "Action": ["secretsmanager:GetSecretValue"],
# "Resource": "arn:aws:secretsmanager:eu-west-1:*:secret:/prod/order-service/*"
# }
# Never do this:
# spring.datasource.password=hardcoded-password ← DO NOT COMMIT
# AWS_SECRET_ACCESS_KEY=... ← DO NOT PUT IN IMAGEKey Points to Remember
- 1Never commit secrets to source control — use .gitignore, pre-commit hooks, and git-secrets scanning
- 2Kubernetes Secrets are base64-encoded (not encrypted) — enable etcd encryption at rest in production clusters
- 3Volume-mounted secrets are safer than environment variables — env vars are exposed in process listings and debug outputs
- 4Vault dynamic secrets generate short-lived credentials per request — leaked credentials expire automatically
- 5Vault Agent Injector injects secrets as files via pod annotations, removing Vault SDK dependency from applications
- 6Cloud IRSA (IAM Roles for Service Accounts) eliminates static AWS credentials entirely — pods assume roles dynamically
Interview Questions
Sign in to ask AriaWhy are Kubernetes Secrets not truly secure by default and how do you improve their security?
What are dynamic secrets in Vault and why are they more secure than static credentials?
Why is injecting secrets as mounted files safer than injecting them as environment variables?
How does IRSA eliminate the need for static AWS access keys in EKS pods?
How would you implement automatic secret rotation without restarting application pods?
Ask Aria about Secrets Management
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.