Data Management — Cheat Sheet
Microservices · 5 topics. Download the PDF or the Instagram carousel and share it.
Database per Service Pattern
Each microservice owns its data store (SQL, NoSQL, graph) chosen for its workload; inter-service data access happens only through APIs, never direct DB queries.
- ✓Each microservice owns its private data store — no other service can query it directly
- ✓Different services can use different DB technologies chosen for their workload (polyglot persistence)
- ✓Cross-service reads: call the API (synchronous) or maintain a local materialised view (async)
- ✓Referential integrity becomes application-level validation + compensating transactions
- ✓The Outbox pattern solves dual-write: persist event to DB in the same transaction as the state change
- ✓Shared database is the most common microservices anti-pattern — it tightly couples schemas and teams
# docker-compose.yml (dev) — each service has its own DB instance
services:
order-service:
image: order-service:latest
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://order-db:5432/orders
order-db:
image: postgres:16
volumes: [order-data:/var/lib/postgresql/data]
inventory-service:
image: inventory-service:latest
environment:
SPRING_DATASOURCE_URL: jdbc:mongodb://inventory-mongo:27017/inventory
inventory-mongo:
image: mongo:7
# Kubernetes NetworkPolicy — deny direct DB access from other services
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: order-db-policy }
spec:
podSelector: { matchLabels: { app: order-db } }
ingress:
- from: [{ podSelector: { matchLabels: { app: order-service } } }]
ports: [{ port: 5432 }]SAGA Pattern for Distributed Transactions
A SAGA splits a multi-service transaction into local transactions linked by events (choreography) or an orchestrator, with compensating transactions for rollback.
- ✓SAGA breaks a distributed transaction into local transactions; each step publishes an event (choreography) or responds to a command (orchestration).
- ✓Compensating transactions undo the effect of completed steps — they must be idempotent because messages can be delivered more than once.
- ✓Choreography = decentralised, services react to events; Orchestration = centralised, a coordinator drives the sequence.
- ✓SAGAs provide eventual consistency, not ACID — a failure window exists where partial state is visible across services.
- ✓Use an idempotency key (e.g., orderId + stepName) stored in a DB table to detect and skip duplicate event/command delivery.
- ✓Always design intermediate states (PAYMENT_PENDING, INVENTORY_RESERVED) explicitly in your domain model — they are real business states in a SAGA world.
// Order Service — starts the SAGA
@Service
public class OrderService {
public void placeOrder(OrderRequest req) {
Order order = orderRepo.save(new Order(req, OrderStatus.PENDING));
// Publish event → triggers Payment Service
eventBus.publish(new OrderCreatedEvent(order.getId(), req.getAmount(), req.getUserId()));
}
// Compensating transaction — called if payment or inventory fails
@EventListener
public void onPaymentFailed(PaymentFailedEvent event) {
orderRepo.updateStatus(event.getOrderId(), OrderStatus.CANCELLED);
eventBus.publish(new OrderCancelledEvent(event.getOrderId()));
}
}
// Payment Service — reacts to OrderCreatedEvent
@Service
public class PaymentService {
@EventListener
public void onOrderCreated(OrderCreatedEvent event) {
try {
Payment payment = paymentGateway.charge(event.getUserId(), event.getAmount());
paymentRepo.save(payment);
// Success → triggers Inventory Service
eventBus.publish(new PaymentCompletedEvent(event.getOrderId()));
} catch (PaymentException e) {
// Failure → triggers compensating transaction in Order Service
eventBus.publish(new PaymentFailedEvent(event.getOrderId()));
}
}
}Event Sourcing
State is derived by replaying a log of immutable domain events rather than storing the current snapshot; enables audit trails, temporal queries, and event-driven projections.
- ✓Event Sourcing stores facts (events), not state — current state is derived by replaying events
- ✓Append-only event store: OrderCreated → ItemAdded → PaymentReceived → OrderShipped
- ✓Snapshots bound replay cost for aggregates with long event histories
- ✓Read models (projections) are built separately by consuming the event stream — CQRS
- ✓Multiple independent projections can consume the same event stream for different read needs
- ✓Schema evolution: never modify stored events — add upcasters to transform old events to new versions
// Event base type
public sealed interface OrderEvent permits
OrderCreated, ItemAdded, PaymentReceived, OrderShipped {}
public record OrderCreated(String orderId, String customerId, Instant at) implements OrderEvent {}
public record ItemAdded(String orderId, String sku, int qty, BigDecimal price) implements OrderEvent {}
// Aggregate — rebuilt by replaying events
public class Order {
private String id;
private String status;
private List<OrderItem> items = new ArrayList<>();
public static Order rebuild(List<OrderEvent> events) {
Order order = new Order();
events.forEach(order::apply);
return order;
}
private void apply(OrderEvent event) {
switch (event) {
case OrderCreated e -> { this.id = e.orderId(); this.status = "CREATED"; }
case ItemAdded e -> items.add(new OrderItem(e.sku(), e.qty(), e.price()));
case PaymentReceived e -> this.status = "PAID";
case OrderShipped e -> this.status = "SHIPPED";
}
}
}
// Event store (append-only)
CREATE TABLE event_store (
stream_id VARCHAR(36),
seq BIGINT,
event_type VARCHAR(100),
payload JSON,
occurred_at TIMESTAMP,
PRIMARY KEY (stream_id, seq)
);CQRS Pattern
Command Query Responsibility Segregation separates the write model (commands mutate state) from the read model (queries hit optimised read stores), each scaling independently.
- ✓CQRS splits reads and writes into separate models: command side uses a normalised write store; query side uses a denormalised read store.
- ✓The two sides are kept in sync asynchronously via domain events — the system is eventually consistent.
- ✓A key benefit: the read store can use a completely different technology (Elasticsearch, Redis, Cassandra) optimised for query patterns.
- ✓CQRS and Event Sourcing are independent patterns — CQRS can be used with a simple SQL write store (not just event-sourced state).
- ✓The cost: eventual consistency, two codebases, and event infrastructure. Only adopt CQRS when you have genuinely different read/write needs.
- ✓Start without CQRS; introduce it when read/write requirements diverge beyond what indexes and views can solve.
// ── COMMAND SIDE ─────────────────────────────────────────────
// Command — represents an intent to change state
public record PlaceOrderCommand(String customerId, List<OrderItem> items) {}
// Command handler — executes business logic, publishes event
@Service
@RequiredArgsConstructor
public class OrderCommandHandler {
private final OrderRepository orderRepo; // write store
private final ApplicationEventPublisher events;
@Transactional
public String handle(PlaceOrderCommand cmd) {
Order order = Order.place(cmd.customerId(), cmd.items());
orderRepo.save(order);
events.publishEvent(new OrderPlacedEvent(order.getId(), order.getCustomerId(),
order.getTotal(), Instant.now()));
return order.getId();
}
}
// ── QUERY SIDE ────────────────────────────────────────────────
// Read model — denormalised projection for fast queries
@Document(indexName = "order-summaries") // Elasticsearch document
public class OrderSummaryDocument {
private String orderId;
private String customerName; // denormalised from Customer service
private String customerEmail;
private BigDecimal total;
private String status;
private Instant placedAt;
}
// Query handler — reads from the optimised read store
@Service
@RequiredArgsConstructor
public class OrderQueryHandler {
private final OrderSummaryRepository esRepo; // Elasticsearch repo
public Page<OrderSummaryDocument> findByCustomer(String customerId, Pageable page) {
return esRepo.findByCustomerId(customerId, page);
}
}Eventual Consistency
Distributed systems that reject strict ACID guarantees accept that replicas converge to the same value given enough time; design UIs and workflows around this reality.
- ✓Eventual consistency means replicas converge over time — there is always a window where different services see different data.
- ✓CAP theorem: AP systems (most microservices) trade strong consistency for availability and partition tolerance.
- ✓Return write results immediately from the command side; do not block on the query projection catching up.
- ✓Idempotent consumers must handle out-of-order and duplicate events gracefully — use version fields to detect stale events.
- ✓Design explicit intermediate states (PAYMENT_PENDING, CANCELLING) for Saga workflows — these states WILL be visible to users.
- ✓Optimistic locking (@Version in JPA) on the write side detects concurrent modifications before they corrupt data.
// Eventual consistency example: // // 1. User places order → Order Service writes to DB, publishes OrderPlaced event // Order Service DB: orderId=123, status=PLACED ✓ // // 2. User immediately queries "my orders" via Order Query Service // Query Service reads Elasticsearch projection — still processing the event // Query Result: orderId=123 NOT YET VISIBLE ← stale read // // 3. ~50ms later: Order Query Service processes OrderPlaced event // Elasticsearch: orderId=123, status=PLACED ✓ // User's next refresh: orderId=123 IS visible // // The window: ~50ms to a few seconds depending on consumer speed and load // Pattern 1: Optimistic UI — assume success, show immediately // After POST /orders, add the new order to the UI state locally // without waiting for the query endpoint to reflect it orderList.add(newOrder); // client-side state update — eventual consistency UX trick