Bulkhead Pattern
IntermediateIsolates failure by limiting concurrent calls to a downstream dependency (semaphore or thread-pool bulkhead), preventing one slow service from exhausting shared resources.
Overview
The Bulkhead pattern (named after ship watertight compartments) prevents one slow or failing downstream dependency from exhausting all available threads or connections, which would cause a cascade failure that takes down unrelated features. Two Resilience4j implementations exist: semaphore bulkhead (limits concurrent in-flight calls) and thread-pool bulkhead (allocates a dedicated thread pool per dependency — true isolation, but more resource-intensive). Without bulkheads, a slow payment service can saturate the shared Tomcat thread pool (200 threads), making order lookups, user profiles, and unrelated endpoints all timeout — even though the payment service issue is isolated.
Semaphore bulkhead with Resilience4j
Semaphore bulkhead limits concurrent calls by tracking in-flight requests. Cheaper than thread-pool bulkhead; falls through to fallback when the semaphore is full.
# application.properties — semaphore bulkhead
resilience4j.bulkhead.instances.paymentService.max-concurrent-calls=10
resilience4j.bulkhead.instances.paymentService.max-wait-duration=0ms
# max-wait-duration=0: reject immediately when full (fail-fast)
@Service
public class CheckoutService {
@Bulkhead(name = "paymentService",
type = Bulkhead.Type.SEMAPHORE,
fallbackMethod = "paymentFallback")
public PaymentResult processPayment(PaymentRequest req) {
return paymentClient.charge(req); // max 10 concurrent calls
}
// Called when bulkhead is full (BulkheadFullException)
private PaymentResult paymentFallback(PaymentRequest req,
BulkheadFullException ex) {
log.warn("Payment bulkhead full — circuit protecting main thread pool");
return PaymentResult.rejected("Payment service temporarily unavailable");
}
}
// Monitor: Resilience4j metrics exposed via Micrometer
// resilience4j.bulkhead.available.concurrent.calls{name="paymentService"}
// Alert when available calls drops to 0 consistently → bulkhead too smallThread-pool bulkhead for true isolation
Thread-pool bulkhead runs calls in a dedicated thread pool, providing true isolation at the cost of context switching. Use for blocking clients (RestTemplate, JDBC) in reactive apps.
# application.properties — thread-pool bulkhead
resilience4j.thread-pool-bulkhead.instances.inventoryService.core-thread-pool-size=5
resilience4j.thread-pool-bulkhead.instances.inventoryService.max-thread-pool-size=10
resilience4j.thread-pool-bulkhead.instances.inventoryService.queue-capacity=20
resilience4j.thread-pool-bulkhead.instances.inventoryService.keep-alive-duration=20ms
@Service
public class ProductService {
// Runs on dedicated inventoryService thread pool (not Tomcat pool)
@Bulkhead(name = "inventoryService",
type = Bulkhead.Type.THREADPOOL,
fallbackMethod = "inventoryFallback")
public CompletableFuture<InventoryResponse> checkInventory(String sku) {
return CompletableFuture.supplyAsync(
() -> inventoryClient.checkStock(sku)
);
// Returns CompletableFuture because thread-pool bulkhead is async
}
private CompletableFuture<InventoryResponse> inventoryFallback(
String sku, BulkheadFullException ex) {
return CompletableFuture.completedFuture(
InventoryResponse.unknown(sku));
}
}Combining bulkhead with circuit breaker and timeout
Production resilience uses all three patterns together: bulkhead limits concurrency, timeout prevents indefinite waiting, circuit breaker stops calls to failed services.
@Service
public class RecommendationService {
// Layer order matters: Bulkhead → CircuitBreaker → Retry → TimeLimiter
// Innermost annotation executes first
@TimeLimiter(name = "recommendations") // outermost: fail fast on timeout
@CircuitBreaker(name = "recommendations", fallbackMethod = "defaultRecs")
@Bulkhead(name = "recommendations") // innermost: limit concurrency
public CompletableFuture<List<Product>> getRecommendations(Long userId) {
return CompletableFuture.supplyAsync(
() -> mlRecommenderClient.getForUser(userId)
);
}
private CompletableFuture<List<Product>> defaultRecs(Long userId, Exception ex) {
// Graceful degradation: return cached bestsellers
return CompletableFuture.completedFuture(popularProductsCache.getTop10());
}
}
# Configs
resilience4j.timelimiter.instances.recommendations.timeout-duration=1s
resilience4j.bulkhead.instances.recommendations.max-concurrent-calls=20
resilience4j.circuitbreaker.instances.recommendations.failure-rate-threshold=50
resilience4j.circuitbreaker.instances.recommendations.sliding-window-size=10Key Points to Remember
- 1Bulkhead prevents thread pool exhaustion: one slow service cannot starve all other endpoints.
- 2Semaphore bulkhead is lightweight (counts permits); thread-pool bulkhead provides true pool isolation.
- 3Thread-pool bulkhead requires CompletableFuture return type — it offloads the call to its own pool.
- 4Combine: Bulkhead (concurrency limit) + TimeLimiter (timeout) + CircuitBreaker (failure threshold).
- 5Set max-wait-duration=0ms to fail fast when the bulkhead is full — blocking is worse than fast failure.
- 6Size bulkheads based on expected concurrency and downstream SLAs — too small = false positives; too large = no protection.
Interview Questions
Sign in to ask AriaWhat problem does the Bulkhead pattern solve and why is it named after a ship's compartment?
What is the difference between a semaphore bulkhead and a thread-pool bulkhead?
How would you configure a bulkhead in Spring Boot to protect your Tomcat thread pool from a slow downstream service?
In what order should you stack @Bulkhead, @CircuitBreaker, @Retry, and @TimeLimiter and why?
How would you monitor and alert on bulkhead saturation in production?
Ask Aria about Bulkhead Pattern
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.