Home/Learn/Microservices/Retry Pattern

Retry Pattern

Intermediate
Resilience

Transient failures are retried with exponential back-off and jitter; combine with idempotency on the server side to avoid duplicate effects.

Overview

The Retry pattern transparently re-attempts a failed operation on transient errors (timeouts, 503 Service Unavailable, connection resets) without surfacing the failure to the caller. Pure fixed-interval retry is dangerous under load — all clients retry in sync, creating thundering-herd spikes. Exponential backoff grows the wait time with each attempt (0.5s, 1s, 2s, 4s); adding random jitter desynchronises concurrent retries. In Resilience4j, @Retry combines with @CircuitBreaker — the circuit opens after consecutive retries fail, preventing retry storms. Crucially, retries must only be applied to idempotent operations: retrying a POST /payments without an idempotency key will double-charge the customer.

Resilience4j @Retry with exponential backoff + jitter

Configure retry in application.properties and annotate the service method. Resilience4j supports exponential backoff with random jitter out of the box.

Properties + Java — Resilience4j retry with exponential jitter
# application.properties
resilience4j.retry.instances.inventoryService.max-attempts=3
resilience4j.retry.instances.inventoryService.wait-duration=500ms
resilience4j.retry.instances.inventoryService.enable-exponential-backoff=true
resilience4j.retry.instances.inventoryService.exponential-backoff-multiplier=2
resilience4j.retry.instances.inventoryService.randomized-wait-factor=0.5
# wait times: ~500ms, ~1s, ~2s (randomised by ±50%)
resilience4j.retry.instances.inventoryService.retry-exceptions=  java.net.SocketTimeoutException,  org.springframework.web.client.HttpServerErrorException$ServiceUnavailable

@Service
public class OrderService {

    @Retry(name = "inventoryService", fallbackMethod = "inventoryFallback")
    @CircuitBreaker(name = "inventoryService")  // combine with CB
    public InventoryResponse checkInventory(String sku) {
        return inventoryClient.checkStock(sku);  // retried on transient errors
    }

    // Fallback only called after all retries are exhausted
    private InventoryResponse inventoryFallback(String sku, Exception ex) {
        log.warn("Inventory check failed after retries for {}: {}", sku, ex.getMessage());
        return InventoryResponse.unavailable(sku);
    }
}

Spring WebClient reactive retry

WebClient (reactive) uses Reactor's retryWhen for non-blocking retry with backoff. Filter on retryable status codes to avoid retrying client errors.

Java — WebClient reactive retry with backoff + jitter
@Service
public class PaymentClient {

    private final WebClient webClient;

    public Mono<PaymentResponse> charge(PaymentRequest req) {
        return webClient.post()
            .uri("/payments")
            .header("Idempotency-Key", req.getIdempotencyKey())  // required!
            .bodyValue(req)
            .retrieve()
            .onStatus(status -> status.is5xxServerError(),
                resp -> Mono.error(new RetryableException(resp.statusCode())))
            .bodyToMono(PaymentResponse.class)
            .retryWhen(
                Retry.backoff(3, Duration.ofMillis(500))
                    .maxBackoff(Duration.ofSeconds(4))
                    .jitter(0.5)
                    .filter(ex -> ex instanceof RetryableException)
                    .onRetryExhaustedThrow((spec, signal) ->
                        new PaymentServiceException("Payment service unavailable"))
            )
            .timeout(Duration.ofSeconds(10));
    }
}

When NOT to retry and retry budgets

Retrying the wrong errors amplifies load. Never retry 4xx client errors. Use retry budgets to prevent retry storms from overwhelming degraded services.

Java — retry budgets and which errors to retry
// NEVER retry these — retrying won't help:
// 400 Bad Request    → your request is malformed
// 401 Unauthorized   → your credentials are wrong
// 403 Forbidden      → you don't have permission
// 404 Not Found      → resource doesn't exist
// 409 Conflict       → business logic conflict (e.g. duplicate order)

// SAFE to retry:
// 429 Too Many Requests   → with Retry-After header backoff
// 500 Internal Server Error → only if idempotent
// 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout

// Retry budget: limit total retry attempts across all concurrent requests
// Prevents: 10 users × 3 retries = 30 simultaneous requests when service is already struggling

resilience4j.retry.instances.paymentService.max-attempts=2  // be conservative
# + CircuitBreaker opens after N failures, stopping all retries

// Retry storm prevention: when service is degraded, more retries = more damage
// Solution: circuit breaker + retry together
// 1st retry: immediate (maybe transient glitch)
// 2nd retry: after backoff+jitter
// Circuit opens: no more retries until service recovers

Key Points to Remember

  • 1Only retry idempotent operations — retrying POST /payments without an idempotency key causes double-charging.
  • 2Exponential backoff + jitter prevents thundering-herd: all clients retrying at exactly the same interval.
  • 3Never retry 4xx client errors — they indicate a problem with your request, not a transient server issue.
  • 4Combine @Retry with @CircuitBreaker: the circuit opens after exhausted retries, preventing retry storms.
  • 5Set max-attempts conservatively (2–3) — aggressive retries amplify load on already-degraded services.
  • 6Monitor retry rate as a metric — a sudden spike indicates upstream instability, not normal operation.

Interview Questions

Sign in to ask Aria
1

Why must you combine retries with idempotency keys for POST requests?

MediumStripe
2

What is exponential backoff with jitter and why is jitter necessary?

MediumAmazon
3

Which HTTP status codes should you retry and which should you not?

EasyNetflix
4

How does combining @Retry with @CircuitBreaker prevent retry storms?

HardUber
5

What is a retry budget and how does it protect a degraded upstream service?

HardGoogle

Ask Aria about Retry Pattern

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Sign in to save personal notes for this topic.

Discussion

Sign in to join the discussion.

Loading discussion…