Cheat SheetsApache KafkaSpring Integration

Spring Integration — Cheat Sheet

Apache Kafka · 3 topics. Download the PDF or the Instagram carousel and share it.

Cheat Sheet · AiCanCode.org
Spring Integration
Apache Kafka3 topicsQuick revision reference
1

Kafka with Spring Boot

spring-kafka auto-configures KafkaTemplate for production and a ConcurrentKafkaListenerContainerFactory for consumption; set bootstrap-servers in application.properties.

  • ✓spring-kafka auto-configures KafkaTemplate and a listener container factory from application.properties.
  • ✓Use ErrorHandlingDeserializer to prevent a malformed message from crashing the entire consumer thread.
  • ✓Set listener concurrency to match the number of partitions divided by expected instance count.
  • ✓MANUAL_IMMEDIATE ack mode commits the offset only after your handler confirms success — essential for at-least-once delivery.
  • ✓DefaultErrorHandler replaces the old SeekToCurrentErrorHandler (spring-kafka 2.8+); use DeadLetterPublishingRecoverer for DLT routing.
  • ✓Enable spring.kafka.producer.properties.enable.idempotence=true to prevent duplicate publishes on producer retries.
Properties + Java — Spring Boot Kafka producer
# application.properties — producer
spring.kafka.bootstrap-servers=kafka-1:9092,kafka-2:9092,kafka-3:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer

# Reliability settings
spring.kafka.producer.acks=all          # wait for all ISR replicas
spring.kafka.producer.retries=3
spring.kafka.producer.properties.enable.idempotence=true

# Throughput
spring.kafka.producer.batch-size=65536     # 64 KB batch
spring.kafka.producer.properties.linger.ms=5

# Sending via auto-configured KafkaTemplate
@Service
public class OrderProducer {
    @Autowired KafkaTemplate<String, OrderEvent> template;

    public void publish(OrderEvent event) {
        CompletableFuture<SendResult<String, OrderEvent>> future =
            template.send("order-events", event.getOrderId(), event);
        future.whenComplete((result, ex) -> {
            if (ex != null) {
                log.error("Send failed", ex);
            } else {
                log.info("Sent to partition {} offset {}",
                    result.getRecordMetadata().partition(),
                    result.getRecordMetadata().offset());
            }
        });
    }
}
2

@KafkaListener

@KafkaListener annotates a method to consume records from one or more topics; supports batch consumption, error handlers, and retry with @RetryableTopic for DLT routing.

  • ✓@KafkaListener manages the consumer lifecycle, poll loop, and deserialization — you write only business logic.
  • ✓concurrency = "N" starts N consumer threads per listener; effective up to the partition count of the topic.
  • ✓Batch mode (type: batch) delivers a List<ConsumerRecord<K,V>> per call — much more efficient for bulk DB writes.
  • ✓@RetryableTopic creates retry topics with exponential back-off and a DLT automatically — the modern replacement for manual retry loops.
  • ✓Use @DltHandler in the same class to handle records that exhausted all retries (alert, persist, replay later).
  • ✓Inject Acknowledgment + Channel for manual ack mode; inject @Header(KafkaHeaders.RECEIVED_PARTITION) / OFFSET for record metadata.
Java — @KafkaListener Variants
// 1. Single-record listener — one ConsumerRecord per call
@Component
public class OrderConsumer {

    // topics: list of topic names
    // groupId: overrides spring.kafka.consumer.group-id for this listener
    // concurrency: starts N concurrent consumer threads (up to partition count)
    @KafkaListener(
        topics      = {"orders", "order-corrections"},
        groupId     = "order-processor",
        concurrency = "3"
    )
    public void onOrder(
            ConsumerRecord<String, Order> record,  // full record with headers
            @Header(KafkaHeaders.RECEIVED_PARTITION) int partition,
            @Header(KafkaHeaders.OFFSET) long offset) {

        log.info("Processing order {} from partition {} offset {}",
                 record.key(), partition, offset);
        orderService.process(record.value());
    }

    // 2. Simplified — just the value (Spring resolves via type)
    @KafkaListener(topics = "payments", groupId = "payment-service")
    public void onPayment(Payment payment) {
        paymentService.handle(payment);
    }

    // 3. Regex subscription — all topics matching the pattern
    @KafkaListener(topicPattern = "order.*", groupId = "analytics")
    public void onAnyOrderEvent(ConsumerRecord<String, String> record) {
        analyticsService.ingest(record);
    }
}
3

Dead Letter Topic (DLT)

Failed records after exhausting retries are forwarded to a DLT (topic-name.DLT); a separate listener monitors the DLT for alerting, inspection, or replaying.

  • ✓A DLT receives records that could not be processed after all retries are exhausted — application-managed in Kafka (unlike broker-managed in RabbitMQ).
  • ✓@RetryableTopic creates retry topics and a DLT automatically; @DltHandler handles DLT records in the same class.
  • ✓DeadLetterPublishingRecoverer with DefaultErrorHandler provides manual DLT routing with exception-specific configuration.
  • ✓addNotRetryableExceptions() routes records immediately to the DLT for permanent errors (schema validation, invalid data) without wasting retries.
  • ✓Monitor DLT depth as an SLA metric — alert on any DLT arrival; a growing DLT indicates a systemic processing failure.
  • ✓Replay DLT records after fixing the root cause by re-publishing to the original topic — keep the DLT consumer idempotent.
Java — @RetryableTopic + @DltHandler
@Component
@RequiredArgsConstructor
public class PaymentConsumer {

    private final PaymentService paymentService;
    private final AlertService alertService;

    // Spring Kafka creates automatically:
    //   payments-retry-0  (1s delay)
    //   payments-retry-1  (2s delay)
    //   payments-retry-2  (4s delay)
    //   payments.DLT      (after 3 retries exhausted)
    @RetryableTopic(
        attempts = "4",    // 1 original + 3 retries
        backoff  = @Backoff(delay = 1000, multiplier = 2.0),
        dltTopicSuffix = ".DLT",
        autoCreateTopics = "true",
        kafkaTemplate = "kafkaTemplate"
    )
    @KafkaListener(topics = "payments", groupId = "payment-service")
    public void onPayment(ConsumerRecord<String, Payment> record) {
        paymentService.process(record.value());
        // Throws RuntimeException → forwarded to retry topic
        // After 3 retries → forwarded to payments.DLT
    }

    // DLT handler — called when record lands in payments.DLT
    @DltHandler
    public void handleDlt(
            ConsumerRecord<String, Payment> record,
            @Header(KafkaHeaders.RECEIVED_TOPIC)  String topic,
            @Header(KafkaHeaders.EXCEPTION_MESSAGE) String errorMsg) {

        log.error("Payment permanently failed after retries. topic={} key={} error={}",
                  topic, record.key(), errorMsg);

        // Options:
        // 1. Alert the on-call team
        alertService.critical("Payment DLT: " + record.key(), errorMsg);
        // 2. Persist to a failed_payments table for manual inspection
        // 3. Emit a compensating event
    }
}
Learn this free with Aria, your AI tutor → AiCanCode.org/learn/kafka