Exactly-Once Semantics (EOS)
AdvancedExactly-once semantics guarantees each message is delivered and processed precisely once, even during failures. It requires idempotent producers, transactional APIs, and read_committed consumers.
Overview
EOS combines three mechanisms: (1) Idempotent producer prevents duplicate messages from retries; (2) Transactional producer wraps operations across topics/partitions atomically; (3) read_committed consumer sees only committed transaction data.
Transactional Producer in Spring Boot
Set enable.idempotence=true, configure transactional.id, and wrap sends in executeInTransaction.
@Configuration
public class TransactionalProducerConfig {
@Bean
public ProducerFactory<String, Order> producerFactory() {
Map<String, Object> config = new HashMap<>();
config.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
config.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "order-producer-1");
config.put(ProducerConfig.ACKS_CONFIG, "all");
return new DefaultKafkaProducerFactory<>(config);
}
}
// Atomic publish to multiple topics
kafkaTemplate.executeInTransaction(ops -> {
ops.send("orders", order.getId(), order);
ops.send("order-events", order.getId(), new OrderCreatedEvent(order));
return true; // throw to abort
});
// Consumer must use read_committed
spring.kafka.consumer.properties.isolation.level=read_committedKey Points to Remember
- 1EOS = idempotent producer + transactional writes + read_committed consumer
- 2enable.idempotence=true prevents producer-retry duplicates using sequence numbers
- 3transactional.id must be unique per producer instance across restarts
- 4read_committed consumers skip messages from aborted transactions
- 5EOS has ~20% throughput overhead — use only when truly required
Interview Questions
Sign in to ask AriaWhat are the three components required for exactly-once semantics in Kafka?
What does an idempotent producer guarantee and how does it work?
What is isolation.level=read_committed and why is it needed for EOS?
What is the performance cost of enabling exactly-once semantics in Kafka?
How does Kafka's transactional API atomically write to multiple partitions?
Ask Aria about Exactly-Once Semantics (EOS)
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.