Kafka Streams State Stores
IntermediateState stores are local RocksDB databases embedded in a Streams application enabling stateful operations like aggregations, joins, and windowed computations.
Overview
Kafka Streams keeps state locally (RocksDB) and backs it up to a changelog topic. When a Streams task is reassigned, it restores state from the changelog. State stores are the foundation of count, aggregate, reduce, join, and windowed queries.
Aggregation with State Stores + Interactive Queries
Kafka Streams DSL creates state stores automatically. You can read them from REST APIs via interactive queries.
StreamsBuilder builder = new StreamsBuilder();
KStream<String, OrderEvent> orders = builder.stream("order-events");
KTable<String, CustomerStats> stats = orders
.groupByKey()
.aggregate(
CustomerStats::new,
(customerId, event, agg) -> { agg.add(event); return agg; },
Materialized.<String, CustomerStats, KeyValueStore<Bytes, byte[]>>
as("customer-stats-store")
.withValueSerde(new CustomerStatsSerde())
);
// Read state store from REST endpoint
@GetMapping("/stats/{customerId}")
public CustomerStats getStats(@PathVariable String customerId) {
ReadOnlyKeyValueStore<String, CustomerStats> store =
streams.store(StoreQueryParameters.fromNameAndType(
"customer-stats-store", QueryableStoreTypes.keyValueStore()));
return store.get(customerId);
}Key Points to Remember
- 1State stores are local RocksDB instances backed by Kafka changelog topics
- 2Fault tolerance: state restored from changelog on task reassignment
- 3Interactive queries allow external APIs to read state in real time
- 4In-memory stores are faster but lose state on restart
- 5num.standby.replicas pre-warms state on standby tasks for fast recovery
Interview Questions
Sign in to ask AriaWhat is a state store in Kafka Streams and how is it made fault-tolerant?
What is a changelog topic in Kafka Streams and when is it created?
How do interactive queries work and what are their limitations?
What is the difference between an in-memory store and a RocksDB store?
How do standby replicas reduce state restoration time during failover?
Ask Aria about Kafka Streams State Stores
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.