Statefulness lets an event-processing application recall earlier events and connect them to the current one. A stateless operation can decide from one event alone; a stateful operation retrieves, updates, and stores information accumulated across related events. section 7.0 section 7.1
In Kafka Streams, state is needed whenever processing must track progress or information over time. Stateful operations such as reduce, aggregate, and join use records’ keys to associate events with the state they update. State stores are created automatically and are persistent by default. section 7.2 section 7.5
Accumulate each player's score
A poker-score stream can use player ID as its key and each round’s score as its value. Grouping by key keeps players separate; reduce then adds every incoming score to that player’s running total. Unlike a single-round score, the total depends on scores already processed. section 7.2
Figure 7.1 A reduce takes several inputs and merges them into a single result of the same type.
Use the reduce diagram to see how several values become one running result of the same type. section 7.2
A practical stateful topology needs a few deliberate choices:
- Ensure records have valid keys, or select a new key when grouping or joining by a different attribute; Kafka Streams can repartition data when needed. section 7.5
- Name state stores and repartition topics explicitly. Adding or removing an upstream operator can change generated names, causing an application to start with an empty new store and changelog. section 7.4.5 section 7.5
- Choose an in-memory store only when appropriate; persistent state stores are the default. section 7.5
Test stateful topologies with TopologyTestDriver by piping input records, asserting output records, and reading the named state store to verify its latest contents. For stream-time behavior, supply timestamps or advance input-topic time so time-driven processing can run. section 14.1.4
