Materialized configures the state store behind a Kafka Streams stateful operation. Use it with aggregations such as reduce and count to supply key and value Serdes, give the store a stable name, select a store implementation, and tune logging or caching behavior. section 7.2.1 section 7.4.7
State stores hold key-value pairs as byte arrays, so Serdes serialize records into the store and deserialize retrieved values. Stateful aggregation results are KTables; convert one with toStream when forwarding results to downstream stream operators or an output topic. section 7.2.1
Use Materialized to make stateful processing explicit and maintainable:
- Name each store uniquely with Materialized.as("store-name"). Stable names prevent topology edits from changing the on-disk directory and changelog-topic names; duplicate names cause a TopologyException. section 7.4.5
- Provide Serdes with Materialized.with or withKeySerde and withValueSerde so store keys and values can be serialized. section 7.2.1 section 7.4.6
- Pass a StoreSupplier, such as Stores.inMemoryKeyValueStore, to Materialized.as to use an in-memory store. It can improve access speed but has bounded memory and restores from its changelog after restart. section 7.4.6
- Configure a store-specific changelog cleanup policy with withLoggingEnabled, or disable caching for one stateful operation with withCachingDisabled. Disabling logging can leave state unable to restore. section 7.4.7
A named in-memory score total
Group poker-game scores by key, sum each player’s values, and materialize the result in a named in-memory store. The Serdes remain necessary even though the store is not persistent: pokerScoreStream.groupByKey().reduce(Double::sum, Materialized.<String, Double>as(Stores.inMemoryKeyValueStore("memory-poker-score-store")).withKeySerde(Serdes.String()).withValueSerde(Serdes.Double())).toStream().to("total-scores", Produced.with(Serdes.String(), Serdes.Double())); section 7.4.6
A supplied store name also enables Interactive Queries. Configure application.server with a unique host and port per application instance, then query the named local store that owns the relevant partition; state is distributed across instances as tasks are distributed. section 13.2 section 13.3
Listing 13.1 Naming state stores to enable them for querying
Materialized.<String, LoanAppRollup>as(
Stores.inMemoryKeyValueStore(loanAppStoreName)) #1
.withKeySerde(stringSerde)
.withValueSerde(loanAppRollupSerde))
Materialized.<String, LoanAppRollup>as(
loanAppStoreName) #2
.withKeySerde(stringSerde)
.withValueSerde(loanAppRollupSerde))
Materialized.with(stringSerde, loanAppRollupSerde) #3
Compare a named in-memory store, a named default persistent store, and an unnamed Serde-only configuration. Only the named alternatives are eligible for Interactive Queries. section 13.2
