A changelog is an append-oriented stream of keyed updates whose useful view retains the latest value for each key. Unlike an event log, which preserves every record, a changelog represents current state through successive updates. Log compaction removes older records for duplicate keys while retaining the newest value. section 8.1.1 section 7.4.7
Kafka Streams uses internal changelog topics to make state stores durable. When a stateful operation writes a key-value record to its store, Kafka Streams also writes it to the backing topic. In-memory stores rebuild from the changelog after restart; persistent stores normally restore from a checkpoint offset, but rebuild from the topic when local state is lost or corrupt. section 7.4 section 7.4.1
Changelog behavior affects several operational choices:
- Caching suppresses intermediate aggregation updates: records reach the changelog when the cache flushes at commit time or when full. Disable caching globally or per materialized store when every update must be observed. section 7.2.1 section 7.4
- Use stable state-store names with Materialized.as(...) to keep the associated changelog name stable when topology operators change. section 7.4.5
- Keep logging enabled for restorable stores. A null value deletes a key permanently; for large key spaces, combine compact and delete cleanup policies when compaction alone does not constrain growth. section 7.4.7
Customer purchase updates
Key purchase activity by customer ID to turn independent purchase events into an update stream. Multiple records for the same customer append over time, but the changelog view keeps that customer’s latest record; a KTable can model this update stream. section 8.1.1
Figure 8.4 In a changelog, each incoming record overwrites the previous one with the same key. You’d have four events with a record stream, but for an update stream, you have only two.
Compare an event stream with a keyed update stream: repeated keys collapse to their latest values in the changelog view. section 8.1.1
