Compacted topic
A compacted topic is a Kafka topic whose log compaction retains the latest record for each key while allowing obsolete records to be removed.
Kafka for Architects
A compacted topic is a Kafka topic configured to retain at least the latest record associated with each non-null key.
- How it is used
- Stores recoverable current state for keyed configuration, profiles, changelogs, balances, or prices.
- How deep it goes
- Explains retention semantics, key partitioning, sequential reads, delayed cleanup, and consumer-visible older records.
- Examples
- Uses account balances, customer profiles, configuration data, changelogs, and current product prices.
- What you can do after
- Operate/configure a topic for current keyed state while evaluating sequential replay, delayed cleanup, and storage trade-offs.
Kafka Streams in Action, Second Edition
A compacted topic is a Kafka topic whose log retains the latest record for each key rather than retaining every historical record indefinitely.
- How it is used
- Supports retaining recoverable latest state for keyed event streams and Kafka Streams state-store changelogs.
- How deep it goes
- Explains key-based cleanup, background log cleaning, configuration, tombstones, and combined policies, while deferring broader compaction details.
- Examples
- Uses stock ticker prices, duplicate keyed updates, tombstones, and a Kafka Streams changelog configuration.
- What you can do after
- Configure Kafka topics with compaction and choose tombstones or combined cleanup policies for keyed state retention.
Read how Kafka Streams in Action, Second Edition explains compacted topic →
Related concepts
- changelogincludes · A changelog topic is a Kafka topic that records state-store updates so processing state can be recovered or restored.
- partitioning keyuses · A partitioning key is a value or attribute used to determine which partition stores or receives a record or data item.
- topickind of · A topic is a named messaging destination or stream to which producers publish messages or records and from which consumers receive or read them.
- Apache Kafkakind of · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.
- database tableused by · A database table is a relational database structure that organizes records into rows and defined columns.
- migrationused by · A migration is a process of transferring or transforming data, software, or system state from a source form or environment to a target form or environment.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.