A tombstone is a Kafka record with a key and a null value. In a compacted topic, it marks the key for deletion: compaction removes earlier records with that key, then Kafka eventually removes the tombstone itself. section 2.6.2
Tombstones affect Kafka Streams and compacted logs in closely related ways:
- Use a null value to delete a record from a KTable; the deletion ultimately removes the record from its state store and backing changelog topic. section 8.2
- Forward incoming null-valued records as tombstone markers without evaluating them; filter and filterNot also produce tombstones for records that fail their predicates. section 8.3
- Choose compaction for updates and deletions, while independent standalone events use log deletion instead. section 2.6.2
Deleting a KTable record
A KTable receives a key-value pair whose value is null. Kafka Streams interprets that pair as a deletion, emits a tombstone, and ultimately removes the key from the table, state store, and backing changelog topic. section 8.2
Figure 2.7 On the left is a log before compaction. You’ll notice duplicate keys with different values. These duplicates are updates. On the right is after compaction, which retains the latest value for each key, but it’s smaller in size.
Compare a log before and after compaction: duplicate keys represent updates, while the compacted log retains the latest value for each key and occupies less space. section 2.6.2
