A tombstone is a Kafka record with a non-null key and a null value. It marks the keyed record for deletion during log compaction. For example, when an account no longer exists, AccountService publishes the account number as the key and null as the value. section 2.5.1
Tombstones matter because deletion in a compacted topic has timing and delivery consequences:
- Compacted topics require keys, and key-based partitioning keeps records for the same key in one partition. section 2.5.1
- Compaction retains the latest value for each key, while a null-valued record is eventually removed as a deletion marker. section 2.5.2
- Kafka retains tombstones uncompacted for the configured delete-retention period, allowing disconnected consumers time to observe deletions. section 2.5.2
- Compaction is periodic rather than immediate, so consumers cannot treat a tombstone as an instant physical removal from the topic. section 2.5.4
Deleting an account and synchronizing a database
Suppose a compacted Balances topic contains a record for ACC-1447. AccountService publishes <ACC-1447, null> when the account is deleted. A connected consumer can process that tombstone; a JDBC sink connector interprets it as a delete for the matching database key. If a consumer was disconnected, tombstone retention helps it learn about the deletion after reconnecting. section 2.5.2 section 7.3.8
Figure 2.16 During the compaction process, the data within a topic is rewritten. For each key, only the latest value is retained, and messages where the value is null are deleted.
Use the compaction diagram to trace how the latest value survives for each key and how null-valued records represent deletions before being removed. section 2.5.2
