A dead-letter topic (DLT) holds messages a consumer cannot process so other messages can continue through the pipeline. The redirected messages still need inspection and handling; routing them elsewhere is not a fix for the underlying failure. section 11.2.2
The failure and the component involved determine whether this option is available:
- A consumer may encounter a message with missing required fields or an incompatible format. Define a policy for such messages, and test that the consumer handles them. section 6.5.1
- A Kafka Connect sink connector can route a non-retriable message to a DLT and keep processing, rather than fail and stop. section 7.3.6
- A Kafka Connect source connector does not support DLTs. A schema change that makes its output incompatible with Kafka’s expected schema can instead stop the connector. section 7.3.10
A database rejects one record
Suppose a sink connector reads from Kafka and tries to insert a record that violates a database integrity constraint. Repeating the insertion will not correct the data. Routing that record to a DLT lets the connector continue with other records, while the rejected record awaits investigation. section 7.3.6
Decide in advance what happens after repeated failures: whether to use a DLT, when to alert or escalate, and how to inspect and reprocess its messages. Alert as soon as a message reaches the DLT. Kafka provides no built-in DLT management process, so teams must establish their own handling; an unmonitored DLT can leave failed events forgotten. section 10.1.4 section 7.3.10 section 11.2.2
