liveBook · Manning

Dead letter queue

As explained in Grokking Software Architecture

A dead letter queue is a specialized holding queue for messages that remain unprocessable after a defined number of delivery attempts.

A dead letter queue (DLQ) is a holding area for messages that still fail after a configured number of delivery attempts. Moving them out of the main queue preserves them for inspection while letting other messages continue. section 7.4.4

The message that keeps respawning

A consumer crashes on the same message, the broker redelivers it, and the cycle repeats. After a finite delivery limit, the broker moves that message to the DLQ instead of letting it clog the main queue. Fix the cause, then requeue the message. section 7.4.4

Keep the safety valve useful:

  • Set a finite delivery limit; unlimited retries can waste compute and keep a failing message in circulation. section 7.4.4
  • Monitor the DLQ. A growing pile can signal a code bug, configuration problem, or permissions issue. section 7.4.4
  • Distinguish temporary failures from permanent ones: transient glitches may merit retries, while invalid requests or poison pills can go directly to the DLQ. section 10.1
"RetryPolicy": {
  "MaxDeliveryCount": 5,        // Try 5 times
  "BackoffStrategy": "Exponential", // Wait longer each time (1s, 2s, 4s, 8s)
  "DeadLetteringOnExpiration": true // If 5 fails, move to DLQ
}

A sample retry-policy configuration shows a delivery limit, exponential backoff, and dead-lettering on expiration. section 7.4.4

How Grokking Software Architecture covers it

How deep it goes
Defines retry-limit routing and monitoring, while placing retry configuration in infrastructure settings.
Examples
Uses a crashing consumer and a retry-policy configuration with a delivery limit and exponential backoff.
What you can do after
Operate a DLQ by setting a finite delivery limit, monitoring failures, and requeueing messages after fixing their cause.
How it connects
Connects dead letter queues with message brokers, consumers, retries, acknowledgments, and poison pills.

Also explained in

Compare how 6 books explain dead letter queue →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Software Architecture. The section links point to the parts of the book it draws on.