A retry is a subsequent attempt to complete a producer operation after a transient failure or missing acknowledgment. Kafka resends an unacknowledged batch according to the producer’s retries setting. section 3.2.5
Retries are bounded by timeout settings rather than necessarily by a small retry count. delivery.timeout.ms limits the complete send cycle, including retries and acknowledgment waits; request.timeout.ms limits each individual request. section 3.2.5 section 3.2.5
Design retries with their operational consequences in mind:
- Treat lost acknowledgments as a duplicate-risk scenario: a broker may have written the batch before the producer retries it. section 3.2.5
- Protect partition order when retries overlap. A later batch can be acknowledged before an earlier batch is resent, causing out-of-order storage. section 3.2.5 section 7.4.1
- Use producer idempotence with positive retries and acks=all; recent Kafka versions also recommend no more than five in-flight requests per connection. section 7.4.1
Lost acknowledgment
A producer sends batch 1, receives no acknowledgment, and sends batch 2. If batch 2 succeeds first and batch 1 is later retried, partition order can change. Idempotence lets the broker use producer identifiers and per-partition sequence numbers to reject out-of-sequence writes for retry. section 7.4.1
