At-most-once delivery prioritizes avoiding duplicate processing: records are considered delivered before processing completes, so an error can cause records to be lost. section 4.2.2
Apply the guarantee at the producer or consumer boundary:
- Set the producer to acks=0; it sends records without waiting to learn whether the broker received or appended them. section 4.2.2
- Commit consumer offsets immediately after retrieving a batch and before processing it; a later failure will not trigger reprocessing. section 4.3.5
- Treat the commit as the point of no return: processing failures afterward can skip records when the consumer restarts. section 4.3.5
Batch processing trade-off
A consumer polls a batch, commits its offsets, and then begins processing. If the application stops after the commit, Kafka resumes beyond that batch: duplicates are avoided, but unprocessed records are lost. section 4.3.5
Figure 4.20 Asynchronous processing with auto-committing can lead to potentially lost records.
Use the diagram to trace how asynchronous processing can leave records unfinished when a commit advances the consumer position. section 4.3.5
