Exactly-once delivery ensures that records are neither lost nor delivered more than once, even when retries or failures occur. Kafka uses transactions to provide this guarantee. section 4.2.2
Choose the Kafka mechanism according to the scope of the guarantee:
- Use an idempotent producer to deliver records in order and only once to a topic partition. section 4.4
- Use a transactional producer to publish to multiple topics atomically: all messages succeed together or none do. section 4.4
- Configure consumers with isolation.level=read_committed so they consume only committed records from transactional topics. section 4.7
Retry and transaction failure
A producer may resend a batch when an acknowledgment does not arrive, although the broker already appended it. With transactions, an aborted transaction is ignored by read_committed consumers; the consumer position advances while aborted data remains invisible. section 4.2.2 section 4.2.2
Enable transactional producers with a unique transactional.id for each producer. Treat exactly-once delivery as distinct from at-least-once processing, where retries or restart after an uncommitted offset can reprocess records. section 4.7 section 4.3.5
