A sequence number is an ordered identifier attached by an idempotent Kafka producer to each message. It helps the partition leader detect duplicates, missing messages, and messages arriving out of order. This matters when a producer does not receive an acknowledgment: retrying may otherwise write the same message twice. section 5.1.3
Sequence numbers work with a producer ID and idempotence. When a previously persisted message is retried, the broker resends the acknowledgment instead of writing it again. When a message arrives out of sequence, the broker sends a negative acknowledgment, prompting the producer to retry. section 5.1.3
Use sequence numbers with these operational boundaries:
- Enable idempotence with `enable.idempotence=true`; use `acks=all`, retries greater than zero, and no more than five in-flight requests per connection. section 5.1.3
- Treat ordering as a partition-scoped guarantee: messages must come from the same producer and reach the same partition. section 6.1.1
- Do not infer ordering across partitions or among messages produced by different producers. section 6.1.1
Enable sequence-based duplicate detection
A console producer can enable idempotence alongside durable acknowledgments: `kafka-console-producer.sh --topic products.prices.changelog.min-isr-2 --bootstrap-server localhost:9092 --producer-property acks=all --producer-property enable.idempotence=true`. If an acknowledgment is lost after persistence, the retry is recognized rather than appended as a duplicate.
