An order-processing service might turn one order into shipment, billing, and inventory events. A retry after failure must not create a second set, while a skipped order would leave downstream systems incomplete. Exactly-once processing addresses this risk for Kafka read-process-write pipelines. section 7.4.2
The guarantee has a boundary. Kafka does not provide exactly-once delivery by default: retries can produce duplicates, so consumers must be designed with idempotence in mind. Kafka transactions instead make processing an input and publishing its Kafka outputs an atomic operation, including writes to multiple topics. section 10.1.3 section 7.6
Figure 7.24 OrderService reads, processes, and writes each event atomically.
The order pipeline shows why the shipment, billing, and inventory outputs need to succeed together rather than appear as partial results. section 7.4.2
In Kafka Streams, an all-or-nothing transaction coordinates:
- Producing records to output topics. section 8.3.4
- Updating state backed by changelog topics. section 8.3.4
- Committing consumed-record offsets. Set processing.guarantee to EXACTLY_ONCE_V2 to enable this processing guarantee. section 8.3.4
Choose the guarantee for the effects that actually need it, rather than treating it as a blanket promise for every downstream system. Kafka does not currently support two-phase commit; the transactional outbox pattern is often used as a workaround. Kafka Streams provides exactly-once processing without custom deduplication logic for its supported pipeline. section 7.6 section 8.1.3
