Kafka transaction
A Kafka transaction is an atomic operation that groups record production and related offset commits so their effects are committed together or aborted.
Kafka Streams in Action, Second Edition
A Kafka transaction is an atomic unit that groups record production and related offset commits so they either all succeed or all abort.
- How it is used
- Ensures exactly-once processing when producing records, especially across partitions or during consume-transform-produce flows.
- How deep it goes
- Walks through producer initialization, transaction control, consumer isolation, last stable offsets, and transactional offset commits, while deferring transaction design internals.
- Examples
- Uses a financial duplicate-prevention scenario and a stock compliance-reporting consume-transform-produce application with TopicPartition offsets.
- What you can do after
- Build it from scratch with KafkaProducer transaction calls, read_committed consumers, and sendOffsetsToTransaction() for atomic processing.
Read how Kafka Streams in Action, Second Edition explains Kafka transaction →
Apache Kafka in Action
A Kafka transaction is an atomic operation that groups message writes and related offset commits so they either all succeed or all roll back.
- How it is used
- Keeps multi-partition writes, and optionally related consumer offsets, consistent during message processing.
- How deep it goes
- Works through producer setup, commit and abort behavior, consumer visibility, coordinator markers, and the delay caused by unfinished transactions.
- Examples
- Uses a Python balance-transfer producer, a trip-booking analogy, and PostgreSQL alternatives for storing processed data and offsets.
- What you can do after
- Operate/configure it with a transactional producer and read-committed consumers, while accounting for unfinished-transaction delays.
Read how Apache Kafka in Action explains Kafka transaction →
Kafka for Architects
A Kafka transaction is an atomic operation that groups record production and related offset handling so their effects can be committed or aborted together.
- How it is used
- Coordinates read-process-write pipelines whose outputs must remain consistent across retries, failures, and multiple Kafka topics.
- How deep it goes
- Defines the Kafka-specific exactly-once model and consumer isolation, without providing a low-level implementation walkthrough.
- Examples
- Uses an OrderService that reads ORDERS and writes SHIPMENTS, BILLING, and INVENTORY atomically.
- What you can do after
- Apply Kafka transactions to read-process-write workflows requiring exactly-once results across multiple output topics.
Related concepts
- exactly-once deliveryaddresses · Exactly-once delivery is a messaging or processing guarantee that each message or input produces its intended effect once, without loss or duplicate effects, despite retries or failures.
- producer idempotenceuses · Producer idempotence is a Kafka delivery property that makes producer retries safe from creating duplicate records while preserving record ordering.
- atomicityhas part · Atomicity is the property that an operation or group of operations takes effect as an indivisible, all-or-nothing unit, without observable partial effects.
- isolationhas part · Isolation is the separation of components, processes, resources, or contexts to limit interaction, interference, or the propagation of effects between them.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.