Partitioner

As explained in Apache Kafka in Action

A partitioner is the Kafka component that selects the partition to which a record is sent.

A partitioner selects the Kafka partition for a record. Producers use it to distribute topic data, while records with the same key are routed to the same partition. section 4.5 section 1.3

A record follows this production path:

• The producer serializes keys and values into byte arrays. section 8.1.2
• The partitioner hashes the key to choose a partition, or uses round-robin distribution when no key is available. section 8.1.2
• The producer buffers messages by partition, forms batches, and sends them to the partition leaders. section 8.1.2

Bank transfer events

A bank transfer application produces an event for each fund transfer. The producer uses a partitioner to select where each event is written, allowing the topic’s partitions to support parallel processing. section 1.3 section 4.5

Figure 1.2 The components and data flow of Kafka

Trace how producers, partitioners, partition leaders, brokers, and consumers participate in Kafka’s data flow. section 1.3

How Apache Kafka in Action covers it

How deep it goes

Explains selection after serialization, key hashing, keyless round-robin routing, and explicit partition overrides.

Examples

Uses bank transfer events and partition-based producer buffers to illustrate routing and delivery.

What it assumes

Assumes familiarity with Kafka producers and treats serialized keys and values as byte-array inputs.

What you can do after

Apply Kafka producer partitioning with keyed, keyless, or explicitly assigned records.

How it connects

Connects producers with partitions, topics, message keys, serialization, batching, brokers, and consumer groups.

liveBookmanning.com

Read Apache Kafka in Action on liveBook Get the book at manning.com

Related concepts

• Kafka producerused by · A Kafka producer is an application or client that publishes records to Kafka topics.
• StreamPartitionerincludes · StreamPartitioner is a Kafka Streams interface for determining the output partition assigned to a record.
• partitionproduces · A partition is a distinct subdivision of a larger data, storage, or work unit that can be stored, managed, or processed separately.
• partitioning keyuses · A partitioning key is a value or attribute used to determine which partition stores or receives a record or data item.

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.