liveBook · Manning

Key-based distribution strategy

As explained in Apache Kafka in Action

Key distribution is the pattern by which message keys and their associated records are spread across partitions.

A key-based distribution strategy assigns each message to a partition according to its key. Messages sharing a key remain in the same partition, preserving their relative order while allowing different keys to be processed across partitions. section 4.2.1

Product-price updates

Produce records such as `coffee pads:10`, `cola:2`, and `energy drink:3` with key parsing enabled. Consume them with key printing enabled to verify that repeated keys, such as `coffee pads`, stay together even though records appear in a different overall order. section 4.2.1

Plan for distribution before choosing keys:

  • Prefer many keys, such as customer IDs, when traffic is broadly distributed. section 4.2.1
  • Watch for a few high-traffic keys, because they can overload individual partitions and brokers. section 4.2.1

How Apache Kafka in Action covers it

How deep it goes
Explains keyed routing, demonstrates Kafka commands, and warns about skewed traffic; it stops short of detailed partitioner internals.
Examples
Demonstrates product-price records, customer IDs, and high-traffic social-network members.
What you can do after
Operate/configure Kafka console producers and consumers to create, inspect, and evaluate keyed partition placement.
How it connects
Connects message keys with partitions, ordering, hashing, brokers, workload balancing, and bottlenecks.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.