Suppose a customer changes address twice. If the updates land in different Kafka partitions, a consumer cannot rely on processing them in order. A partitioning key identifies the entity whose events must stay together. Using the customer ID as each message’s key sends that customer’s updates to the same partition, where their order is preserved. section 2.4.1 section 2.7
A Kafka message has a key distinct from its value, which carries the application payload. By default, the producer selects a partition from the key; brokers do not interpret the key’s business meaning. Ordering is guaranteed within a partition, not across a topic. Choose a stable key for the set of events that business logic requires in sequence. section 2.4.1 section 2.4.7 section 3.2.1
Figure 2.11 Selecting the key for the Balances topic. The account number is used as the key, ensuring that messages for the same account are routed to the same partition. This setup guarantees that updates for each account are received in the correct order.
Account numbers serve as keys so balance updates for one account share a partition and retain their order. section 2.4.1
Select a key according to the work the topic must support:
- For ordered updates to an account or customer, use the account or customer identifier—not an address that may change. section 2.4.1
- If transactions do not need per-entity ordering, a transaction ID can still help distribute them across partitions. A round-robin strategy is another option when ordering by key is unnecessary. section 2.4.2 section 3.2.6
- Check whether the chosen keys spread work adequately. A poorly balanced partitioning strategy can undermine processing parallelism. section 3.2.1 section 3.2.6
Without a key, the default producer strategy places batches on partitions using sticky partitioning; it provides no per-key co-location or ordering. Producers can instead select another partitioning strategy, such as round-robin. Make that choice against the workflow’s ordering and throughput requirements, rather than assuming that all events in a topic will be processed together. section 3.2.1 section 10.1.3
Documenting a customer-profile key
For a customer-profile topic, define the Kafka message key as a customer ID in the AsyncAPI message binding. That makes the ordering decision visible to teams before they settle every detail of the value’s format. section 2.4.2
Keys also determine what a compacted topic retains: at least the most recent message for each unique key. Such topics require non-null keys; a null value with a valid key is a tombstone for deletion. Compaction does not turn the topic into a quick key lookup: consumers still read the log sequentially and may encounter older messages before cleanup finishes. section 2.5.1 section 4.2.12
