A Kafka topic splits its records across partitions so consumer instances can process them in parallel, potentially on different servers. This distribution is an architectural choice: records in different partitions may not be processed together, and ordering is guaranteed only within a partition. section 3.2.1 section 5.2.2
Choose how producers assign records according to the work consumers must do:
- Use a stable message key when related records must share a partition and retain their order. Kafka’s default strategy derives the partition from the key’s hash and the partition count. section 3.2.1
- Without a key, the default sticky strategy sends a batch to one partition, then rotates for another batch; it does not guarantee per-key ordering or co-location. section 3.2.1
- Use round-robin assignment when even distribution matters more than key-based ordering. Explicit partition selection and custom partitioners are alternatives when the standard strategies do not fit. section 3.2.1 section 3.2.6
Customer data topics
For customer profiles, use the customer identifier as the key so updates stay in order. For transactions, use the transaction identifier to organize processing, while the business timestamp expresses their natural order. Record these decisions alongside the topic design: a partition count alone does not specify which records belong together. section 3.2.2 section 2.2.4
Partitioning limits whole-topic operations. Finding each day’s ten most expensive orders requires gathering orders across partitions; a daily warehouse bulk upload likewise needs a consumer to assemble data from every partition. One partition simplifies collection but gives up parallel processing and scalability. section 5.2.2 section 5.2.9
One partition also provides a single partition order when global ordering is required, at the cost of parallelism. It does not make timestamps an arrival order: network delays can cause messages from different producers to reach the broker out of timestamp order. section 5.2.10
Listing 2.1 An initial configuration for the Customer 360 project
asyncapi: 3.0.0
info:
title: Customer 360 Project
version: 1.0.0
description: >
This project aims to aggregate data from various sources, providing a
comprehensive view of customer information.
channels:
customerProfile: #1
description: Storing profile information of customers.
address: CUSTOMER_PROFILES #2
bindings: #3
kafka:
partitions: 9 #4
transactions:
description: Information about customer transactions.
address: TRANSACTIONS
bindings:
kafka:
partitions: 9
The AsyncAPI configuration records partition counts for two topics. Document the producer’s key-selection rule separately; AsyncAPI has no standard field for the partitioning strategy. section 2.2.4 section 3.2.2
