A stream join combines related records from multiple streams or tables when their keys or other conditions match. It supports event enrichment, such as producing a more complete object from an order and its payment. section 8.3.6
Architecturally, a join can move enrichment and aggregation out of a final Kafka consumer into an intermediate streaming service. The service publishes a reusable aggregated result, while downstream services read prepared data. section 8.0
Design a join around these constraints:
- Match records with an explicit condition, such as order.id = payment.order_id. section 8.3.6
- Use the same number of partitions for every input topic. section 8.3.6
- Use the same key distribution strategy and partitioning logic so matching records reach corresponding partitions. section 8.3.6
- Treat key alignment as application responsibility because Kafka Streams does not validate or enforce it at runtime. section 8.3.6
Orders and payments
An order keyed by orderId can join a payment when the payment topic is repartitioned with the same orderId key. Partitions with the same index are assigned to the same processing task, allowing corresponding records to be processed together. The design must also account for orders and payments arriving at different times or in either order. section 8.3.6
Figure 8.13 Partitions with the same index are assigned to the same task, and this assignment remains fixed. Elements in input topics that arrive in partitions with the same index will be processed together.
Use the partition-assignment diagram to see why equal partition indexes bring records from joined topics to the same Kafka Streams task. section 8.3.6
