Copartitioning
Copartitioning is the arrangement of input topics with compatible partitioning so records with corresponding keys can be processed together.
Kafka Streams in Action, Second Edition
Co-partitioning is the arrangement in which participating Kafka Streams inputs have the same partition count and compatible key types so corresponding records can be processed together.
- How it is used
- Enables Kafka Streams and ksqlDB joins by aligning keyed inputs across corresponding partitions.
- How deep it goes
- Defines partition-count, key-type, and partitioner requirements, then explains repartitioning and the GlobalKTable exception.
- Examples
- Uses retailer-and-café purchases, clickstream enrichment, Java stream-table code, and ksqlDB customer-reward queries.
- What you can do after
- Operate/configure Kafka Streams or ksqlDB join inputs by checking keys, partition counts, and repartitioning needs.
Read how Kafka Streams in Action, Second Edition explains copartitioning →
Kafka for Architects
Copartitioning is the arrangement of multiple input topics with matching partition counts and partitioning logic so records with corresponding keys are processed together.
- How it is used
- Supports stream joins by routing corresponding records to the same processing task.
- How deep it goes
- Defines partition, key, and partitioner requirements and assigns correctness responsibility to programmers.
- Examples
- Uses Orders and Payments topics, orderId alignment, paymentId replacement, and same-index task assignment.
- What you can do after
- Configure copartitioned Kafka Streams inputs and rekey records before joining them.
Related concepts
- joinused by · A join is an operation that combines records or rows from multiple data sources according to specified matching conditions or relationships.
- stream-global-table joincontrasts with · A stream-global-table join combines records from a KStream with matching records in a fully replicated GlobalKTable.
- GlobalKTablecontrasts with · GlobalKTable is a Kafka Streams abstraction that replicates all records from an underlying topic to every application instance for read-only access.
- foreign key associationcontrasts with · A foreign key association is a relationship or join between records whose key in one entity or table references a key in another.
- StreamPartitioneruses · StreamPartitioner is a Kafka Streams interface for determining the output partition assigned to a record.
- repartitionproduced by · Repartitioning is the redistribution of records or items across partitions according to a partitioning rule, potentially changing the number of partitions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.