Event enrichment

As explained in Kafka for Architects

Event enrichment is the combination of related events to produce a more complete representation of a business object or occurrence.

Event enrichment combines related events to produce a more complete representation of a business object or occurrence. A common pattern joins an order with its corresponding payment using order.id = payment.order_id, then emits an aggregated object. section 8.3.6

Reliable enrichment depends on two architectural conditions:

• Joined topics must have the same partition count, key distribution strategy, and partitioning logic. section 8.3.6
• Records with the same join key must reach corresponding partitions so Kafka Streams assigns them to the same task. section 8.3.6
• Applications must handle events arriving at different times and in either order. section 8.3.6

Figure 8.13 Partitions with the same index are assigned to the same task, and this assignment remains fixed. Elements in input topics that arrive in partitions with the same index will be processed together.

Partition diagrams show why matching partition indexes let related records be processed together. section 8.3.6

How Kafka for Architects covers it

How deep it goes

Explains matching, arrival timing, copartitioning, and Kafka Streams task assignment.

Examples

Uses order and payment events joined through order identifiers and aligned partitions.

What you can do after

Apply it via Kafka Streams by aligning join keys, partitions, and processing tasks.

How it connects

Connects stream joins with keys, partitioning, aggregation, and Kafka Streams tasks.

liveBookmanning.com

Read Kafka for Architects on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.