liveBook · Manning

Data enrichment

As explained in Apache Kafka in Action

Data enrichment is the process of augmenting records with related attributes from another data source.

Data enrichment augments each streaming record with related attributes from a lookup table. A sales stream keyed by product ID can retrieve current product master data and add the manufacturer before producing an enriched stream. section 12.4.3

Enriching sales with product data

Treat rarely changing product master data as a table backed by a compacted topic. For every sales message, use its product ID to find the matching table value, then combine the sales and product records. The lookup executes as messages arrive rather than waiting for a complete dataset. section 12.4.3

How Apache Kafka in Action covers it

How deep it goes
Explains keyed lookup state and join execution, but does not detail full operational implementation.
Examples
Uses sales and product data, adding a product manufacturer through a KStream-to-KTable join.
What it assumes
Assumes keyed streaming records and table-shaped reference data, while grounding joins in message-by-message lookup.
What you can do after
Apply it via KStream/KTable joins to attach reference attributes to incoming records.
How it connects
Connects stream processing, streaming joins, compacted topics, tables, and keyed records.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.