liveBook · Manning

Lookup operation

As explained in Apache Kafka in Action

A lookup is the retrieval of a corresponding value from stored state by a key or identifier.

A lookup retrieves the value associated with a message key from stored state. In stream processing, it replaces an impossible cross-table operation over infinite streams: process one message, find its corresponding table entry, and use both values for a join or enrichment. section 12.4.3

Enriching sales with product data

Treat compacted product master data as a table keyed by product ID. For each keyed sales message, look up the current product record and add its manufacturer before producing enriched sales data. The same pattern joins loyalty cards with customers. section 12.4.3

KStream salesData = builder.stream("salesData");    #1
KTable masterData = builder.table("products");    #2
ValueJoiner enricher = (salesData, masterData) -> {    #3
    salesData.manufacturer = masterData.manufacturer;
    return salesData;
};
KStream salesData_enriched = salesData.join(
    masterData, enricher);    #4

The topology creates a sales stream and product table, then passes the stream value and looked-up table value to a joiner. section 12.4.3

How Apache Kafka in Action covers it

How deep it goes
Explains per-message state lookup through stream-table joins and data enrichment, without descending into state-store internals.
Examples
Demonstrates product-sales enrichment and loyalty-card customer joins using keyed tables.
What it assumes
Assumes messages, keys, streams, and tables, while establishing why infinite streams require stateful lookup instead of cross tables.
What you can do after
Apply it via stream-table join APIs to enrich incoming records with current master or reference data.
How it connects
Connects streaming joins, stream processing, tables, data streams, keys, and data enrichment.

Also explained in

Compare how 2 books explain lookup operation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.