liveBook · Manning

Table-to-table join

As explained in Kafka Streams in Action, Second Edition

A table-table join is a Kafka Streams operation that combines two KTable representations and recalculates results when either table changes.

A table-to-table join combines two changing table views into a new result. Unlike a stream-stream join, it has no window: an update on either side updates that table and can trigger a recalculated join result. A ValueJoiner computes the result and may return any value type. section 8.6.4

Choose the join form from the relationship between table keys:

  • Primary-key join: join tables whose records use the same primary key. Their source topics must have the same number of partitions so matching keys are processed together. section 8.6.4 section 8.6
  • Foreign-key join: use a field in one table’s value that matches the other table’s primary key. This supports relationships such as client contracts keyed by client ID and commodity prices keyed by commodity code. section 8.6.4
  • ksqlDB foreign-key join: join a table’s primary key to a nonprimary-key column on the other table; joining against a value column removes the co-partitioning restriction. section 11.6

Reward activity counts by member

An activity-count table keyed by last_name can join a rewards-members table keyed by member_id through its last_name column. In ksqlDB, create the result table with a JOIN on those columns and EMIT CHANGES, so the joined table follows later changes. section 11.6

In Kafka Streams, a foreign-key join extracts the matching key from the left value and uses repartition topics to route lookup and result work. Processing also tracks the original value with a hash; after a right-side update, matching composite keys are scanned so all affected results can be produced. section 8.6.4

Figure 8.25 Left-side table in a table–table join is updated, and a new join result is triggered.
figure

Inspect the left-side update path: updating one KTable first changes its state, then potentially emits a new joined result. section 8.6.4

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains update triggers, partition constraints, foreign-key joins, and repartition-based processing without detailing API configuration.
Examples
Uses client contracts with commodity prices and activity counts joined to rewards members.
What it assumes
Introduces co-partitioning through topic partition counts and teaches repartitioning as a remedy for incompatible join inputs.
What you can do after
Apply it via Kafka Streams or ksqlDB to produce joined table results when either input table changes.
How it connects
Connects KTables to ValueJoiner, co-partitioning, repartitioning, primary keys, foreign keys, and ksqlDB.

Also explained in

Compare how 2 books explain table-to-table join →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.