A join combines related records to produce a new event or enrich an existing one. In event processing, it uses state to recall matching records and connect them with current records, much like joining database tables. section 7.0 section 1.4.5
Choose the join shape by the data relationship:
- Use a stream-stream join to combine events sharing a key within a time window. Both inputs must be co-partitioned: the same key field and type, with the same partition count. section 7.3 section 11.6
- Use a stream-table join to enrich incoming stream records with a table lookup; only new stream-side records produce results. section 11.6
- Use table-table joins for primary-key relationships or foreign-key relationships based on a value field. section 11.7
In the Kafka Streams Java DSL, `KStream.join` takes the other stream, a `ValueJoiner`, a `JoinWindows` definition, and optional `StreamJoined` settings. The joiner combines the two values into an output object; the window limits how far before or after an event a matching record may occur. Serdes are required because join participants are materialized in windowed state stores. section 7.3.1
Reward promotion within 30 minutes
Key cafe and retail purchases by customer ID, then join purchases occurring within 30 minutes. The joined result can assign reward points from the retail purchase amount. A subsequent left outer stream-table join can add member details such as name and membership year. section 7.3 section 11.6
Listing 11.34 Creating a stream-stream join for potential customer rewards
CREATE STREAM customer-rewards-stream AS
SELECT c.custId AS customerId, #1
s.total as amount, #2
CASE #3
WHEN s.total < 25.00 THEN 15
WHEN s.total < 50.00 THEN 50
ELSE 75
END AS reward_points
FROM coffee-purchase-stream c
INNER JOIN store-purchase-stream s
WITHIN 30 MINUTES GRACE PERIOD 2 MINUTES #4
ON c.custId = s.custId #5
Inspect a ksqlDB stream-stream join that matches customer IDs within 30 minutes and uses a CASE expression to compute reward points. section 11.6
