liveBook · Manning

Data-driven windowing

As explained in Kafka Streams in Action, Second Edition

Data-driven windowing is the technique of defining a stream window through record values or behavioral conditions instead of predetermined time boundaries.

Data-driven windowing defines an aggregation through record values and behavioral conditions rather than fixed time boundaries. A sensor aggregation can start or accept readings when a predicate matches, stop when another predicate rejects a reading, and emit after 10 seconds without a closing or additional high reading. section 10.3

A custom Kafka Streams Processor API implementation typically combines:

  • Predicates that decide when to start, include, or stop an aggregation. section 10.3

Sensor readings with unreliable connections

Aggregate qualifying sensor readings and update the aggregation’s start time, end time, readings, and average. If no closing or additional high reading arrives for 10 seconds, clean out and forward the unfinished aggregation. section 10.3

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains predicates, state stores, stream-time handling, and punctuation implementation, while omitting complete aggregation code.
Examples
Uses unreliable sensor connections, high readings, closing readings, keyed SensorAggregation state, and a ten-second cleanup threshold.
What it assumes
Assumes familiarity with Kafka Streams Processor API applications and introduces the custom aggregation logic around that foundation.
What you can do after
Build custom Kafka Streams Processor API windowing with predicates, a state store, stream-time tracking, and scheduled cleanup.
How it connects
Connects Kafka Streams, the Processor API, state stores, predicates, stream time, punctuation, and session windows.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.