liveBook · Manning

Windowing

As explained in Kafka Streams in Action, Second Edition

A windowed operation processes records within bounded time intervals and produces results associated with those intervals.

Windowing bounds a stream aggregation to discrete time intervals. Instead of carrying one ever-growing total, it produces results that describe a particular period, making changes in current behavior visible. Hourly pizza sales, for example, reveal a lunch rush that a daily total hides. In Kafka Streams, windowing is available on KStream aggregations, whose records are events over time; KTable aggregations are point-in-time groupings and do not use windows. section 9.0

Start with a keyed stream, then place windowedBy after groupByKey. The window definition determines which records belong together before aggregate, reduce, or count computes a result. A windowed aggregation changes the output key to Windowed<K>, which carries both the original key and its window. Either unwrap that key before writing downstream or use a Windowed serde when the interval must remain part of the result. section 9.1.1

Choose a window shape from the question the aggregation must answer:

  • Hopping windows have a fixed size and advance by a smaller or equal interval. They can overlap, so an event may contribute to successive windows. section 9.1.1
  • Tumbling windows are fixed-size windows over time. section 9.7
  • Sliding windows have a time-based size, but record behavior determines which records enter a window. section 9.7
  • Session windows are driven by activity and can grow while records arrive within the configured inactivity gap. section 9.7

One-minute temperature view every 10 seconds

For a temperature-monitoring stream, group readings by sensor location and define a one-minute TimeWindow that advances every 10 seconds. Each result reports the recent average and highest reading, rather than statistics accumulated since startup. The overlap is intentional: every update compares the latest minute of production conditions with the preceding one. Use TimeWindows.ofSizeWithNoGrace(Duration.ofMinutes(1)).advanceBy(Duration.ofSeconds(10)) before the aggregation. section 9.1 section 9.1.1

A window can emit intermediate updates while it remains open. Stateful operations cache updates, forwarding them on commits or cache flushes and persisting them to the state store and changelog topic. When downstream consumers need one completed value, configure EmitStrategy.onWindowClose() or suppress results until the window closes. Suppression offers strict or eager buffering; EmitStrategy is the simpler final-result option. section 9.3

Timestamps drive window opening and closing through stream time: the largest timestamp a processor has seen. Stream time advances only forward and is task- and partition-specific, not global. Out-of-order records are normally processed, though grace-period rules can affect windowed operations; low or bursty input may leave stream time short of the point needed to advance a window. section 9.6 section 9.7

Testing a suppressed window

With TopologyTestDriver, feed records into a one-minute suppressed window and first assert that no output is available. Then pipe a record timestamped 75 seconds later to move stream time forward; the closed prior window emits its count. Explicit timestamps or advancing a TestInputTopic's event time are the way to drive and verify window behavior. section 14.1.4

Figure 9.9 Average temperatures of the past minute, reported every 10 seconds
figure

This timeline makes the defining hopping-window trade-off concrete: a one-minute calculation is refreshed every 10 seconds, so consecutive results overlap. section 9.1.1

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Builds windowed aggregations through Java APIs, output-key handling, emission control, stream time, and timestamp-driven tests.
Examples
Uses pizza staffing, IoT temperature monitoring, page-view sessions, and a suppressed tumbling-window test.
What it assumes
Builds on KStream and KTable aggregations while briefly distinguishing their different fit for windowing.
What you can do after
Build it from scratch with Kafka Streams window APIs and test closure by advancing input timestamps.
How it connects
Connects KStream aggregation, timestamps, stream time, state stores, changelog topics, suppression, and EmitStrategy.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.