A time-windowed aggregation computes an aggregate for records assigned to a particular time window. In Kafka Streams, group records by key, apply a window definition, and aggregate into a KTable; each updated result replaces the prior result for that key and window. section 9.1.1 section 9.1.6
Window choice determines which records contribute to each result. Hopping windows measure changes every x over the last y, tumbling windows measure events in fixed periods, sliding windows support rolling changes, and session windows group behavior events occurring close together. section 9.1.6
Listing 9.3 Adding windowing to the aggregation for the Iot temperature readings
KStream<String,Double> iotHeatSensorStream =
builder.stream("heat-sensor-input",
Consumed.with(stringSerde, doubleSerde));
iotHeatSensorStream.groupByKey() #1
.windowedBy(TimeWindows.ofSizeWithNoGrace(
Duration.ofMinutes(1)) #2
.advanceBy(Duration.ofSeconds(10)))
.aggregate(
() -> new IotSensorAggregation(tempThreshold), #3
aggregator,
Materialized.with(stringSerde, aggregationSerde))
.toStream().to("sensor-agg-output",
Produced.with(serdeString,
sensorAggregationSerde)) #4
This pipeline groups IoT temperature readings, creates one-minute hopping windows advancing every 10 seconds, and materializes an aggregation before writing results to an output topic. section 9.1.1
A windowed aggregation changes the output key type: Kafka Streams wraps the original key in Windowed, which holds both the key and Window. The window provides start and end times, so retaining the Windowed key preserves the timeframe needed to interpret a result. Configure a Windowed serde to write that key, or map to the underlying key when downstream consumers do not need window timing. section 9.1.1 section 9.1.6
Treat result handling as part of the aggregation design:
- Convert the aggregation KTable with toStream() before inspecting or publishing its updates. section 9.1.6
- Use peek to retrieve window.startTime() and window.endTime() alongside the aggregate for logging or analysis. section 9.1.6
- Map window start and end times into the aggregate value when an output format needs a plain key but still needs the timeframe. section 9.1.6
- Partition output using the underlying key when sending Windowed keys requires records for a logical key to reach the correct partition. section 9.1.6
Late temperature readings
A record can arrive after records with later timestamps because Kafka processes records in offset order. Without grace, a record that would otherwise belong to a closed window is excluded. Configure grace to admit eligible out-of-order records for a period after closure; after that period, late records are dropped. Choose the grace duration case by case. section 9.2
Results may update while a window remains open. Suppression can hold those updates until the window closes, yielding only a final aggregate. An unbounded buffer keeps suppressing until closure; a strict bounded buffer can limit records or bytes and shut the application down gracefully if the limit is exceeded. section 9.3.1
