liveBook · Manning

Time window

As explained in Apache Kafka in Action

A time window is a bounded temporal interval over which stream records are aggregated, correlated, or otherwise processed.

A time window bounds a stream-processing operation to an interval. It lets an application count recent events or correlate records from two otherwise unbounded streams. For example, sales and product-price changes can be joined when records with the same product ID fall within one minute of each other; the processor retains data from both streams for that period. section 12.4.3 section 12.5.1

Choose the window shape to match the question:

  • Tumbling windows are consecutive and nonoverlapping. Use them when each message should belong to one interval, such as a billing period. section 12.5.2
  • Sliding windows answer continuously updated questions such as how many transactions occurred in the last five minutes. section 12.5.2
  • Hopping windows have a length and an advancement interval. Use them for periodic results over overlapping periods, such as a daily report covering the previous seven days. section 12.5.2
  • Session windows close after a period without messages, making them useful for analyzing user sessions. section 12.5.2
KStream<String, String> authRequests = builder.stream("auth_requests");
KTable<Windowed<String>, Long> windowedCounts = authRequests
    .groupByKey()    #1
    .windowedBy(TimeWindows.ofSizeWithNoGrace(
        Duration.ofMinutes(1)))    #2
    .count();    #3
KStream<String, Long> possibleFrauds = windowedCounts
    .toStream()
    .filter((windowedKey, count) -> count > 3)    #4
    .map((windowedKey, count) -> #5
        KeyValue.pair(windowedKey.key(), count));     #5
possibleFrauds.to("possible_frauds");    #6

A Kafka Streams pipeline groups authentication requests by key, counts them in one-minute windows, and writes qualifying counts to a fraud-alert topic. Note the exact threshold: the code uses count > 3, so it emits counts of four or more. section 12.5.3

Decide which clock defines the interval: event time, producer creation time, broker log-append time, or processing time. Timestamps need not arrive in ascending order, so a late message may be ignored without suitable lateness configuration. For closely spaced, overlapping results, sliding windows generally avoid the frequent recalculation associated with hopping windows. section 12.5.1 section 12.5.2

How Apache Kafka in Action covers it

How deep it goes
Defines four window types and discusses timestamp choice and late arrivals, but leaves lateness configuration at a high level.
Examples
Uses a one-minute product-price-and-sale join and a one-minute count of authentication attempts for fraud detection.
What it assumes
Introduces window types through practical uses while employing Kafka topics, keyed streams, and aggregation syntax in examples.
What you can do after
Apply it via Kafka Streams or Flink SQL to count authentication attempts or correlate price changes with sales.
How it connects
Connects stream-stream joins and aggregations with event time, processing time, keyed streams, and grace periods.

Also explained in

Compare how 6 books explain time window →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.