Sliding window

As explained in Kafka for Architects

A sliding window is a stream-processing time interval defined by the timestamp difference between records being considered for a join or computation.

A sliding window bounds a stream join by the difference between event timestamps. It lets related records wait for one another without requiring simultaneous arrival. section 8.3.6

Use sliding windows when asynchronous events must still be matched:

• Keep each event in a state store until its matching event arrives or the window expires. section 8.3.6
• Match records when their timestamp difference is less than the configured interval; discard unmatched records after expiry. section 8.3.6
• Key the participating streams consistently and satisfy copartitioning requirements so matching records are processed together. section 8.3.6

Orders and payments

An order and its payment may arrive at different times. With a one-day sliding window, Kafka Streams retains each event and joins the pair when their timestamps differ by less than one day; otherwise, the unmatched event is discarded. section 8.3.6

Figure 8.14 When the Payments and Orders topics are joined, they must have the same number of input partitions and the same key (orderId).

Use the partition diagram to verify that joined topics share the same key and partition count, allowing corresponding records to reach the same task. section 8.3.6

How Kafka for Architects covers it

How deep it goes

Defines timestamp-based matching, state-store waiting, expiry, and partitioning constraints without detailing implementation code.

Examples

Uses a one-day Orders-and-Payments join in which unmatched events expire.

What it assumes

Assumes event timestamps, stream keys, partitions, and state stores; distinguishes sliding windows from other window types.

What you can do after

Apply it via Kafka Streams to join records whose event timestamps fall within a configured interval.

How it connects

Connects stream joins with event timestamps, state stores, keys, partitions, and copartitioning.

liveBookmanning.com

Read Kafka for Architects on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Build a Large Language Model (From Scratch)A data-sampling method that moves a fixed-size window across text to create overlapping training examples.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA technique that moves a fixed-size window across text or other data to create overlapping local segments.
• 

  liveBookmanning.com
  Kafka Streams in Action, Second EditionA sliding window is a fixed-duration stream-processing window whose boundaries are determined by record timestamps and whose contents change as records enter or leave.
• 

  liveBookmanning.com
  API Design PatternsA sliding window is a bounded set of currently active versions that advances as newer versions replace older ones.

Compare how 5 books explain sliding window →

Related concepts

• text chunkproduces · A text chunk is a bounded segment of a larger text treated as a unit for processing, storage, or retrieval.
• time windowkind of · A bounded temporal interval or event scope over which observations or events are grouped, evaluated, or processed.
• convolutionused by · Convolution is an operation that applies a kernel across an input, combining values from overlapping local regions with shared weights to produce outputs.
• tokenuses · A token is a discrete unit of text, such as a word, word fragment, character, or symbol, represented and processed as an element by a language model.
• timestampuses · A timestamp is a value representing a specific point in time, typically associated with an event, observation, or data item.

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.