liveBook · Manning

Out-of-order events

As explained in Kafka Streams in Action, Second Edition

An out-of-order record is a record whose timestamp does not follow the timestamp order of records already processed.

An out-of-order event has a timestamp earlier than the preceding record. Kafka processes records by offset, so network delays or producer payload timestamps can make timestamp order differ from arrival order. section 9.2

In a windowed aggregation, an event arriving after its window closes may be excluded even though its timestamp places it inside that window. Kafka Streams handles this with grace: a configured period after closure during which such records remain eligible. section 9.2

Configure window handling according to the accuracy and lateness your application can tolerate:

  • Use TimeWindows.ofSizeAndGrace to admit out-of-order records during a grace period after the window closes. section 9.2
  • Use a WithNoGrace window constructor when records arriving after closure should be excluded. section 9.2
  • Treat the grace duration as application-specific; longer grace can improve calculation accuracy but delays final acceptance of late records. section 9.2

A delayed event in a sales window

Suppose records for a sales window arrive in offset order, but one record carries an earlier event timestamp because a producer or network path delayed it. Without grace, the closed window drops it; with grace, Kafka Streams can include it in the aggregation. section 9.2

Figure 9.26 Grace is the amount of time you’ll allow out-of-order records into a window after its configured close time.
figure

Shows grace as the interval during which out-of-order records remain admissible after a window’s configured close time. section 9.2

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains timestamp disorder, window effects, grace configuration, and late-record dropping, but leaves duration selection application-specific.
Examples
Uses delayed producer records, embedded payload timestamps, network partitions, and window diagrams to illustrate lateness.
What it assumes
Assumes familiarity with Kafka record ordering and stream aggregation while grounding the concept in timestamps and windows.
What you can do after
Configure Kafka Streams windows to accept or reject records arriving after their event-time window closes.
How it connects
Connects Kafka offsets and producers with timestamps, network partitions, Kafka Streams, KStream aggregation, windowing, and grace periods.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.