liveBook · Manning

Streaming

As explained in Digital Twins in Action

A data stream is a continuously or repeatedly produced sequence of data records from one or more sources.

Stream processing handles data as it arrives, rather than waiting for a batch to finish. In a digital twin, continuous ingestion and updating help the digital representation track changes in its physical counterpart. The useful output is often a current trend or event, not every raw reading. section 4.4.2

Consider a temperature sensor that sends one reading each second. A 30-second sliding average smooths noise while remaining responsive: add each new reading, remove the oldest reading outside the window, and recalculate. The same pattern supports minimums, maximums, and alerts when a recent trend crosses a threshold. section 4.4.2

Figure 4.7 A sliding window of length 4 over a data stream. As the window moves through time, older readings fall out, and new readings are added.
figure

The sliding-window diagram makes the update rule visible: new readings enter while older readings leave. Window size determines which recent data contributes to each result. section 4.4.2

A practical stream-processing path has three parts:

  • Prepare incoming data. Decode sensor-specific formats and transform protocols before ingestion; link readings to their physical context, such as the room containing a sensor. section 3.5 section 1.3.2
  • Update a useful state as records arrive. Maintain a recent window and compute an aggregate instead of treating each reading as an independent decision. section 4.4.2
  • Expose the result. Dashboards can show changing temperatures; a traffic map can overlay sensor and mobile-device data to show current congestion. section 1.3.2

A five-minute rail movement window

A public train-movement feed provides positioning and movement events. For each relevant movement message, keep its arrival time and whether its status is late; discard entries older than five minutes and calculate the share of movements that are on time. This turns individual events into a changing operational measure. section 4.4.2

Processing location matters when records are large. Sending an entire video feed enables continuous monitoring but consumes substantial bandwidth and storage. Alternatives send footage only when a relevant change is detected, or process video at the edge and transmit derived events such as a vehicle detection or occupancy count. Choose what to send according to the detail the twin needs and the resources available. section 2.3.3

How Digital Twins in Action covers it

How deep it goes
Works through sliding-window aggregation and a train-event calculation, but does not detail a general-purpose stream-processing engine.
Examples
Uses second-by-second temperature readings and a five-minute window over public train-movement events.
What it assumes
Assumes incoming sensor readings or event records; introduces sliding-window aggregation through a one-reading-per-second example.
What you can do after
Build it from scratch: maintain a recent event window and calculate an updated operational measure as records arrive.
How it connects
Connects streaming ingestion to digital twins, sensor decoding, sliding windows, dashboards, and edge processing.

Also explained in

Compare how 19 books explain streaming →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Digital Twins in Action. The section links point to the parts of the book it draws on.