liveBook · Manning

Stream processing

Stream processing is the incremental processing of data items as they become available, without requiring the complete dataset to be present first.

How 20 Manning books explain it

Functional Programming in Scala, Second Edition

Stream processing is the incremental computation of results from sequences of data as elements become available.

How it is used
Frames efficient external I/O as composable incremental transformations rather than monolithic loops.
How deep it goes
Develops Pull and Stream internals, pipes, effects, error handling, and scoped resource finalization.
Examples
Uses file temperature conversion, line counting, infinite integers, normalization pipes, and resource-managed file reading and writing.
What you can do after
Build it from scratch with Pull, Stream, Pipe, effects, error handlers, and resource finalizers.

Read how Functional Programming in Scala, Second Edition explains stream processing →

The Joy of JavaScript

A stream is a sequence of one or more values or events that can be consumed incrementally over time.

How it is used
Unifies incremental processing of collections, file chunks, and recurring asynchronous events under a sequence-of-values model.
How deep it goes
Works through async file iteration, a push-capable array, custom Observables, and composed operators; treats Observable as a proposal.
Examples
Uses chunked blockchain-file validation, an event-emitting array, periodic random numbers, and number-transforming Observable pipelines.
What you can do after
Build from scratch a small Observable operator pipeline, or apply async iteration to process and validate file blocks incrementally.

Read how The Joy of JavaScript explains stream processing →

Apache Kafka in Action

Stream processing is the continuous computation of results over records as they arrive.

How it is used
Enables near-real-time analysis and responses to changes in data flowing through Kafka.
How deep it goes
Explains event-at-a-time processing and its contrast with batch results, while deferring implementation details.
Examples
Uses an online store’s product-price changes, customer segments, website updates, notifications, and sales alerts.
What you can do after
Evaluate/understand when near-real-time event processing suits a business decision better than scheduled batch processing.

Read how Apache Kafka in Action explains stream processing →

Build AI-Enhanced Web Apps

Streaming behavior is the incremental delivery or processing of output as portions become available.

How it is used
Uses streaming to make AI chat responses visible during generation and reduce perceived waiting time.
How deep it goes
Explains the client-server flow, asynchronous chunks, buffering, and error handling, then implements streaming through SDK utilities and hooks.
Examples
Demonstrates Gemini text streaming, a React chat API and client, and a weather-data chain updating a streamable UI.
What you can do after
Apply streaming via the Vercel AI SDK to deliver incremental model output in a React chat interface.

Read how Build AI-Enhanced Web Apps explains stream processing →

100 Java Mistakes and How to Avoid Them

The Java Stream API provides pipelines for processing sequences of elements through intermediate and terminal operations.

How it is used
Processes sequences through fluent transformation and result-producing pipelines while replacing some manual loops and index handling.
How deep it goes
Explains lazy execution, side-effect risks, single consumption, and reduce contracts, including sequential-versus-parallel failure modes.
Examples
Demonstrates predicate partitioning, conditional skip, repeated consumption, hash-code reduction, and inclusive numeric ranges.
What you can do after
Apply it via Java Stream API pipelines while avoiding side effects, reuse, incompatible reductions, and unsupported null handling.

Read how 100 Java Mistakes and How to Avoid Them explains stream processing →

Kafka for Architects

An event stream is a continuously available sequence of records representing events produced by a system.

How it is used
Positions continuous event transformation between Kafka producers and downstream consumers when low-latency results matter.
How deep it goes
Defines continuous processing and explains routing, enrichment, and state at an architectural level, without detailing streaming-programming implementations.
Examples
Uses address-based routing and order enrichment with customer data to illustrate event transformation.
What you can do after
Evaluate/understand it as a low-latency processing layer and choose where filtering, joins, and aggregation belong.

Read how Kafka for Architects explains stream processing →

LLMs in Production

Streaming is the incremental retrieval and processing of a dataset without loading the entire dataset into memory.

How it is used
Uses streaming to prepare large training datasets and deliver LLM responses incrementally.
How deep it goes
Shows dataset loading and response-streaming examples, while noting that streaming services involve broader complexity.
Examples
Uses TinyStories training data and a simple chat application that displays generated responses incrementally.
What you can do after
Apply it via Hugging Face dataset loading and a client-side readable stream.

Read how LLMs in Production explains stream processing →

Effective Platform Engineering

An event stream is an ordered, continuously produced sequence of events that can be consumed by software systems and is not modified after creation.

How it is used
Coordinates team-related platform integrations and helps scale services as platform adoption grows.
How deep it goes
Explains consumers, asynchronous benefits, and configuration reconciliation, but does not specify stream implementation or delivery guarantees.
Examples
Uses team sync events to illustrate namespace configuration and integrations with Terraform Cloud, Jira, and observability tools.
What you can do after
Apply it by publishing platform events and assigning configuration responsibilities to independent consumers.

Read how Effective Platform Engineering explains stream processing →

Machine Learning Platform Engineering

Response streaming delivers generated output incrementally as it becomes available instead of waiting for the complete response.

How it is used
Deploys incremental responses in a Chainlit RAG chat interface so users see progress while an answer is generated.
How deep it goes
Shows an async generator yielding sources and model chunks, then integrates streaming into a validated chat handler.
Examples
Uses a documentation-answering RAG assistant with a Chainlit interface, source events, and token events.
What you can do after
Apply it via Chainlit by connecting an async RAG response generator to a validated chat handler.

Read how Machine Learning Platform Engineering explains stream processing →

Go in Practice, Second Edition

An event stream is an ordered flow of discrete event records delivered continuously over time.

How it is used
Processes large files, uploads, and log messages incrementally rather than waiting for all input.
How deep it goes
Works through line scanning, buffered copying, and multipart reading; discusses network reliability and response-buffering trade-offs.
Examples
Uses newline-separated JSON logs, file-to-file copying, multipart file uploads, network log messages, and template responses.
What you can do after
Build it from scratch with bufio.Scanner or MultipartReader, or apply it via io.Copy to move data incrementally.

Read how Go in Practice, Second Edition explains stream processing →

The Art of Code

Streaming is a processing technique that handles data incrementally as it becomes available instead of retaining the entire result in memory.

How it is used
Addresses export memory pressure while also using Java streams to express transformations declaratively.
How deep it goes
Traces batch fetching, disk writes, and memory release in an export, without specifying the export implementation.
Examples
Uses an Excel export and a Java pipeline that filters, uppercases, and collects words.
What you can do after
Evaluate/understand streaming and pipeline choices against memory pressure, readability, traversal costs, and parallel overhead.

Read how The Art of Code explains stream processing →

Modern C, Third Edition

A stream is an abstract C input or output sequence associated with a file, device, or other data source or destination.

How it is used
Routes C library input and output through streams associated with files or standard destinations.
How deep it goes
Defines the opaque interface and demonstrates output and file modes, while leaving stream implementation details unexplored.
Examples
Uses character and string output, an append-mode log file, and formatted output directed to a stream.
What you can do after
Operate file streams by choosing access modes, checking whether opening succeeded, and directing output to a stream.

Read how Modern C, Third Edition explains stream processing →

Build Applications with Local AI Models on a Mac

Delivery of a generated response incrementally as its text becomes available, rather than all at once.

How it is used
Makes local chatbot responses feel responsive by displaying generated text as it arrives.
How deep it goes
Explains chunk iteration, immediate printing, and the flow between a Python script, Ollama server, and model.
Examples
Uses an internet-explanation prompt in a Python script and a Streamlit chat interface with an updating placeholder.
What you can do after
Apply it via Ollama’s Python library to display incremental model responses.

Read how Build Applications with Local AI Models on a Mac explains stream processing →

100 C++ Mistakes and How to Avoid Them

A stream is an interface that represents a sequential flow of data between a program and an input or output source.

How it is used
Replaces risky formatted C-style I/O and reduces housekeeping when processing container sequences.
How deep it goes
Explains stream operators, input failure checks, and transform-filter-sum operations without examining implementation internals.
Examples
Demonstrates parsing numeric and text values, and transforming, filtering, and summing integers.
What you can do after
Apply C++ stream operators for typed I/O and ranges and views for sequence transformations.

Read how 100 C++ Mistakes and How to Avoid Them explains stream processing →

Deep Learning with PyTorch, Second Edition

Sending generated output incrementally to a client as it becomes available rather than waiting for the full result to finish.

How it is used
Reduces response latency by delivering generated text incrementally to clients.
How deep it goes
Explains asynchronous token delivery alongside request queues, batching, and a background model worker.
Examples
Uses generated text tokens and a restaurant analogy of batched orders served as dishes become ready.
What you can do after
Build a FastAPI text-generation endpoint that streams generated tokens to clients.

Read how Deep Learning with PyTorch, Second Edition explains stream processing →

Digital Twins in Action

A data stream is a continuously or repeatedly produced sequence of data records from one or more sources.

How it is used
Keeps digital twins current by ingesting and processing continuously arriving data from physical systems.
How deep it goes
Explains sensor decoding, sliding-window updates, and video transmission choices, with illustrative processing code.
Examples
Uses temperature readings, live train-movement events, and webcam object detection to illustrate streams.
What you can do after
Evaluate stream-ingestion and video-transmission choices for a digital twin.

Read how Digital Twins in Action explains stream processing →

.NET in Action, Second Edition

Stream is a .NET abstraction for reading and writing sequences of bytes across sources such as files, network responses, memory, and console I/O.

How it is used
Uses streams to read and write data across files, network responses, memory, and console output.
How deep it goes
Demonstrates file and console stream setup, JSON parsing, and testing; defers append and seek use cases.
Examples
Uses JSON file processing, line-by-line text search, and a MemoryStream-backed unit test.
What you can do after
Apply .NET Stream APIs to process file or network data and supply in-memory test input.

Read how .NET in Action, Second Edition explains stream processing →

Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform

Node.js streaming is the incremental processing and transfer of data through Node.js stream objects rather than loading the complete data set at once.

How it is used
Delivers video through HTTP responses in a microservices application.
How deep it goes
Explains readable streams and piping, then stops short of stream internals.
Examples
Uses a local video file, Azure Storage, forwarding services, and a browser video element.
What you can do after
Build from scratch a basic Express endpoint that streams a local video file to an HTTP response.

Read how Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform explains stream processing →

Modern C

How it is used
Directs character and string output to terminal defaults or file-backed streams.
How deep it goes
Explains FILE* as opaque, demonstrates output and file opening, and defers pointer details and text-binary distinctions.
Examples
Demonstrates a manual puts implementation and writing a log file through fopen and fputs.
What you can do after
Apply fputs and fputc to output through a stream, or open a file and check whether opening succeeded.

Read how Modern C explains stream processing →

ASP.NET Core in Action, 3e

File streaming is the technique of processing file content incrementally as it is transferred instead of storing the entire file before processing.

How it is used
Processes large request or upload data incrementally to avoid holding the entire file in memory at once.
How deep it goes
Defines request-body Stream and contrasts direct streaming with IFormFile buffering, but defers implementation details.
Examples
Contrasts buffered IFormFile uploads with direct streaming for large files.
What you can do after
Evaluate when to avoid IFormFile for large uploads and consider direct streaming instead.

Read how ASP.NET Core in Action, 3e explains stream processing →

Related concepts

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.