Streaming
Streaming is the incremental handling of a sequence of data items, including their processing, transfer, or consumption as they become available.
Functional Programming in Scala, Second Edition
Stream processing is the incremental computation of results from sequences of data as elements become available.
- How it is used
- Frames efficient external I/O as composable incremental transformations rather than monolithic loops.
- How deep it goes
- Develops Pull and Stream internals, pipes, effects, error handling, and scoped resource finalization.
- Examples
- Uses file temperature conversion, line counting, infinite integers, normalization pipes, and resource-managed file reading and writing.
- What you can do after
- Build it from scratch with Pull, Stream, Pipe, effects, error handlers, and resource finalizers.
Read how Functional Programming in Scala, Second Edition explains streaming →
The Joy of JavaScript
A stream is a sequence of one or more values or events that can be consumed incrementally over time.
- How it is used
- Unifies incremental processing of collections, file chunks, and recurring asynchronous events under a sequence-of-values model.
- How deep it goes
- Works through async file iteration, a push-capable array, custom Observables, and composed operators; treats Observable as a proposal.
- Examples
- Uses chunked blockchain-file validation, an event-emitting array, periodic random numbers, and number-transforming Observable pipelines.
- What you can do after
- Build from scratch a small Observable operator pipeline, or apply async iteration to process and validate file blocks incrementally.
Build AI-Enhanced Web Apps
Streaming behavior is the incremental delivery or processing of output as portions become available.
- How it is used
- Improves perceived responsiveness in AI chat by displaying generated content before the full response is ready.
- How deep it goes
- Explains client-to-server chunk delivery and implements streaming with SDK utilities and React hooks, while delegating connection and buffering details.
- Examples
- Uses a Gemini chat API route, a `useChat` page, a `textStream` iteration, and a LangChain.js chain updating streamed UI.
- What you can do after
- Apply it via the Vercel AI SDK to stream chat responses through a Next.js route and update a React interface incrementally.
Apache Kafka in Action
Stream processing is the continuous computation of results over records as they arrive.
- How it is used
- Enables near-real-time analysis and responses to changes in data flowing through Kafka.
- How deep it goes
- Explains event-at-a-time processing and its contrast with batch results, while deferring implementation details.
- Examples
- Uses an online store’s product-price changes, customer segments, website updates, notifications, and sales alerts.
- What you can do after
- Evaluate/understand when near-real-time event processing suits a business decision better than scheduled batch processing.
100 Java Mistakes and How to Avoid Them
The Java Stream API provides pipelines for processing sequences of elements through intermediate and terminal operations.
- How it is used
- Processes sequences through fluent transformation and result-producing pipelines while replacing some manual loops and index handling.
- How deep it goes
- Explains lazy execution, side-effect risks, single consumption, and reduce contracts, including sequential-versus-parallel failure modes.
- Examples
- Demonstrates predicate partitioning, conditional skip, repeated consumption, hash-code reduction, and inclusive numeric ranges.
- What you can do after
- Apply it via Java Stream API pipelines while avoiding side effects, reuse, incompatible reductions, and unsupported null handling.
Read how 100 Java Mistakes and How to Avoid Them explains streaming →
Kafka for Architects
An event stream is a continuously available sequence of records representing events produced by a system.
- How it is used
- Positions continuous event transformation between Kafka producers and downstream consumers when low-latency results matter.
- How deep it goes
- Defines continuous processing and explains routing, enrichment, and state at an architectural level, without detailing streaming-programming implementations.
- Examples
- Uses address-based routing and order enrichment with customer data to illustrate event transformation.
- What you can do after
- Evaluate/understand it as a low-latency processing layer and choose where filtering, joins, and aggregation belong.
LLMs in Production
Streaming is the incremental retrieval and processing of a dataset without loading the entire dataset into memory.
- How it is used
- Uses streaming to load oversized training data incrementally and deliver generated LLM text before the full response is ready.
- How deep it goes
- Works through dataset-loading and server-to-client response examples, while leaving streaming-service infrastructure largely unexplored.
- Examples
- Shows TinyStories loaded with streaming=True, a FastAPI streaming service, and a JavaScript chat client that displays arriving text.
- What you can do after
- Apply it via Hugging Face dataset loading or FastAPI StreamingResponse and a JavaScript readable stream.
Effective Platform Engineering
An event stream is an ordered, continuously produced sequence of events that can be consumed by software systems and is not modified after creation.
- How it is used
- Coordinates team-related platform integrations and helps scale services as platform adoption grows.
- How deep it goes
- Explains consumers, asynchronous benefits, and configuration reconciliation, but does not specify stream implementation or delivery guarantees.
- Examples
- Uses team sync events to illustrate namespace configuration and integrations with Terraform Cloud, Jira, and observability tools.
- What you can do after
- Apply it by publishing platform events and assigning configuration responsibilities to independent consumers.
Read how Effective Platform Engineering explains streaming →
Machine Learning Platform Engineering
Response streaming delivers generated output incrementally as it becomes available instead of waiting for the complete response.
- How it is used
- Deploys incremental responses in a Chainlit RAG chat interface so users see progress while an answer is generated.
- How deep it goes
- Shows an async generator yielding sources and model chunks, then integrates streaming into a validated chat handler.
- Examples
- Uses a documentation-answering RAG assistant with a Chainlit interface, source events, and token events.
- What you can do after
- Apply it via Chainlit by connecting an async RAG response generator to a validated chat handler.
Read how Machine Learning Platform Engineering explains streaming →
Go in Practice, Second Edition
An event stream is an ordered flow of discrete event records delivered continuously over time.
- How it is used
- Processes large files, uploads, and log messages incrementally rather than waiting for all input.
- How deep it goes
- Works through line scanning, buffered copying, and multipart reading; discusses network reliability and response-buffering trade-offs.
- Examples
- Uses newline-separated JSON logs, file-to-file copying, multipart file uploads, network log messages, and template responses.
- What you can do after
- Build it from scratch with bufio.Scanner or MultipartReader, or apply it via io.Copy to move data incrementally.
Read how Go in Practice, Second Edition explains streaming →
The Art of Code
Streaming is a processing technique that handles data incrementally as it becomes available instead of retaining the entire result in memory.
- How it is used
- Reduces memory pressure during large concurrent exports by writing database batches directly to a temporary file.
- How deep it goes
- Explains a production-oriented batch-to-disk workflow and contrasts stream traversal, collector, and parallelization tradeoffs without lower-level runtime internals.
- Examples
- Uses word filtering and uppercasing, person age and name analysis, and a failing concurrent Excel export.
- What you can do after
- Apply it via Java streams or batch-to-disk export design when memory pressure or large-data performance warrants it.
Modern C, Third Edition
A stream is an abstract C input or output sequence associated with a file, device, or other data source or destination.
- How it is used
- Deploys streams to direct unformatted, formatted, diagnostic, terminal, and persistent-file output through common C interfaces.
- How deep it goes
- Explains FILE* abstraction, output errors, file modes, lifecycle operations, and format constraints, while deferring opaque implementation details.
- Examples
- Uses putchar and puts equivalents, an append-mode logfile, freopen redirection, and fprintf format tables.
- What you can do after
- Operate/configure C streams by opening files, selecting modes, directing output, checking failures, and closing or flushing resources.
100 C++ Mistakes and How to Avoid Them
A stream is an interface that represents a sequential flow of data between a program and an input or output source.
- How it is used
- Uses streams for type-directed character input, output, and sequential processing of container values.
- How deep it goes
- Explains insertion, extraction, conversion, failure checks, bases, and range composition without descending into stream implementation internals.
- Examples
- Demonstrates istringstream extraction, octal configuration, and transforming, filtering, and summing integer values.
- What you can do after
- Apply it via C++ stream operators, check conversion failures, configure numeric bases, and compose readable sequence operations.
Read how 100 C++ Mistakes and How to Avoid Them explains streaming →
Deep Learning with PyTorch, Second Edition
Sending generated output incrementally to a client as it becomes available rather than waiting for the full result to finish.
- How it is used
- Improves responsiveness in text-generation APIs by delivering generated tokens before the full response is ready.
- How deep it goes
- Explains token delivery through an asynchronous generator and result queues, but does not detail the response-streaming implementation.
- Examples
- Uses generated text for chatbot or writing-assistant clients and illustrates a FastAPI streaming endpoint.
- What you can do after
- Apply it via FastAPI’s StreamingResponse to send generated tokens as they become available.
Read how Deep Learning with PyTorch, Second Edition explains streaming →
Digital Twins in Action
A data stream is a continuously or repeatedly produced sequence of data records from one or more sources.
- How it is used
- Keeps a digital twin current by ingesting incoming data and updating trends or events without waiting for a batch.
- How deep it goes
- Works through sliding-window aggregation and a train-event calculation, but does not detail a general-purpose stream-processing engine.
- Examples
- Uses second-by-second temperature readings and a five-minute window over public train-movement events.
- What you can do after
- Build it from scratch: maintain a recent event window and calculate an updated operational measure as records arrive.
.NET in Action, Second Edition
Stream is a .NET abstraction for reading and writing sequences of bytes across sources such as files, network responses, memory, and console I/O.
- How it is used
- Uses streams to read and write data across files, network responses, memory, and console output.
- How deep it goes
- Demonstrates file and console stream setup, JSON parsing, and testing; defers append and seek use cases.
- Examples
- Uses JSON file processing, line-by-line text search, and a MemoryStream-backed unit test.
- What you can do after
- Apply .NET Stream APIs to process file or network data and supply in-memory test input.
Read how .NET in Action, Second Edition explains streaming →
Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform
Node.js streaming is the incremental processing and transfer of data through Node.js stream objects rather than loading the complete data set at once.
- How it is used
- Delivers video through HTTP responses in a microservices application.
- How deep it goes
- Explains readable streams and piping, then stops short of stream internals.
- Examples
- Uses a local video file, Azure Storage, forwarding services, and a browser video element.
- What you can do after
- Build from scratch a basic Express endpoint that streams a local video file to an HTTP response.
Modern C
- How it is used
- Routes unformatted output to standard destinations or files through a shared stream interface.
- How deep it goes
- Explains output interfaces, file modes, and error checks while deferring FILE implementation details.
- Examples
- Demonstrates a manual puts implementation, appending a log line, and redirecting stdout to a file.
- What you can do after
- Apply C stdio functions to write output to standard streams or an opened file.
ASP.NET Core in Action, 3e
File streaming is the technique of processing file content incrementally as it is transferred instead of storing the entire file before processing.
- How it is used
- Handles large request bodies efficiently without retaining all uploaded content simultaneously.
- How deep it goes
- Defines request-body streaming and contrasts buffering, but defers implementation details for large-file streaming.
- Examples
- Uses injected Stream, IFormFile, IFormFileCollection, and OpenReadStream as upload-handling examples.
- What you can do after
- Evaluate upload approaches and avoid buffering large files through IFormFile.
Related concepts
- Apache Kafkaincludes · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.
- pull modeluses · A pull model is a pattern in which a consumer or recipient initiates requests for data, messages, configuration, or work according to its needs or readiness.
- promiseuses · A JavaScript Promise is an object representing the eventual completion or failure of an asynchronous operation and its resulting value or reason.
- batchingcontrasts with · Batching is the grouping of multiple data items, requests, or operations so they can be processed together as a unit.
- data streamuses · A data stream is an ordered sequence of data items or events made available incrementally for consumption or processing.
- WebSocketused by · WebSocket is a network protocol that enables persistent, bidirectional, full-duplex communication between endpoints over a single TCP connection.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.