Stream processor
A stream processor is a continuously operating computation or component that transforms, filters, combines, generates, or routes data in a stream.
MongoDB 8.0 in Action, Third Edition
A stream processor is a continuously running aggregation pipeline that consumes streaming data from a source and sends processed output to a destination.
- How it is used
- Keeps processed views current by continuously transforming incoming events and sending results to a collection or downstream stream.
- How deep it goes
- Works through stage ordering, temporary previews, persistent processors, monitoring, and checkpoint recovery, with connection setup required for placeholder examples.
- Examples
- Uses live solar-device reports, windowed collection output, and illustrative validation, enrichment, and IoT sensor pipelines.
- What you can do after
- Build it from scratch in Atlas with mongosh, then start a named processor and inspect its statistics or sampled output.
Read how MongoDB 8.0 in Action, Third Edition explains stream processor →
Logs and Telemetry
A stream processor is a Fluent Bit component that continuously queries, aggregates, filters, and generates events from buffered event streams.
- How it is used
- Applies buffer-backed SQL-style analytics to derive, filter, and route observability events.
- How deep it goes
- Works through runnable SELECT, CREATE STREAM, aggregation, windowing, tag, and chaining configurations, while deferring dedicated streaming-platform capabilities.
- Examples
- Demonstrates random-input SELECT queries, tagged derived streams, and five-second tumbling-window threshold routing.
- What you can do after
- Apply it via Fluent Bit configuration to query buffered events, create derived streams, and route calculated results.
Apache Kafka in Action
A stream processor is a component that examines, transforms, combines, produces, or routes records in a continuously processed data stream.
- How it is used
- Turns incoming records into selected, transformed, routed, or aggregated output within continuous processing pipelines.
- How deep it goes
- Works through source, filter, mapping, branching, and counting operations, while deferring streaming SQL details.
- Examples
- Uses product price changes for threshold filtering, dollar-to-cent conversion, price-band routing, and keyed counts.
- What you can do after
- Apply it via Kafka Streams to filter prices, transform values, route branches, and count records by key.
Related concepts
- data sourcehas part · A data source is an origin, system, repository, or process from which data is obtained for use or processing.
- aggregation pipelinekind of · An aggregation pipeline is an ordered sequence of data-processing stages that filters, transforms, combines, or summarizes records.
- sink nodehas part · A sink node is a terminal or receiving component in a data or flow system that accepts incoming data, messages, or flow for consumption, processing, or storage.
- data aggregationuses · Data aggregation is the combination of multiple values, records, observations, or results into a consolidated or summarized output.
- stream processing instancepart of · A stream processing instance is an Atlas resource namespace associated with connection, deployment, and security configuration for running stream processors.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.