Streaming processes a sequence incrementally: each available element can be transformed, emitted, or folded into a result without first loading the whole sequence. This matters for large files, infinite inputs, and external sources where a List-like whole-input API may consume too much memory. The goal is not merely a loop that reads one line at a time, but a composable description of that loop: add filtering, conversion, counting, or a moving average as separate transformations. section 13.8 section 15.6
A Stream describes outputs; a Pull describes the lower-level computation that produces outputs and eventually returns a result. Pull can be stepped incrementally, yielding either a completed result or one output plus the remaining Pull. Stream presents a collection-like API over the outputs, while Pull retains control over sequencing and termination. This separation makes finite, infinite, and partially evaluated computations fit the same model. section 15.2 section 15.6
Streaming transformations stay useful when they are independent values that can be combined:
- Use map, filter, take, and folds to transform or consume outputs one at a time; take enables partial evaluation of an otherwise infinite computation. section 15.2.1
- Represent a reusable transformation as Pipe[F, I, O], a function from an input Stream to an output Stream, then compose pipes with andThen. section 15.2.2 section 15.3
- Lift effects into a Stream with eval or mapEval, so effectful acquisition or per-element work remains explicit in the computation. section 15.3
- Handle failures within the stream and attach completion behavior when resources or cleanup are involved. section 15.3.2
A file conversion pipeline
A temperature converter can read input lines, trim them, discard blank and comment lines, parse valid numbers, convert Fahrenheit values to Celsius, and write outputs. Each stage is a small Pipe, composed into one conversion pipeline. This replaces a specialized monolithic file loop with transformations that can be inspected, reused, and rearranged independently. section 15.3.3 section 15.6
Streaming must also account for termination. A producer should release resources when it is exhausted or fails; a consumer that stops early, such as take(5), must also trigger cleanup for the discarded remainder. Resource scopes and finalizers make this rule part of stream construction rather than an obligation for every terminal operation. section 15.3.3 section 15.6
def scope: Stream[F, O] = Pull.OpenScope(self, None) def take(n: Int): Stream[F, O] = self.take(n).void.scope
Inspect how take opens a scope so early termination still runs resource cleanup. section 15.3.3
The pattern extends beyond files: messages, state machines, actors, HTTP requests and responses, UI events, and distributed processing can all be viewed as streams. A production-oriented library may add capabilities such as concurrency, fan-in–fan-out, interruption, and richer effect support, but the central design remains incremental, compositional computation over outputs. section 15.4 section 15.6
