liveBook · Manning

Streaming

As explained in The Joy of JavaScript

A stream is a sequence of one or more values or events that can be consumed incrementally over time.

A stream is a sequence of values emitted over time. Each value is an event, whether it is a mouse movement, a file chunk, or an item from an array. Processing values as they arrive lets the same broad model serve finite collections and ongoing sources without requiring every value at once. section 9.3.1 section 9.2.3

A promise delivers one eventual result; obtaining another requires another request. A stream fits a chat feed, WebSocket, or UI event source that may deliver many results. An iterator is pull-oriented: the consumer requests the next value. In a push arrangement, a subscribed producer notifies the subscriber when a value becomes available. This difference affects who controls delivery, not whether the values are synchronous or asynchronous. section 8.6 section 9.3.1

Figure 9.2 The difference between pull and push. With pull, the consumer must always initiate a request for data. With push, when the consumer subscribes, the producer sends data as it becomes available, stopping when the consumer unsubscribes or when there’s no more data to send.

The pull/push diagram makes the change in delivery control visible: repeated consumer requests on one side, producer notifications after subscription on the other. section 9.3.1

Read a file a block at a time

A Node.js read stream can be consumed with `for await (const chunk of dataStream)`. An async generator can retain an unfinished fragment between chunks, parse complete semicolon-delimited JSON blocks, and `yield` each block. A second loop can count or validate those blocks as they arrive. This avoids reading the whole file before processing it and keeps parsing separate from the work done with each block. section 9.2.3 section 9.3.2

For a subscriber-driven pipeline, an Observable represents a source independently of its eventual subscriber. `subscribe({ next, error, complete })` starts a lazy Observable; unsubscribing can stop an ongoing source. Operators such as `map` and `filter` subscribe to an upstream source and produce another Observable, so transformations can be assembled before any data flows. The Observable API described here was a proposal, while the operator examples build a small toolkit rather than relying on operators defined by that proposal. section 9.4 section 9.4.3

Turn events into a result

`Observable.of(1, 2, 3, 4)` can feed a pipeline that skips an item, filters even values, squares them, and reduces them to a total. The same operator style can process blocks from an async generator: skip the initial block, add and validate each subsequent block, retain successes, and count them. The source and transformations remain distinct from the final subscriber. section 9.4.3 section 9.4.5

How The Joy of JavaScript covers it

How deep it goes
Works through async file iteration, a push-capable array, custom Observables, and composed operators; treats Observable as a proposal.
Examples
Uses chunked blockchain-file validation, an event-emitting array, periodic random numbers, and number-transforming Observable pipelines.
What it assumes
Assumes comfort with JavaScript composability and asynchronous programming while explaining stream basics and the push/pull distinction.
What you can do after
Build from scratch a small Observable operator pipeline, or apply async iteration to process and validate file blocks incrementally.
How it connects
Connects promises, async generators, iterators, publish/subscribe, Observables, and functional composition through their ways of delivering and transforming values.

Also explained in

Compare how 19 books explain streaming →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Joy of JavaScript. The section links point to the parts of the book it draws on.