liveBook · Manning

ReadableStream

As explained in Secrets of the JavaScript Ninja, Third Edition

A Web Streams API interface representing a source of data that consumers can read incrementally.

An HTTP response need not arrive all at once. A fetch response body is a ReadableStream: a source of chunks that can be consumed incrementally with `for await…of`. The stream is read-only to its consumer; its producer controls what becomes available. section 11.2.1

Listing 11.5 Creating a ReadableStream
import { setTimeout } from "node:timers/promises";
 
const ninjas = [
  "Leonardo",
  "Donatello",
  "Raphael",
  "Michelangelo",
];
 
const source = new ReadableStream<string>({
  async start(controller) { #A
    for (const ninja of ninjas) { #A
      controller.enqueue(ninja); #A
      await setTimeout(100); #A
    } #A
    controller.close(); #A
  },
});

const received: string[] = []; #B
for await (const ninja of source) { #B
  received.push(ninja); #B
} #B

console.assert(
  received.length === 4,
  "Four ninjas were received",
);

A custom source enqueues strings in `start`; the loop receives them as they arrive. section 11.2.1

The `start` method runs immediately and can enqueue values regardless of whether anyone is reading. That makes this a push stream. Enqueued values wait in the stream’s internal queue until consumed. Call `controller.close()` when production ends: otherwise, the async loop can keep waiting for another value. section 11.2.1

Listing 11.6 Piping a stream
const ninjaKanji = "忍者"; #A
 
const bytes = new TextEncoder().encode(ninjaKanji); #B
const splitAt = Math.floor( #C
  Math.random() * bytes.length, #C
); #C
const chunk1 = bytes.subarray(0, splitAt); #C
const chunk2 = bytes.subarray(splitAt); #C
 
const source = new ReadableStream({ #D
  start(controller) { #D
    controller.enqueue(chunk1); #D
    controller.enqueue(chunk2); #D
    controller.close(); #D
  }, #D
}); #D
 
const received: string[] = []; #E
const sink = new WritableStream<string>({ #E
  write(text) { #E
    received.push(text); #E
  }, #E
}); #E

await source #F
  .pipeThrough(new TextDecoderStream()) #F
  .pipeTo(sink); #F

console.assert(
  received.join("") === ninjaKanji,
  "TextDecoderStream correctly decodes across chunk boundaries",
);

Byte chunks pass through a decoder stream before a writable sink collects the resulting strings. section 11.2.2

A ReadableStream can also be the source of a pipeline. `pipeThrough()` passes its output through a transformation, while `pipeTo()` sends output to a WritableStream sink. Chunk boundaries need not match output boundaries: a TextDecoderStream can combine bytes from separate chunks into decoded characters. section 11.2.2

Listing 11.8 Pulling calculations from a stream
import { setTimeout } from "node:timers/promises";
 
function createFibonacciStream(count: number) {
  let a = 0;
  let b = 1;
  let remaining = count;
  return new ReadableStream<number>({ #A
    pull(controller) { #A
      controller.enqueue(a); #A
      [a, b] = [b, a + b]; #A
      if (--remaining === 0) controller.close(); #A
    }, #A
  }); #A
}

const first10: number[] = [];
for await (const n of createFibonacciStream(10)) { #B
  first10.push(n); #B
  await setTimeout(0); #B
} #B

console.assert(
  first10.join(", ") ===
    "0, 1, 1, 2, 3, 5, 8, 13, 21, 34",
  "First 10 Fibonacci numbers",
);

A Fibonacci source uses `pull` to produce numbers as the stream requests more data. section 11.3.1

Use `pull` when production should respond to demand rather than begin in `start`. Consumers read push and pull streams the same way, but the choice can affect performance and memory use. Neither label guarantees an empty queue: pull streams can buffer results too. section 11.3.1

How Secrets of the JavaScript Ninja, Third Edition covers it

How deep it goes
Works through construction, async iteration, closure, piping, and push-versus-pull behavior, including queueing and buffering.
Examples
Uses a delayed string source, byte chunks decoded across boundaries, and a pull-driven Fibonacci sequence.
What it assumes
Assumes familiarity with `for await…of` and fetch while showing how to construct a ReadableStream.
What you can do after
Build it from scratch with `start` or `pull`, consume it asynchronously, and pipe its output through a transform into a sink.
How it connects
Connects async iterators and fetch with WritableStream, TransformStream, TextDecoderStream, and push and pull streams.

Also explained in

Compare how 2 books explain ReadableStream →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Secrets of the JavaScript Ninja, Third Edition. The section links point to the parts of the book it draws on.