liveBook · Manning

Transform stream

As explained in JavaScript in Depth

A Web Streams API object with writable and readable sides for transforming streamed chunks.

A transform stream sits between a producer and a consumer, changing chunks while data is still flowing. In the Web Streams API, a TransformStream connects a writable side to a readable side through a transform function. Converting text from one encoding to another is one use. section 10.1 section 10.4.3

The two sides have roughly the same dual-buffer structure as a Node.js transform. If the readable queue fills, backpressure reaches the writable side, which stops accepting data until downstream catches up. This only protects the pipeline when callers respect the signal: application code can still push data in ways that bypass the normal pace. section 10.4.3 section 10.6.2

The failure can be cumulative. Each transform adds places for chunks to wait; several stages can turn ignored backpressure into growing buffers, garbage-collection pressure, and eventual memory exhaustion. Use pipeThrough() and pipeTo() rather than manually forwarding chunks: built-in piping carries backpressure and errors through the pipeline and handles cleanup. section 10.6.2 section 10.8

// Web Streams
const transform = new TransformStream({
  async transform(chunk, controller) {
    const result = processChunk(chunk);  // #A
    controller.enqueue(result);
  }
});

The transform callback processes a chunk and enqueues its result. Marking this callback async when processChunk() is synchronous adds promise work without making the computation asynchronous. section 10.7.1

TransformStream is part of the WHATWG Streams Standard, which specifies observable behavior rather than a runtime’s allocation strategy. Implementations can therefore have different allocation profiles. Web Streams are the portable fit for data already arriving through fetch(); they do not erase the per-chunk promise costs of their architecture. section 10.2 section 10.4 section 10.7.1

How JavaScript in Depth covers it

How deep it goes
Explains the writable-to-readable connection, dual buffering, backpressure propagation, and promise costs without prescribing runtime-specific allocation internals.
Examples
Uses text-encoding conversion and an async transform that enqueues the result of synchronous chunk processing.
What you can do after
Apply it via pipeThrough() and pipeTo() while respecting backpressure and avoiding unnecessary async transform callbacks.
How it connects
Connects TransformStream to WritableStream, ReadableStream, backpressure, pipeThrough(), and pipeTo().

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.