liveBook · Manning

Backpressure

As explained in JavaScript in Depth

A flow-control mechanism in which a consumer signals a producer to slow or stop producing when buffered data exceeds a manageable level.

A fast producer feeding a slow consumer can turn a stream into a growing memory buffer. Backpressure coordinates their rates: when downstream cannot keep up, upstream must slow or stop. When that coordination fails, queued chunks increase heap use, add garbage-collection work, and can eventually exhaust memory. section 10.3.1 section 10.6.2

The three JavaScript streaming models control production differently:

  • Node.js streams push chunks. A Readable's push() and a Writable's write() return false when their buffers exceed a highWaterMark. These are advisory signals: producers must stop. A Writable's 'drain' event signals when writing can resume. section 10.3.1
  • Web Streams normally pull. The runtime calls pull() when the queue has room; controller.desiredSize reports remaining capacity, with zero or less meaning stop. Capturing a controller and calling enqueue() from an independent event feed bypasses that protection. section 10.4.1 section 10.4.2
  • Async generators pause at yield until the consumer requests another value with next(). They need no explicit backpressure signal, but cannot buffer ahead within the generator. section 10.5.1 section 10.5.3

Backpressure must travel backward through every stage. A Readable–Transform–Writable pipeline has at least four buffers: the Readable's, the Transform's inbound and outbound buffers, and the Writable's. Each has its own highWaterMark. One stage that ignores its signal can let chunks accumulate upstream. section 10.3.3 section 10.6.2

When a push feed meets a pull stream

Suppose messages arrive from an event feed whether or not a ReadableStream consumer is ready. Forwarding every message through a captured controller may look like streaming, but it lets the queue grow when consumption slows. If the feed supports pause and resume, pause it when desiredSize reaches zero, then resume it from pull(); cancel should release the feed. section 10.4.2

function streamFromFeed(feed) {
  return new ReadableStream({
    start(controller) {
      feed.on('message', (msg) => {
        controller.enqueue(msg);
        if (controller.desiredSize <= 0) feed.pause(); // #A
      });
      feed.on('close', () => controller.close());
    },
    pull() {
      feed.resume(); // #B
    },
    cancel() { feed.destroy(); } // #C
  });
}

The adapter connects queue capacity to the feed's pause and resume controls instead of enqueueing indefinitely. section 10.4.2

Figure 10.9 Resident memory for a fast producer feeding a slow consumer on Node.js v24. Honoring backpressure holds memory flat with about one chunk in flight; ignoring it lets the queue grow without bound until memory climbs. The curve shown is a Web Stream with an extracted controller. A Node.js pipeline that ignores the write() return value produces the same runaway. Absolute megabytes vary by machine; the shape does not.

Compare flat memory when production is paced with rising memory when an extracted controller ignores backpressure; the plotted amounts are machine-dependent. section 10.6.3

Prefer built-in pipeline coordination: stream.pipeline() handles Node.js backpressure, errors, and cleanup; Web Streams' pipeTo() and pipeThrough() do the same. Their mechanics still cost something per chunk: Node.js uses event dispatch, while Web Streams and async iterables incur promise work. For high-throughput synchronous data, consider that cost—but preventing unbounded buffering matters more. section 10.3.2 section 10.4.4 section 10.6.3

How JavaScript in Depth covers it

How deep it goes
Traces signals and queue behavior across three models, including transform buffers, bypasses, memory failure, and per-chunk costs.
Examples
Uses a pausable event-feed adapter and a fast-producer/slow-consumer memory comparison to expose broken flow control.
What it assumes
Builds on familiarity with JavaScript callbacks, promises, and incremental data flow while explaining each model's backpressure mechanism.
What you can do after
Diagnose/debug with it by locating ignored signals or bypassed pull cycles, then use built-in pipelines or pace the source.
How it connects
Connects highWaterMark, TransformStream, async iteration, promise scheduling, and garbage collection to pipeline flow control.

Also explained in

Compare how 10 books explain backpressure →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.