liveBook · Manning

Buffering

As explained in JavaScript in Depth

An internal data structure that temporarily stores stream data between production and consumption.

Pausing a Node.js Readable stops data events, not chunk production: chunks accumulate in its internal buffer until flow resumes. section 10.3.1

A pipeline has separate buffers for the Readable, a Transform’s inbound and outbound data, and the Writable. Each has its own highWaterMark. Exceeding one prompts an advisory signal for upstream stages to slow down; ignoring it lets chunks accumulate. section 10.3.1

Writable backpressure

When writable.write() returns false, stop writing until the 'drain' event. Continuing to write can add data to the buffer despite the signal. section 10.3.1

Figure 10.5 Data flow through a Node.js pipeline with a single Transform. Data moves downwards through four internal buffers: the Readable's buffer, the Transform's inbound buffer, the Transform's outbound buffer, and the Writable's buffer. Each buffer has its own highWaterMark. Backpressure propagates upwards: when any buffer exceeds its mark, the upstream stage is signaled to slow down. Every signal is advisory, and every buffer is a point where chunks can accumulate and survive into old generation if the signal is ignored.

Trace how data passes through four distinct buffers and how backpressure signals move upstream. section 10.3.1

Web Streams call the underlying source’s pull() when the internal queue has room, tying production to the consumer’s pace. Enqueuing outside that pull cycle can bypass this protection. section 10.4.1 section 10.9

How JavaScript in Depth covers it

How deep it goes
Explains paused-stream accumulation, per-stage highWaterMark thresholds, and how ignored signals let buffers grow.
Examples
Uses paused Readables, Writable write() and drain signals, multi-buffer pipelines, and Web Streams pull cycles.
What you can do after
Diagnose chunk accumulation when a Readable is paused or a producer ignores backpressure signals.
How it connects
Connects buffering to Readable and Writable streams, Transform streams, backpressure, highWaterMark, and garbage collection.

Also explained in

Compare how 8 books explain buffering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.