liveBook · Manning

High-water mark

As explained in JavaScript in Depth

A stream buffering threshold that controls how much data may be queued before backpressure is applied.

A high-water mark is a stream buffering threshold, not a hard memory limit. When a buffer reaches it, the stream signals the producer to slow down. Whether that signal protects memory depends on the stream model and whether the producer respects it. section 10.3.1 section 10.4.2

In Node.js, push() and write() return true while their buffers have room and false after the high-water mark is exceeded. The producer must stop, wait for drain, or otherwise coordinate with the consumer; nothing enforces the limit. Ignoring the signal lets chunks accumulate, increasing heap use, garbage-collection work, and eventually the risk of an out-of-memory crash. section 10.3.1 section 10.3.2

A pipeline multiplies the places where buffering can occur. A Readable, a Transform's inbound and outbound sides, and a Writable each have a high-water mark and a backpressure signal. Built-in piping preserves this flow more reliably than manually pushing data, while pipeline() additionally propagates errors and destroys streams on failure. section 10.3.3 section 10.3.2

The same idea has different enforcement mechanisms:

  • Node.js exposes an advisory boolean from push() and write(); producers can ignore it. section 10.3.1
  • Web Streams expose desiredSize: positive means room remains, zero means capacity, and negative means the queue has exceeded its mark. section 10.4.2
  • Web Streams normally stop calling pull() when desiredSize is not positive, but external controller.enqueue() calls can bypass that protection. section 10.4.2
  • CountQueuingStrategy measures queued values, whereas ByteQueuingStrategy measures total byte size. section 10.4.2

A producer that must stop

For a Writable, treat a false return from write() as a pause point: wait for the drain event before resuming writes. In a Web Stream fed by an event source, inspect desiredSize and pause the source when it reaches zero or below; resume it from pull(). section 10.3.2 section 10.4.2

Figure 10.5 Data flow through a Node.js pipeline with a single Transform. Data moves downwards through four internal buffers: the Readable's buffer, the Transform's inbound buffer, the Transform's outbound buffer, and the Writable's buffer. Each buffer has its own highWaterMark. Backpressure propagates upwards: when any buffer exceeds its mark, the upstream stage is signaled to slow down. Every signal is advisory, and every buffer is a point where chunks can accumulate and survive into old generation if the signal is ignored.

Trace how high-water marks and advisory backpressure propagate through the four buffers of a Transform pipeline. section 10.3.3

How JavaScript in Depth covers it

How deep it goes
Explains buffer thresholds, producer signals, pipeline accumulation, and runtime-specific enforcement, stopping short of implementation internals.
Examples
Uses Node.js push(), write(), drain, Transform pipelines, desiredSize, pull(), and externally fed Web Streams.
What it assumes
Assumes familiarity with streams and consumers, while explaining buffering, backpressure, and producer coordination directly.
What you can do after
Diagnose/debug with it by locating accumulating buffers and respecting push(), write(), drain, desiredSize, and pull() signals.
How it connects
Connects high-water marks to backpressure, streams, pipelines, transforms, buffering, and garbage collection.

Also explained in

Compare how 3 books explain high-water mark →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.