liveBook · Manning

Data chunk

As explained in JavaScript in Depth

A discrete piece of data delivered during incremental stream processing.

A data chunk is the discrete unit a stream delivers for incremental processing. Instead of waiting for an entire file, response, or query result, a pipeline receives one chunk, processes it, and advances. This bounds the working set, but only while processed chunks become unreachable. section 10.0 section 10.7.2

The same chunk behaves differently across streaming architectures. Node.js streams deliver chunks through event dispatch; Web Streams request them through a pull cycle; async iteration consumes them one at a time through promises. Backpressure determines whether production follows consumer pace or queues grow. section 10.0 section 10.4.1 section 10.5.3

Treat each chunk as transient unless the algorithm genuinely needs cross-chunk state:

  • Honor backpressure: a fast producer feeding a slow consumer can keep memory nearly flat with about one chunk in flight; ignoring it allows queues to grow. section 10.6.3
  • Retain summaries rather than raw chunks when possible: counters, checksums, and hashes preserve aggregate information without keeping the data alive. section 10.7.2
  • Account for model-specific overhead: the measured 4 KB chunk allocated zero promises through a synchronous Node.js stream, eight through an async iterable, and twenty-seven through a Web Stream pipeline on Node.js v24. section 10.6.3

The hidden retention bug

A transform that pushes every incoming chunk into an array appears to process incrementally, but retained references prevent garbage collection. A closure that stores the latest chunk creates the same kind of lifetime extension; keep only the state needed for the next result. section 10.7.2

Figure 10.8 Promises allocated per 4 KB chunk on Node.js v24. Node.js streams allocate none, paying their per-chunk cost in event dispatch instead; async iterables allocate eight; a Web Stream pipeline allocates twenty-seven. Exact counts vary by runtime, but the ordering holds.

Compare the promise allocation cost of moving one 4 KB chunk through Node.js streams, async iterables, and Web Streams. section 10.6.3

How JavaScript in Depth covers it

How deep it goes
Explains delivery, buffering, promise allocation, backpressure, and garbage-collection consequences, but not chunk serialization formats.
Examples
Uses 4 KB measurements, a ReadableStream pull cycle, retained-chunk arrays, closures, and slow-consumer memory growth.
What you can do after
Diagnose chunk-related memory growth and choose processing patterns that preserve backpressure and release raw data promptly.
How it connects
Connects chunks to streams, backpressure, promises, async iterators, buffering, and garbage collection.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of JavaScript in Depth. The section links point to the parts of the book it draws on.