How deeply does Clojure: The Essential Reference cover work stealing?

Explains chunk splitting and idle-worker stealing, but does not detail the underlying scheduling algorithm.

Work stealing

As explained in Clojure: The Essential Reference

Work-stealing is a parallel scheduling technique in which an idle worker takes unprocessed tasks from another worker's queue.

Work stealing helps keep workers busy during a parallel fold. A fold splits a supported collection into chunks, reduces the chunks in parallel, and combines their results. Workers may split chunks further; when a worker becomes free, it can take unfinished work from another worker. section 7.1.1

This matters when work takes unpredictable amounts of time: one busy worker need not leave others idle. It improves on basic thread pooling in that situation, but parallelism is not automatically faster for small jobs. section 7.1.1

A fold with parallel work available

After `(require '[clojure.core.reducers :as r])`, `(r/fold + (into [] (range 1000000)))` returns `499999500000`. For a 1,000-element vector, the supplied benchmark instead found `r/fold` slower than `reduce`; use the extra scheduling work where the workload warrants it. section 7.1.1

How Clojure: The Essential Reference covers it

Cover of Clojure: The Essential Reference

liveBook manning.com

Read Clojure: The Essential Reference on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Learn Concurrent Programming with GoWork stealing is a scheduling technique in which an execution worker takes tasks from another worker's queue to improve utilization.
liveBook manning.com

Learn Haskell by ExampleWork stealing is a scheduling technique in which an idle execution worker takes available work from a shared pool.
liveBook manning.com

C++ Concurrency in Action, Second EditionWork stealing is a scheduling technique in which an idle worker takes tasks from another worker’s queue.
liveBook manning.com

100 Go Mistakes and How to Avoid ThemWork stealing is a scheduling technique in which an underutilized processor obtains runnable tasks from another processor's queue.

Compare how 5 books explain work stealing →

Related concepts

green threadused by · A green thread is a lightweight user-space execution unit whose scheduling and context switching are managed by a runtime and multiplexed onto operating-system threads.
reduction operationused by · A reduction operation combines multiple input values or partial results according to a combining rule to produce fewer results, often a single aggregate.
parallel streamused by · A parallel stream is a Java stream configured to allow its operations to process elements concurrently across multiple execution threads.
parallelismimplements · Parallelism is the concurrent execution of multiple computational operations, often across multiple processing resources.
work queue patternuses · The work queue pattern stores pending units of work for workers or processors to retrieve and process, often asynchronously.

This explainer was generated by Manning's knowledge graph from the text of Clojure: The Essential Reference. The section links point to the parts of the book it draws on.