liveBook · Manning

Spark overflow

As explained in Learn Haskell by Example

Spark overflow is the condition in which a full spark pool discards newly created sparks.

Spark overflow occurs when a fixed-size spark pool fills faster than Haskell execution contexts can consume it. Newly created sparks are then discarded, so they produce no parallel evaluation. section 13.4.3

Image mapping

A pixel-mapping strategy created 1,318,768 sparks, but only 10,163 were converted and 1,210,301 overflowed. The program therefore failed to compute the mapping in parallel. Grouping pixels into larger chunks reduced the count to 1,001 sparks, all converted and none overflowed. section 13.4.3

Avoid overflow by creating fewer sparks while giving each spark more work. parListChunk provides chunked list evaluation; parVectorChunk adapts it by converting a Vector to a list, applying rseq to each chunk, and converting the result back. section 13.4.3

How Learn Haskell by Example covers it

How deep it goes
Explains fixed pool capacity, discarded sparks, runtime statistics, and chunking, but leaves chunk-size selection inexact.
Examples
Uses per-pixel image mapping, runtime SPARKS statistics, and a Vector chunking module as concrete examples.
What it assumes
Assumes familiarity with Haskell parallel Strategies, Vectors, and basic image-mapping code.
What you can do after
Diagnose parallelism failures with SPARKS statistics and apply chunked Strategies to reduce discarded work.
How it connects
Connects spark pools with Haskell execution contexts, parallel evaluation, Strategies, Vectors, and chunked computation.

This explainer was generated by Manning's knowledge graph from the text of Learn Haskell by Example. The section links point to the parts of the book it draws on.