CollFold lets a collection supply the partitioned reduction used by clojure.core.reducers/fold. It matters when a collection can be reduced but has no useful parallel fold: adding key-value reduction to a ConcurrentHashMap removes an error, yet still leaves the work sequential. section 7.1.1
An implementation supplies coll-fold with four inputs:
- The collection and a partition-size argument n. section 7.1.1
- A combining function for partial results and a reducing function for work within partitions. section 7.1.1
Fold a ConcurrentHashMap by partitioning its keys
The demonstrated extension copies the keys into a vector and folds that vector, letting workers process subsets of the original map: (require '[clojure.core.reducers :as r]) (import 'java.util.concurrent.ConcurrentHashMap) (extend-protocol r/CollFold ConcurrentHashMap (coll-fold [m n combinef reducef] (#'r/foldvec (vec (keys m)) n combinef reducef))) Here reducef receives the accumulating value and a key. This particular implementation calls #'r/foldvec directly; that is a private reducers function. In the reported workload, adding only key-value reduction took about 41 seconds, while the CollFold extension took about 0.43 seconds. Those timings describe that workload, not a general speedup guarantee. section 7.1.1
Parallelism is not automatically faster: folding a 1,000-element vector took longer than reduce in the reported benchmark. For lazy processing of predictable tasks, pmap is the suggested alternative; fold is favored when uneven work makes its fork-join scheduling useful. section 7.1.1
