A stateful reducer carries information from one reducing-function invocation to the next. Typical implementations store local state in a volatile! or atom; deduplication and bounded consumption are representative uses. section 7.1.2
Treat stateful reducers carefully when parallelizing:
- Sequential behavior may not have an equivalent parallel meaning; dropping elements per chunk can differ from dropping them globally. section 7.1.2
- Shared state must be thread-aware, because parallel folds can invoke reducers concurrently. section 7.1.2
- Reducers can reject parallel folding unless explicitly parallel-enabled, while stateful transducers may produce inconsistent results when folded in parallel. section 7.1.2
Deduplicating adjacent values
A reducer-dedupe function keeps the previous value in a volatile!, updates it on every invocation, and forwards only values different from their predecessor. Applied to a composed reducer pipeline, it produces [1 3 1 3 1 3 5 1 3 5 1 3 5 7 1 3 5 7]. section 7.1.2
(require '[clojure.core.reducers :as r])
(defn reducer-dedupe [coll]
(r/reducer coll ; #1
(fn [rf]
(let [prev (volatile! ::none)] ; #2
(fn [acc el]
(let [v @prev]
(vreset! prev el)
(if (= v el) ; #3
acc
(rf acc el))))))))
(->> (range 10) ; #4
(r/map range)
(r/mapcat conj)
(r/filter odd?)
reducer-dedupe
(into []))
;; [1 3 1 3 1 3 5 1 3 5 1 3 5 7 1 3 5 7]
Inspect the complete reducer-dedupe implementation and its composed pipeline. section 7.1.2
