liveBook · Manning

Asynchronous buffer

As explained in Clojure: The Essential Reference

An asynchronous buffer is a storage structure filled or drained by computation running independently of its consumer or producer.

An asynchronous buffer decouples a producer sequence from its consumer by computing in the background and temporarily storing elements in memory. Clojure's seque uses one to coordinate producers and consumers operating at different speeds, such as device input, distributed services, or intensive computation. section 9.1.4

Use seque with these contracts and behaviors:

  • Call (seque s) or (seque n-or-q s); the default capacity is 100 items. section 9.1.4
  • Supply a positive integer capacity or an ordered, finite BlockingQueue that accepts nil; LinkedBlockingQueue is recommended. section 9.1.4
  • Receive the elements of s as a sequence, with an empty result for empty or nil input; zero or negative capacity throws IllegalArgumentException. section 9.1.4

Let a fast producer run ahead

(slow-consumer (seque (fast-producer 5))) ;; produce 4 ;; produce 3 ;; produce 2 ;; produce 1 ;; produce 0 ;; consume 4 ;; consume 3 ;; consume 2 ;; consume 1 ;; consume 0 The producer fills the in-memory buffer while the consumer sleeps. section 9.1.4

Listing 9.4. → Sequence generation, Buffered sequence
(seque
  ([s])
  ([n-or-q s]))

Inspect the compact seque signatures before applying the buffering pattern. section 9.1.4

How Clojure: The Essential Reference covers it

How deep it goes
Explains buffering, capacities, queue requirements, exceptions, and producer-consumer timing, but not asynchronous implementation internals.
Examples
Demonstrates fast and slow producers or consumers, filesystem scanning, bounded queues, and sentinel-based custom buffering.
What it assumes
Assumes familiarity with Clojure sequences and introduces queue capacity and BlockingQueue requirements through examples.
What you can do after
Apply it via seque to reduce producer waiting when consumers process sequences at a different speed.
How it connects
Connects sequences with lazy production, consumers, producers, futures, and Java BlockingQueue implementations.

This explainer was generated by Manning's knowledge graph from the text of Clojure: The Essential Reference. The section links point to the parts of the book it draws on.