Chapter 5: The Reduction pattern
This introduction contrasts reduction with Map and illustrates parallel summation through partial sums and pairing rounds.
After this chapter you will be able to
- Reduction and Map contrast. Distinguish reduction from Map in terms of their results and thread cooperation.
- Chunk and subtotal strategy. Describe the two-step plan for summing the exam scores using groups and partial sums.
- Parallel pairing rounds. Trace how repeated pairing reduces a group of scores and explain how simultaneous additions shorten the sequence of rounds.
- Reduction and its operations. Describe reduction as combining a collection of values into one result, and identify examples of operations used for it.
- Associativity and grouping test. Explain associativity and apply the two-part test to distinguish addition or maximum from subtraction, noting the floating-point addition caveat.
- Familiar reduction examples. Describe sum and maximum reductions using the section’s everyday examples.
- Identity elements and padding. Explain what an identity element does and how spare GPU threads use it for safe padding.
- Reduction in Practical Workloads. Describe where reduction appears in familiar software and as a step within larger algorithms.
- Sequential reduction dependency. Explain how a running total reduces an array and why the loop cannot be straightforwardly parallelized.
- Shared counter race condition. Explain how concurrent read-modify-write updates to a shared result can lose additions and produce timing-dependent results.
- Atomic addition for correctness. Explain how replacing the ordinary shared update with atomicAdd prevents lost additions.
- Single counter performance bottleneck. Explain why an atomic addition by every thread can be correct but still slow.
- Tree reduction motivation. Explain why organizing a reduction as a tree is presented as an alternative to plain and atomic shared-counter approaches.
- Tournament reduction structure. Explain how a tournament illustrates tree reduction and how max and sum use the same tree with different combining operations.
- Stride based pairing. Trace a stride-based sum reduction, starting with half the number of elements and halving the stride each round.
- Round completion and safety. Explain why each reduction round must finish before the next begins within a block.
- Round count and speedup. Distinguish the logarithmic number of reduction rounds from total work and actual elapsed-time speedup.
- Per-block thread communication. Explain how shared memory lets threads in a block exchange data and why this helps tree reduction.
- GPU memory hierarchy. Distinguish registers, shared memory, and global memory by their scope and their place in the speed-versus-size hierarchy.
- Per-thread input staging. Describe how each thread loads its input into shared memory for reduction.
- Load completion barrier. Explain why the loading step ends with __syncthreads().
- Identity padding for bounds. Explain how out-of-bounds threads pad an incomplete block and why the padding preserves the sum.
- Shared memory sizing. Explain how to size sdata for the block and how dynamic shared memory avoids manual size coupling.
- Tree reduction rounds. Trace how active threads combine shared-memory values as the stride halves, until the block sum remains in sdata[0].
- Between-round synchronization. Explain why __syncthreads() appears at the bottom of each reduction-loop iteration.
- Block size constraint. Explain why the simple halving-stride reduction needs a power-of-two block size and describe a way to handle other sizes.
- Block Result Handoff. Describe how a block reports its reduced sum to the global result.
- Safe Global Result Updates. Explain why atomicAdd is used when block totals are added to the global result.
- Block barrier behavior. Explain what __syncthreads() requires threads in a block to do and why its waiting cost cannot justify skipping it.
- Barrier scope and kernel boundary. Distinguish block-level synchronization from completion across blocks, and identify where all blocks are guaranteed to have finished.
- Atomic combination of block subtotals. Explain how atomicAdd safely combines block subtotals into a global result and why a block barrier cannot do this job.
- Per-block atomic cost. Explain why the tree reduction uses one atomicAdd per block rather than per element, and apply that relationship to a block count.
- Multi-block reduction strategy. Describe how the program combines independently reduced block slices and identify the alternative two-pass approach.
- Result accumulator initialization. Explain why the result must be initialized to zero before the sum is accumulated.
- Floating-point reduction reproducibility. Explain why floating-point reduction order can change results and how deterministic modes address reproducibility.
- Adapting the reduction operation. Explain how to adapt the sum kernel for maximum or minimum while retaining its tree structure and synchronization.
- Reducibility and ordering requirements. Explain why associativity, an identity, and commutativity matter for this reduction kernel, using an example of an operation that needs a different ordering.
- Result initialization by bytes. Explain why maximum and minimum results need different initialization from sum, and why cudaMemset is unsuitable for INT_MIN and INT_MAX.
- PyTorch reduction underpinnings. Explain how PyTorch sum and max calls relate to the CUDA reductions built in this chapter.
- Custom kernel tradeoffs. Justify when a custom CUDA reduction might be useful instead of a PyTorch library call.
- Shared state safety failures. Explain the common root and typical symptom of the reduction mistakes, using an example from the section.
- Reduction step synchronization. Explain why a reduction needs __syncthreads() after each step and describe the symptoms of leaving it out.
- Result initialization for reduction. Explain why an uninitialized result can corrupt a reduction sum, how the error may appear, and how to prevent it in main().
- Atomic block result updates. Explain why regular addition to the final result can lose block contributions and how atomicAdd fixes it.
- Shared memory sizing mismatch. Explain why changing the block size can break a reduction kernel with a static shared-memory array, and describe two ways to keep the allocation appropriate for the launch.
- Reduction inputs and outputs. Describe what a reduction takes as input and produces, including how the operation or each thread's contribution can vary.
- Reduction thread cooperation. Describe how threads cooperate in reduction rounds and how the section's implementation coordinates and merges their results.
What it builds on
- What makes a problem “mappable”? (chapter 4) · Map applies an operation independently across elements.
- Mistake 1: Accessing another thread’s element (chapter 4) · Threads can race when one reads an element another is writing.
- The three-level hierarchy: grids, blocks, and threads (chapter 3) · The distinction between threads and blocks.
- The global index formula (chapter 3) · How a thread obtains its global index.
- Bounds check (chapter 2) · A bounds check distinguishes valid input indices from out-of-range ones.
- Part A: How many blocks do I need? (chapter 3) · Calculating the number of blocks needed for a given number of elements.
- Step 1: Allocate device memory (set up desks in the grading room) (chapter 2) · Allocating device memory
Section by section
Introduction
This introduction contrasts reduction with Map and illustrates parallel summation through partial sums and pairing rounds.
- Reduction and Map contrast: Distinguish reduction from Map in terms of their results and thread cooperation.
Why it matters: This distinction helps you recognize when independent element-by-element work is not enough. - Chunk and subtotal strategy: Describe the two-step plan for summing the exam scores using groups and partial sums.
Why it matters: Breaking a large sum into partial sums lets groups contribute to the same overall result. - Parallel pairing rounds: Trace how repeated pairing reduces a group of scores and explain how simultaneous additions shorten the sequence of rounds.
Why it matters: Pairing shows how a sum can progress through parallel rounds instead of one long chain of additions.
Key terms: Map, Reduction pattern, cooperation, partial sums, subtotal, chunk, parallel reduction, pairs, rounds, subtotals
5.1 A million values, one answer
This section defines reduction as combining a collection into one value using an associative operation.
- Reduction and its operations: Describe reduction as combining a collection of values into one result, and identify examples of operations used for it.
Why it matters: Recognizing the many-to-one structure helps you identify problems suited to reduction. - Associativity and grouping test: Explain associativity and apply the two-part test to distinguish addition or maximum from subtraction, noting the floating-point addition caveat.
Why it matters: Associativity is what makes different groupings suitable for the reductions described here.
Key terms: reduction, addition, max, min, multiplication, associative, subtotals, floating-point addition
5.2 You already know reduction
The section connects familiar reductions to identity elements, safe GPU padding, and counting with a map followed by a sum.
- Familiar reduction examples: Describe sum and maximum reductions using the section’s everyday examples.
Why it matters: These familiar examples make the reduction pattern easier to recognize in other problems. - Identity elements and padding: Explain what an identity element does and how spare GPU threads use it for safe padding.
Why it matters: Identity padding lets extra threads participate without corrupting the answer. - Counting with map and sum: Describe how to count elements satisfying a condition using a map followed by a sum.
Why it matters: This shows how a count can be expressed using a familiar reduction.
Key terms: sum reduction, max reduction, identity element, padding, spare threads, count, map, sum
5.3 Use cases for reduction
This section identifies uses of reduction in software and parallel algorithms and contrasts reduction with scan.
- Reduction in Practical Workloads: Describe where reduction appears in familiar software and as a step within larger algorithms.
Why it matters: Recognizing reduction helps you spot it both in familiar commands and inside larger computations. - MapReduce Pattern Relationship: Describe how the Map and Reduction patterns relate to the core idea of MapReduce.
Why it matters: This connection shows how the patterns fit into large-scale data processing. - Scan and Reduction Outputs: Distinguish the outputs of scan and reduction.
Why it matters: This distinction prepares you to recognize when intermediate totals matter.
Key terms: reduction, tensor.sum(), histogram computation, Map pattern, Reduction pattern, MapReduce, Scan pattern, running total
5.4 Why naive parallelism fails
The section previews a comparison between sequential reduction and a naive parallel attempt, without yet explaining why the parallel attempt fails.
5.5 The sequential approach
This section explains sequential array summation and why its loop-carried dependency makes straightforward parallelization difficult.
- Sequential reduction dependency: Explain how a running total reduces an array and why the loop cannot be straightforwardly parallelized.
Why it matters: Recognizing this dependency shows why a parallel reduction needs a different approach from a simple running sum. - Reduction and scan outputs: Distinguish the results retained by a reduction and a scan.
Why it matters: The difference in retained results helps distinguish scan from reduction.
Key terms: running total, loop-carried dependency, sum, reduction, Scan pattern, intermediate result
5.6 The shared-variable trap: many threads, one counter
This section explains why naive shared-counter reduction races, how atomic addition fixes correctness, and why a single atomic counter remains a bottleneck.
- Shared counter race condition: Explain how concurrent read-modify-write updates to a shared result can lose additions and produce timing-dependent results.
Why it matters: Understanding lost updates helps you recognize why a seemingly simple parallel sum can be wrong. - Atomic addition for correctness: Explain how replacing the ordinary shared update with atomicAdd prevents lost additions.
Why it matters: Atomic addition provides a correct way for threads to update the same counter. - Single counter performance bottleneck: Explain why an atomic addition by every thread can be correct but still slow.
Why it matters: Correct synchronization alone does not guarantee a fast parallel reduction. - Pairwise private slot: Describe how one thread combining a partner's value into a private slot avoids the shared-counter collision in the smallest tree.
Why it matters: This simple pairwise combination introduces the idea behind a tree of partial sums.
Key terms: race condition, read-modify-write, shared variable, atomic operation, atomicAdd, indivisible, global counter, atomic operations, serializing, private slot
5.7 Tree reduction: from concept to code
This section motivates tree reduction as a way to avoid incorrect concurrent writes and a slow single-counter bottleneck.
- Tree reduction motivation: Explain why organizing a reduction as a tree is presented as an alternative to plain and atomic shared-counter approaches.
Why it matters: This explains why a correct parallel sum also needs a better way to organize cooperation.
Key terms: tree reduction, atomic version, single counter
5.8 The tournament analogy
This section explains tree reduction through a tournament analogy, including stride-based pairing, round coordination, and the difference between fewer rounds and actual speedup.
- Tournament reduction structure: Explain how a tournament illustrates tree reduction and how max and sum use the same tree with different combining operations.
Why it matters: The tournament picture makes it easier to see how many inputs become one result in parallel. - Stride based pairing: Trace a stride-based sum reduction, starting with half the number of elements and halving the stride each round.
Why it matters: Tracing the stride shows which values combine at each stage of the GPU reduction. - Packed active thread benefits: Explain why pairing by a shrinking stride keeps active threads packed and why that is better for the hardware than pairing neighbors.
Why it matters: The pairing arrangement affects how efficiently the GPU runs the reduction. - Round completion and safety: Explain why each reduction round must finish before the next begins within a block.
Why it matters: The next round depends on results from the previous one, so its timing matters for correctness. - Round count and speedup: Distinguish the logarithmic number of reduction rounds from total work and actual elapsed-time speedup.
Why it matters: A large reduction in sequential rounds does not guarantee the same improvement in elapsed time.
Key terms: tree reduction, max reduction, sum reduction, binary associative operator, stride, round, active threads, warps, consecutive addresses, block
5.9 Where threads share data: shared memory
This section introduces shared memory as a fast, per-block space that enables threads to exchange data for tree reduction.
- Per-block thread communication: Explain how shared memory lets threads in a block exchange data and why this helps tree reduction.
Why it matters: Tree reduction requires threads to share intermediate data within a block. - GPU memory hierarchy: Distinguish registers, shared memory, and global memory by their scope and their place in the speed-versus-size hierarchy.
Why it matters: The memory hierarchy explains why shared memory is a useful local workspace for cooperating threads. - Shared memory declaration: State the CUDA keyword used to declare shared memory and distinguish it from standard C/C++.
Why it matters: The keyword identifies a declaration as CUDA shared memory.
Key terms: shared memory, block, tree strategy, registers, global memory, data locality, __shared__, CUDA extension
5.10 Building the kernel step by step
This section previews the three acts of building a tree reduction kernel.
5.11 Act 1: Everyone writes on the whiteboard
This section explains how reduction threads stage input in shared memory, pad incomplete blocks, synchronize, and size the shared array safely.
- Per-thread input staging: Describe how each thread loads its input into shared memory for reduction.
Why it matters: This staging puts each thread’s contribution where the block can use it in the reduction. - Load completion barrier: Explain why the loading step ends with __syncthreads().
Why it matters: Waiting for all writes prevents the reduction from reading unfinished shared data. - Identity padding for bounds: Explain how out-of-bounds threads pad an incomplete block and why the padding preserves the sum.
Why it matters: Identity padding lets a partially filled block participate without invalid input reads or a changed result. - Shared memory sizing: Explain how to size sdata for the block and how dynamic shared memory avoids manual size coupling.
Why it matters: Matching shared storage to the block prevents out-of-bounds accesses and avoids wasting space.
Key terms: global index, global memory, register, shared memory, sdata, tid, __syncthreads(), barrier, idx < n, identity element
5.12 Act 2: The tournament begins
This section traces a shared-memory tree reduction, explains its synchronization, and identifies the block-size constraint of the simple loop.
- Tree reduction rounds: Trace how active threads combine shared-memory values as the stride halves, until the block sum remains in sdata[0].
Why it matters: This is how a block turns many values into one partial sum. - Between-round synchronization: Explain why __syncthreads() appears at the bottom of each reduction-loop iteration.
Why it matters: The next round must use the results produced by the preceding round. - Block size constraint: Explain why the simple halving-stride reduction needs a power-of-two block size and describe a way to handle other sizes.
Why it matters: An unsuitable block size can silently produce an incomplete sum.
Key terms: tree reduction, stride, tid, sdata, __syncthreads(), barrier, blockDim.x, power of 2, identity element
5.13 Act 3: Report the block’s result
This section explains how one thread per block safely adds the block’s reduced sum to the global result.
- Block Result Handoff: Describe how a block reports its reduced sum to the global result.
Why it matters: The block needs one representative to contribute its finished sum. - Safe Global Result Updates: Explain why atomicAdd is used when block totals are added to the global result.
Why it matters: Safe updates ensure that contributions from different blocks are not lost.
Key terms: sdata[0], thread 0, global result, tid, atomicAdd, race condition
5.14 The barrier: __syncthreads()
This section explains how __syncthreads() acts as a block-wide barrier and why it cannot coordinate different blocks.
- Block barrier behavior: Explain what __syncthreads() requires threads in a block to do and why its waiting cost cannot justify skipping it.
Why it matters: The barrier keeps threads from moving ahead before their blockmates are ready. - Barrier scope and kernel boundary: Distinguish block-level synchronization from completion across blocks, and identify where all blocks are guaranteed to have finished.
Why it matters: Knowing the barrier's scope prevents you from treating separate blocks as if they could wait for one another inside the kernel.
Key terms: __syncthreads(), barrier, block, kernel boundary, atomicAdd
5.15 The last piece: atomic operations
This section explains how atomic operations safely combine block subtotals and why tree reduction avoids the performance cost of an atomic update per element.
- Atomic combination of block subtotals: Explain how atomicAdd safely combines block subtotals into a global result and why a block barrier cannot do this job.
Why it matters: This is how the separate block results become one correct answer without a cross-block race. - Per-block atomic cost: Explain why the tree reduction uses one atomicAdd per block rather than per element, and apply that relationship to a block count.
Why it matters: Reducing the number of competing global updates helps the GPU do useful work in parallel. - Reduction performance comparison: Describe what the end-to-end CPU, atomic-per-element GPU, and tree-reduction GPU comparison shows about parallel structure.
Why it matters: The comparison shows why choosing a parallel structure matters as much as getting a correct GPU result.
Key terms: atomicAdd, read-modify-write, global result, __syncthreads(), tree reduction, block, contention, atomic-per-element, memory bandwidth, end-to-end
5.16 Putting it all together: the complete kernel
This section assembles a multi-block integer sum and explains initialization and floating-point reproducibility concerns.
- Multi-block reduction strategy: Describe how the program combines independently reduced block slices and identify the alternative two-pass approach.
Why it matters: This connects a block-level reduction to a sum across the whole array. - Result accumulator initialization: Explain why the result must be initialized to zero before the sum is accumulated.
Why it matters: Even a correctly combined reduction can produce the wrong sum if its starting value is not initialized. - Floating-point reduction reproducibility: Explain why floating-point reduction order can change results and how deterministic modes address reproducibility.
Why it matters: Tiny order-dependent differences can accumulate during machine-learning training, even with the same random seed.
Key terms: multi-block reduction, atomicAdd, two-pass scheme, per-block partials, cudaMemset, d_result, floating-point reductions, associativity, deterministic modes, bit-for-bit reproducibility
5.17 Beyond addition: max, min, and more
This section shows how to adapt a sum reduction to other operations and determine whether an operation fits the reduction approach.
- Adapting the reduction operation: Explain how to adapt the sum kernel for maximum or minimum while retaining its tree structure and synchronization.
Why it matters: The same reduction structure can support several useful operations when its operation-specific pieces match. - Reducibility and ordering requirements: Explain why associativity, an identity, and commutativity matter for this reduction kernel, using an example of an operation that needs a different ordering.
Why it matters: These properties determine whether the kernel can combine values without changing the intended result. - Cross-block combination options: Describe how the operation and data type constrain the final cross-block combine, and state the alternative given when a suitable atomic is unavailable.
Why it matters: Choosing a combining operation is not enough if the final cross-block update lacks a suitable atomic. - Result initialization by bytes: Explain why maximum and minimum results need different initialization from sum, and why cudaMemset is unsuitable for INT_MIN and INT_MAX.
Why it matters: An incorrectly initialized result can invalidate an otherwise correct reduction.
Key terms: combining operation, identity element, atomicMax, atomicMin, associative, commutative, stride-halving tree, cross-block atomic, atomicAdd, block partials
5.18 Reduction in NumPy and PyTorch
5.20 PyTorch
This section relates PyTorch reductions to the CUDA reduction pattern, explains when a custom kernel may be worthwhile, and connects reduction to MPI and OpenMP.
- PyTorch reduction underpinnings: Explain how PyTorch sum and max calls relate to the CUDA reductions built in this chapter.
Why it matters: The library calls reuse the reduction ideas you have already learned, even though their implementations are optimized. - Custom kernel tradeoffs: Justify when a custom CUDA reduction might be useful instead of a PyTorch library call.
Why it matters: This helps you decide whether a custom kernel is justified rather than assuming handwritten code is always necessary. - Reduction across parallel platforms: Describe how MPI and OpenMP express the same idea of combining partial results.
Why it matters: Recognizing partial-result merging helps you transfer the reduction pattern beyond GPUs.
Key terms: tensor.sum(), tensor.max(), hierarchical reduction, per-block partials, identity, custom CUDA kernel, generic dispatched code, fuse, memory round trips, MPI_Allreduce
5.21 Common mistakes
This section identifies unsafe shared-state handling as the common cause of four reduction mistakes that can produce wrong sums.
- Shared state safety failures: Explain the common root and typical symptom of the reduction mistakes, using an example from the section.
Why it matters: A reduction that runs without crashing can still produce an unreliable sum.
Key terms: shared state, sdata, global result, atomicAdd
5.22 Mistake 1: Missing __syncthreads()
This section explains how missing synchronization between reduction steps can produce intermittent wrong sums and how to fix it.
- Reduction step synchronization: Explain why a reduction needs __syncthreads() after each step and describe the symptoms of leaving it out.
Why it matters: A reduction can appear to work while silently producing an incorrect sum.
Key terms: __syncthreads(), barrier, reduction step, sdata
5.23 Mistake 2: Forgetting to initialize result
This section explains how an uninitialized result corrupts a reduction sum and how to prevent it.
- Result initialization for reduction: Explain why an uninitialized result can corrupt a reduction sum, how the error may appear, and how to prevent it in main().
Why it matters: A correct reduction kernel can still produce a wrong total if its result starts with garbage.
Key terms: cudaMalloc, cudaMemset, result
5.24 Mistake 3: Regular add instead of atomic
This section explains why ordinary addition can lose block contributions in a reduction and how atomicAdd prevents the error.
- Atomic block result updates: Explain why regular addition to the final result can lose block contributions and how atomicAdd fixes it.
Why it matters: This mistake can make a multi-block reduction silently return an inconsistent, incorrect total.
Key terms: read-modify-write, atomicAdd, race condition
5.25 Mistake 4: Shared memory size mismatch
This section diagnoses shared-memory sizing mismatches and recaps how reduction combines values across cooperating threads.
- Shared memory sizing mismatch: Explain why changing the block size can break a reduction kernel with a static shared-memory array, and describe two ways to keep the allocation appropriate for the launch.
Why it matters: Matching shared memory to the launch prevents out-of-bounds writes without needlessly reserving space. - Reduction inputs and outputs: Describe what a reduction takes as input and produces, including how the operation or each thread's contribution can vary.
Why it matters: This checklist helps you recognize problems that can use the same reduction pattern. - Reduction thread cooperation: Describe how threads cooperate in reduction rounds and how the section's implementation coordinates and merges their results.
Why it matters: A reduction depends on coordinating partial results rather than having every thread work independently.
Key terms: shared memory, blockDim.x, BLOCK_SIZE, extern __shared__, third launch parameter, reduction, associative operation, sum, max, min
5.26 Summary
Where this is used later
- (chapter intro) (chapter 6) · Reduction combines values into one result.
- Understanding the Scan pattern (chapter 6) · Reduction combines an array into a single result.
- From one counter to 256 bins (chapter 8) · Builds on the idea that reduction can contribute to histogram computation.
- The conflict problem (chapter 8) · The shared-variable trap when many threads update one counter.
- Up-sweep: building the reduction tree (chapter 7) · Builds a reduction tree during the scan algorithm's up-sweep.
- Strided access: skipping elements (chapter 10) · Explores the consequences of strided memory access.
- Hillis-Steele versus Blelloch (chapter 7) · Compares scan algorithms in terms of their efficiency.
- The Hillis-Steele algorithm (chapter 6) · Shared memory as a place where threads share data.
Concepts explained in this chapter
- atomicAddatomicAdd is a CUDA atomic operation that adds a value to a memory location without allowing concurrent updates to interfere.
- identity elementAn identity element is a value that leaves the result of an operation unchanged when combined with another value.
- reduction operationThe reduction pattern combines many input values into one result using an associative operation, often by combining partial results in parallel.
- atomic operationAn atomic operation performs a read-modify-write or similar memory update as an indivisible action with respect to other threads.
- race conditionA race condition is a concurrency error in which unsynchronized reads and writes let one thread overwrite a value before another thread reads it.
- reduction treeA reduction tree is a tree structure in which values are combined through successive levels to produce an aggregate.
- shared memoryShared memory is a fast, explicitly managed memory space accessible to threads within a GPU thread block.
- associativityAssociativity is the property that regrouping operands does not change the result of an operation.
- __syncthreads()__syncthreads is a CUDA barrier operation that makes threads in one block wait until all threads in that block reach the barrier.
- strideMemory stride is the distance between memory locations accessed by successive elements of an access pattern.
- barrierA barrier is a synchronization mechanism that makes participating threads wait until the required threads reach a specified point.
- synchronizationSynchronization is the coordination of concurrent computations so that their execution observes required ordering or completion constraints.
- CUDA kernelA CUDA kernel is a function launched on a GPU to execute concurrently across many GPU threads.
- dynamic shared memoryDynamic shared memory is shared memory whose capacity is specified when a CUDA kernel is launched.
- static shared memoryStatic shared memory is shared memory whose capacity is fixed in a CUDA kernel declaration.
- associative binary operationAn associative operation is an operation whose result is unchanged when its operands are regrouped.
- paddingPadding adds extra elements to a data structure, commonly with neutral values, to meet an algorithm's size requirements.
- read-modify-write operationA read-modify-write is an operation sequence that reads a value, computes an update, and writes the result back.
- mapThe map pattern applies an operation independently to each element of a collection.
- sequential processingSequential execution performs computational steps one at a time rather than concurrently.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
