liveBook · Manning

Map

As explained in Grokking GPU Programming

The map pattern applies an operation independently to each element of a collection.

Suppose you need to double every number in an array. A sequential loop visits each element in turn, but doubling one element does not depend on any other. The operations can instead run independently, one per element. This pattern is called map. section 4.1

A pure map applies the same operation to each index: B[i] = f(A[i]). It can also use several inputs, as in C[i] = f(A[i], B[i]). Each output reads only inputs at its own index and does not depend on another output. section 4.0 section 4.1

Vector addition

For A = [1, 2, 3] and B = [4, 5, 6], compute C[i] = A[i] + B[i]. The result is [5, 7, 9]. Each addition needs only the two values at its own index. section 4.4

Figure 4.1 The Map pattern: one thread, one output element, one independent transformation. Each thread reads from array A, applies function f, and writes the result to array B, with no thread depending on any other. The same pattern covers everything from simple vector addition (two inputs, C[i] = A[i] + B[i]) to neural network activations like ReLU. One caution: ReLU’s max(0, A[i]) is an element-wise maximum (a map), whereas finding the single largest value across the array is a reduction (chapter 5), not a map.

Follow each input through its own independent transformation to its output. section 4.1

In CUDA, assign one thread to each output. Compute a global index with blockIdx.x * blockDim.x + threadIdx.x, and check idx < n before accessing an element. Launch at least n threads; the bounds check keeps extra threads from accessing outside the array. section 4.16 section 4.3

Writing a result back to its own input index is safe from races in a pure map, but it destroys the original value. If later code needs that value, write to a separate output array. Reading neighbors while overwriting inputs is a different hazard: threads can race. section 4.15 section 4.16

Map describes the independent operation, not a guarantee of parallel execution: JavaScript’s .map() runs sequentially, while CUDA can distribute work across GPU threads. A reduction combines values into one result; a stencil reads neighbors; a gather reads at another computed index. section 4.2 section 4.16 section 1.9

How Grokking GPU Programming covers it

How deep it goes
Works through independence, same-index reads, CUDA indexing, bounds checks, and in-place mistakes.
Examples
Uses array doubling, vector addition, and ReLU to illustrate independent transformations.
What it assumes
Assumes familiarity with C loops and function calls, while explaining needed pointer techniques as they arise.
What you can do after
Build it from scratch as a CUDA kernel with one thread per output and a bounds check.
How it connects
Distinguishes map from gather, reduction, and stencil, and connects it to embarrassingly parallel work.

Also explained in

Compare how 12 books explain map →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.