liveBook · Manning

Chapter 4: The Map pattern

What you will learn in chapter 4 of Grokking GPU Programming

This introduction defines the Map pattern, illustrates its independent per-element work, and relates it to GPU execution.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction defines the Map pattern, illustrates its independent per-element work, and relates it to GPU execution.

Key terms: map pattern, independently, one thread per output element, vector addition, input array, output array, index, CUDA core, CPU core, streaming multiprocessor

4.1 What makes a problem “mappable”?

This section explains why independent element-wise work can run in parallel and how to distinguish a pure map from related patterns.

Key terms: map, embarrassingly parallel, thread, transformation, pure map, independence test, gather, stencil, same-index rule

4.2 Where you’ve seen Map before

This section connects Map to familiar programming tools and explains how independent, side-effect-free element operations can run in different execution environments.

Key terms: map, element, row, map pattern, sequential, parallel, CUDA, SIMD, AVX, OpenMP

4.3 One thread, one element

This section explains how Map assigns one output element to each GPU thread using a global index and the single program, multiple data model.

Key terms: Map pattern, global index, blockIdx.x, blockDim.x, threadIdx.x, bounds check, __global__, kernel, single program, multiple data, SPMD

4.4 Map in action: vector addition

This section introduces vector addition as a two-input map that sums corresponding elements of two vectors.

Key terms: two-input map, vector addition, element-wise sum

4.5 Thinking in parallel

This section applies the Parallel Checklist to vector addition before writing a kernel.

Key terms: Parallel Checklist, map pattern, vector addition

4.6 Breaking it down, line by line

This section breaks down a vector-addition Map kernel and its launch, synchronization, error handling, and indexing pitfalls.

Key terms: Map kernel, global-index formula, bounds check, idx, numBlocks, threadsPerBlock, ceiling division, cudaMemcpy, cudaDeviceSynchronize(), blocking

4.7 Map in action: scalar vector multiplication

This section defines scalar vector multiplication as multiplying each vector element by the same scalar.

Key terms: scalar vector multiplication, vector, scalar

4.8 Thinking in parallel

This section explains why scalar-vector multiplication is a Map and how its shared scalar differs from the array arguments.

Key terms: Map, scalar, read-only, int*, int k, passed by value, GPU memory

4.9 Breaking it down, line by line

This section explains how scalar vector multiplication assigns one element to each thread and passes shared scalar and per-element array data to the kernel.

Key terms: scalar, thread, A[idx], parameter, value, pointer, cudaMemcpy, host memory, device, main()

4.10 Map in action: vector ReLU

This section introduces vector ReLU as a per-element Map operation used in neural networks.

Key terms: Map, ReLU, Rectified Linear Unit, B[i] = max(0, A[i]), activation function, training, inference

4.11 Thinking in parallel

This section applies the parallel checklist to vector ReLU, emphasizing independent elementwise work.

Key terms: input array, output, A[idx], thread

4.12 Breaking it down, line by line

This section explains the ReLU map kernel, its floating-point host setup and edge cases, and connects element-wise CUDA maps to an absolute-value exercise and PyTorch operations.

Key terms: ReLU, ternary operator, if/else, float, double, f suffix, scanf, sizeof(float), fmaxf, NaN

4.13 Common mistakes

This section identifies two data-access mistakes that can quietly produce incorrect kernel results.

Key terms: thread, kernel, input

4.14 Mistake 1: Accessing another thread’s element

This section explains why reading another thread’s changing element causes a race and how separate input and output arrays avoid it.

Key terms: race condition, in-place update, input array, map, reduction, scan, stencil, idx > 0, boundary check

4.15 Mistake 2: Accidentally overwriting your input

This section explains how to avoid destroying Map inputs, use const for read-only arrays, and check whether outputs can be computed independently.

Key terms: in-place, pure map, race, separate output array, const, read-only parameters, compile-time error, Map, same-index inputs, independently

4.16 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 3: Scaling up: from one block to many Chapter 5: The Reduction pattern →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.