chapter four
4 The Map pattern
This chapter covers
- The map pattern, a universal parallel primitive
- One thread per element, scaled across multiple blocks
- The Parallel Checklist: four questions you’ll ask of every parallel pattern in this book
- Your first two-input map with vector addition
- Element-wise operations on arrays, from scalar multiplication to a neural network’s ReLU
You’ve already written a map kernel. In chapter 3, when you doubled every element of an array (each thread took one element, multiplied it by two, and stored the result), that was a map, and so was chapter 2’s squaring kernel. Chapter 1 called this pattern map; this chapter shows what makes it work.
The map pattern applies the same operation to many elements, which are each processed independently so that none affects any of the others. This means that no output depends on any other output, and in a pure map every output reads only same-index inputs, never a neighbor’s.