chapter eleven
11 Shared memory
This chapter covers
- The GPU memory hierarchy
- How to declare and use shared memory
- The 3-Step Recipe, the pattern behind this book’s shared memory algorithms
- Bank conflicts and the padding trick that fixes column access
- Tiling and transposing a matrix with a padded shared memory tile
Imagine you’ve written a matrix multiplication kernel. It produces the correct result. But suppose it takes 850 milliseconds on a large matrix. Your colleague’s version takes 45 milliseconds with the same input and output, so it’s ≈19 times faster (850 ÷ 45 = 18.9).
The difference is where the data lives during computation. Your slow kernel reads every value from the GPU’s global memory. Your colleague’s kernel loads data into a small, fast memory right next to the processing cores, and reads from there instead.