When a GPU kernel runs slowly, the threads’ memory access pattern may be the bottleneck. The Snapshot Rule is a simple check: look at where all 32 threads in a warp are pointing at one instant, rather than following one thread over time. Nearby addresses can be served together; scattered addresses waste memory bandwidth. section 10.5 section 10.0
Use the snapshot to compare access patterns:
- When consecutive threads access consecutive elements, the access is coalesced. In the copy example, the warp reads 32 neighboring floats together. section 10.5
- When threads access addresses separated by a stride, memory requests spread out; scattered addresses are worse still. section 10.14
- For a row-major matrix, assigning one thread per row can make each warp’s simultaneous reads far apart. Assigning threads across columns instead lets a warp read neighboring elements in the same row. section 10.10
int idx = blockIdx.x * blockDim.x + threadIdx.x;
The figure shows the ideal snapshot: 32 warp threads reading consecutive floats within one contiguous memory span. section 10.5
Practical takeaway: judge memory access from the warp’s point of view at each load. Consecutive addresses are the goal; data layout and thread assignment can determine whether the warp reaches them. section 10.14 section 10.10
