In CUDA, threads in a block divide the work of copying a chunk from global memory into shared memory—like a bucket brigade. They can then reuse that data while collaborating, instead of each fetching the same values independently. section 11.2 section 11.3
Keep the loading phase correct:
- Have threads load their assigned pieces, then synchronize before any thread reads data written by another. section 11.3
- For a partial chunk, write an identity value into out-of-range slots; skipping the store leaves shared memory uninitialized. section 11.3
Loading for a reduction
With 256 threads, each thread loads one input value. Neighboring loads combine into 8 transactions instead of 256 separate fetches; the block then reduces the values in shared memory. section 11.3
Figure 11.3 The 3-Step Recipe, shown for a single block: load data from global memory into shared memory, collaborate there over several rounds, then write the result back to global memory. The worked example inside the panels is the parallel reduction from chapter 5. Time flows downward, and each solid arrow is one thread’s write. The curve marks the time when each thread executes the __syncthreads() statement, and the empty space below the curve depicts the time that each thread waits for all threads to complete. The horizontal line marks the time when the last thread executes __syncthreads() (the last thread to arrive never waits), after which all threads proceed, and the shared memory state below the line is the result. In later rounds of Step 2, only the threads still holding data are drawn; the remaining threads call __syncthreads() immediately and wait, so every barrier still involves the whole block. In Step 3, the dashed arrow is an atomicAdd: each block’s thread 0 atomically adds its partial sum into the single result variable in global memory.

Trace the load, collaboration, and write-back phases, including where threads wait at block-wide barriers. section 11.3
