When threads in a block share work, one may need to finish writing before others read or start the next step. A barrier makes every thread in that block wait until all have arrived. In CUDA, __syncthreads() also makes the block’s earlier shared and global memory writes visible to its threads after the barrier. section 5.14 section 14.10
Use the barrier where threads depend on one another’s work:
- In a scan, place it after each round so no thread starts the next stride before its partners finish writing. section 6.18
- In tiled matrix multiplication, use it after loading a tile and before threads compute with it. section 12.6
- Every thread in the block must reach the same barrier. Putting it in a branch that some threads skip can hang the kernel or produce incorrect results. section 8.11
for (int stride = 1; stride < n; stride *= 2) {
pout = 1 - pout; pin = 1 - pout;
/* add or copy into buf[pout][tid] */
__syncthreads(); #A
}Compare the scan loop with and without a barrier: the barrier at each round’s end keeps threads from reading unfinished results. section 6.18
A block barrier cannot synchronize different blocks. Ordinary kernel launches provide no in-kernel wait for all blocks; the kernel boundary is the point where all blocks are guaranteed to have finished. section 5.14



