When one thread writes shared data that another thread reads, the write must finish before the read begins. This producer-consumer dependency makes the read depend on the write; a value that is never read creates no such dependency. section 11.4
In a parallel reduction, threads first load values into shared-memory slots. Later rounds read values written by other threads, so __syncthreads() separates each write phase from the reads that follow. It also prevents a thread from overwriting a slot while another thread is still reading it. section 11.4
Use a barrier when cross-thread shared-memory accesses require it:
- Place __syncthreads() after a shared-memory write that another thread will read, and before overwriting a slot another thread may still be reading. section 11.4
- Skip an internal barrier when threads only access their own slots or no thread will access shared memory again. section 11.4
