liveBook · Manning

Write-after-read race

As explained in Grokking GPU Programming

A write-after-read hazard occurs when a write may overwrite a location before a preceding read from it is complete.

Imagine one thread moving on to load the next tile while another is still using the current tile. The new write can replace data before the other thread finishes reading it: a write-after-read race. In shared memory, synchronize before overwriting data that other threads may still be reading. section 12.11

A barrier after loading a tile protects the reads that follow. A second barrier after computing protects those reads from the next iteration’s writes. Without it, a kernel may run without errors yet produce subtly wrong, inconsistent results. section 12.11

tileA[ty][tx] = A[...]; tileB[ty][tx] = B[...];
__syncthreads();    #A
for (k...) sum += ...;   #B
__syncthreads();    #C

The second __syncthreads() separates each tile’s use from the next tile load, preventing early overwrites. section 12.11

Check shared-memory synchronization by asking:

  • Will another thread read what was written? Synchronize before that read. section 11.4
  • Will another thread overwrite data still being read? Synchronize before the overwrite. section 11.4
  • Does every thread in the block reach the barrier? A barrier inside a divergent branch has undefined behavior. section 11.4

How Grokking GPU Programming covers it

How deep it goes
Explains barrier placement between shared-memory phases and tile iterations, including the consequences of missing a barrier.
Examples
Uses tiled matrix multiplication, where the next iteration can overwrite tileA and tileB while threads still read them.
What you can do after
Diagnose incorrect tiled results by checking whether threads overwrite shared tiles before other threads finish reading them.
How it connects
Connects shared memory, __syncthreads(), read-after-write hazards, and tiled matrix multiplication.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.