liveBook · Manning

Read-after-write hazard

As explained in Grokking GPU Programming

A read-after-write hazard occurs when a read may happen before a preceding write to the same location is complete.

A read-after-write hazard occurs when one thread reads a shared-memory slot before another thread’s preceding write is complete. In a parallel reduction, one round writes results that the next round reads; without synchronization, a thread may read too soon. section 11.4

Use a block-wide __syncthreads() barrier to keep those accesses in order:

  • Place a barrier after shared-memory writes that other threads will read, and between reduction rounds that read one another’s results. section 11.4
  • Every thread in the block must reach the barrier. Skip it when accesses are thread-private or shared memory is no longer used. section 11.4

How Grokking GPU Programming covers it

How deep it goes
Explains barrier placement across reduction phases and rounds, plus when barriers are unnecessary.
Examples
Uses reduction threads writing and reading shared-memory slots across successive rounds.
What you can do after
Diagnose unsafe cross-thread accesses and place __syncthreads() barriers where shared-memory writes feed reads.
How it connects
Connects shared memory and thread-block synchronization with write-after-read hazards and parallel reduction.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.