liveBook · Manning

Producer-consumer dependency

As explained in Grokking GPU Programming

A dependency in which a consumer's read of shared data must follow the producer's write to that data.

When one thread writes shared data that another thread reads, the write must finish before the read begins. This producer-consumer dependency makes the read depend on the write; a value that is never read creates no such dependency. section 11.4

In a parallel reduction, threads first load values into shared-memory slots. Later rounds read values written by other threads, so __syncthreads() separates each write phase from the reads that follow. It also prevents a thread from overwriting a slot while another thread is still reading it. section 11.4

Use a barrier when cross-thread shared-memory accesses require it:

  • Place __syncthreads() after a shared-memory write that another thread will read, and before overwriting a slot another thread may still be reading. section 11.4
  • Skip an internal barrier when threads only access their own slots or no thread will access shared memory again. section 11.4

How Grokking GPU Programming covers it

How deep it goes
Explains read-after-write and write-after-read hazards, barrier placement, and when an internal barrier is unnecessary.
Examples
Uses reduction rounds in which threads read and write one another’s shared-memory slots.
What you can do after
Apply __syncthreads() after writes needed by other threads and before overwriting slots they may still read.
How it connects
Connects shared memory, CUDA synchronization, __syncthreads(), and parallel reduction.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.