liveBook · Manning

Read-modify-write operation

As explained in Grokking GPU Programming

A read-modify-write is an operation sequence that reads a value, computes an update, and writes the result back.

A read-modify-write operation reads a value, computes an update, then writes the result back. Though `++` and `+=` look like single actions, threads can interleave these steps: two may read the same old value and overwrite one another’s updates. The result is a race condition, and the final value may vary between runs. section 8.2 section 5.6

Two threads, one counter

If a counter holds 5, two threads can both read 5 and then both write 6. The counter ends at 6 instead of 7 because one increment is lost. An atomic operation makes the update indivisible; for a histogram, that can mean replacing `hist[bin]++` with `atomicAdd(&hist[bin], 1)`. section 8.2 section 8.10

Figure 8.1 The race condition (Scenario A, top) and the correct sequential execution (Scenario B, bottom). When T0 and T1 run concurrently (A), both threads read count=5 and both write 6, so one increment is lost: the final count is 6. When T0 completes before T1 starts (B), the final count is correctly 7.

Compare the interleaved and sequential counter updates to see how thread timing loses an increment. section 8.2

How Grokking GPU Programming covers it

How deep it goes
Explains the three-step update and thread interleavings, then points to atomic updates and other protections.
Examples
Uses a shared counter, bank deposits, a sum accumulator, and histogram bins to illustrate lost updates.
What you can do after
Diagnose lost or inconsistent updates by spotting shared increments or additions without atomic protection, phase separation, or single ownership.
How it connects
Connects race conditions with atomic operations, reductions, and histograms.

Also explained in

Compare how 5 books explain read-modify-write operation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.