liveBook · Manning

Resource contention

As explained in Grokking GPU Programming

Contention occurs when multiple threads compete to access or update the same resource, potentially limiting parallel performance.

Contention happens when multiple threads try to update the same resource at once. In a histogram, threads may all increment the counter for one bin. An atomic update keeps those writes correct, but threads targeting the same address must take turns. section 8.11 section 8.4

How much contention occurs depends on the data. If values are spread evenly across bins, few threads compete for each counter. If an image is mostly black, many threads may target bin 0; if every value is identical, updates can become nearly sequential. section 8.7

A common fix is privatization: each block builds a histogram in shared memory, then merges its counts into the global histogram. This reduces global updates from one per input element to at most one per block per bin. The local updates can still contend, but shared-memory atomics cost less than global-memory atomics. section 8.12 section 8.4

How Grokking GPU Programming covers it

How deep it goes
Traces atomic serialization through hot-bin slowdowns and describes block-local privatization, without detailed hardware analysis.
Examples
Uses a mostly black image, identical values, and evenly distributed histogram bins to illustrate differing contention.
What you can do after
Apply CUDA atomics or block-local histogram privatization to handle competing updates and reduce global contention.
How it connects
Connects contention to atomics, histograms, shared memory, privatization, and OpenMP thread-local accumulation.

Also explained in

Compare how 4 books explain resource contention →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.