liveBook · Manning

Atomic accumulation

As explained in GPU Programming with Triton

Atomic accumulation is the concurrent combination of contributions into shared output locations using atomic updates.

Atomic accumulation combines concurrent contributions at shared global-memory locations using atomic read-modify-write operations. In Triton, tl.atomic_add returns the location’s previous value; a mask can skip invalid addresses. Initialize the output buffer to zero before launch, or contributions will add to uninitialized values. section 4.1.3

tl.atomic_add(pointer, val, mask=None, sem=None, scope=None)

Triton’s tl.atomic_add accepts pointers, values, and an optional mask. section 4.1.3

Shared output address

Four program instances can add partial results to one address. Their execution order may vary, producing different bitwise floating-point results because addition is not associative. section 4.1.3

How GPU Programming with Triton covers it

How deep it goes
Explains read-modify-write semantics, masking, output initialization, contention costs, and floating-point ordering effects.
Examples
Illustrates four program instances adding partial results to one address in variable execution order.
What you can do after
Apply atomic accumulation with Triton’s tl.atomic_add, masked addresses, and a zero-initialized output buffer.
How it connects
Connects reduction patterns with global-memory atomics, compare-and-swap loops, and floating-point non-associativity.

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.