liveBook · Manning

Memory_order_relaxed

As explained in GPU Programming with Triton

Relaxed memory ordering guarantees atomicity of the update without imposing ordering constraints on other memory accesses.

Relaxed ordering preserves the atomicity of an atomic read-modify-write, but places no ordering constraints on other memory accesses. Unlike acquire-release ordering, it does not act as a fence, avoiding fence overhead when surrounding accesses need no ordering. section 4.1.3

In Triton, choose it with `sem="relaxed"` on an atomic operation. It suits accumulation where the atomic update must be indivisible but the visibility order of surrounding loads and stores does not matter. section 4.1.3

Split-K accumulation

In Split-K GEMM, program instances atomically add partial results. Relaxed ordering is suitable when the goal is the final sum, not an ordering of surrounding memory accesses. section 4.1.3

How GPU Programming with Triton covers it

How deep it goes
Defines read-modify-write atomicity and the absence of ordering constraints, and contrasts relaxed semantics with acquire-release ordering.
Examples
Uses atomic addition of partial results in Split-K GEMM as an example.
What you can do after
Apply Triton's `sem="relaxed"` option when only atomicity matters, not the visibility order of surrounding accesses.
How it connects
Connects atomic_add ordering to acquire-release semantics and Split-K GEMM partial-result accumulation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.