When multiple GPU threads update the same location, compare-and-swap provides a way to retry an update if another thread intervenes. In Triton, tl.atomic_cas atomically writes a new value only when the stored value equals the expected comparison value. It returns the old value. section 4.1.3
tl.atomic_cas(pointer, cmp, val, sem=None, scope=None)
The call takes a pointer, comparison value, and replacement value, with optional sem and scope arguments. section 4.1.3
Fallback atomic addition
On hardware without native bfloat16 atomicAdd, a compare-and-swap loop reads the current value, computes a result, and attempts to write it back. If another thread changed the value, the loop retries. Under contention, these retries can cost more than a native atomic operation. section 4.1.3
