Scatter operations update output locations selected by input indices. When multiple inputs target the same location, atomic updates preserve correctness, but heavy collisions serialize at the L2 cache. Reduce values within each program instance before issuing atomic writes to limit contention. section 4.1.3
Common scatter workloads include:
- PyTorch scatter_add and index_add, and nn.Embedding’s backward pass, where repeated indices require summing updates to the same output. section 4.1.3
- Histogram and binning kernels, where input values determine output addresses that may be shared. section 4.1.3
tl.atomic_add(pointer, val, mask=None, sem=None, scope=None)
Triton’s atomic-add interface accepts a pointer, value, optional mask, memory semantics, and scope. section 4.1.3
