When many GPU program instances update the same address, global atomics can serialize. A block-level reduction uses on-chip shuffles and shared memory to combine each instance’s tile into one scalar before it issues a global atomic. section 4.1.3
Reducing atomic contention
With 1,024 values per program instance and 256 instances, reducing first changes 262,144 atomics to 256—one per instance. section 4.1.3

