liveBook · Manning

Block-level reduction

As explained in GPU Programming with Triton

A block-level reduction combines values within one GPU block or Triton program instance using on-chip resources and synchronization.

When many GPU program instances update the same address, global atomics can serialize. A block-level reduction uses on-chip shuffles and shared memory to combine each instance’s tile into one scalar before it issues a global atomic. section 4.1.3

Reducing atomic contention

With 1,024 values per program instance and 256 instances, reducing first changes 262,144 atomics to 256—one per instance. section 4.1.3

How GPU Programming with Triton covers it

How deep it goes
Explains tile-to-scalar aggregation through on-chip shuffles and shared memory, then compares atomic counts before and after reduction.
Examples
Compares 1,024 atomics per program across 256 instances with one atomic per instance.
What you can do after
Apply tl.sum before tl.atomic_add to combine each program’s contributions into one global-memory update.
How it connects
Connects Triton reduction to atomic addition, global-memory contention, and shared-memory execution.

Also explained in

Compare how 2 books explain block-level reduction →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.