liveBook · Manning

Tl.reduce

As explained in GPU Programming with Triton

tl.reduce is a Triton operation that reduces tensor values with a programmer-provided binary combine function.

tl.reduce combines tensor values using a programmer-provided binary function. Like other Triton reduction operations, it works on a tensor tile local to one program instance; its axis selects a tile dimension, not the full global tensor. section 4.1.2

A reduction across the launch grid needs orchestration beyond tl.reduce: write partial results to global memory and reduce them in another kernel, or use atomic operations. section 4.1.2

Figure 4.3 Two-pass reduction dataflow. The first kernel reduces tiles independently, writing one partial sum per program instance to global memory. The second kernel reduces those partials to a single output value
CH04 F03 Fartale

Shows the two-pass pattern: program instances reduce tiles independently, then a second kernel combines their partial sums. section 4.1.2

How GPU Programming with Triton covers it

How deep it goes
Defines the tile-local axis boundary and contrasts it with grid-wide reduction, which requires another kernel or atomics.
Examples
Contrasts tile-local sums and maxima with two-pass and atomic global-sum implementations.
What you can do after
Understand that a tl.reduce operates within one program instance and cannot synchronize across the launch grid.
How it connects
Connects to tl.sum, tl.max, tl.min, tl.argmax, tl.argmin, tl.xor_sum, program instances, global memory, and atomic operations.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.