liveBook · Manning

Roofline model

As explained in GPU Programming with Triton

A roofline model is a performance model that bounds a kernel's achievable throughput using arithmetic intensity, peak compute throughput, and peak memory bandwidth.

The roofline model estimates a GPU kernel’s achievable performance from its arithmetic intensity—the operations performed per byte moved—and the hardware’s compute and memory limits. A kernel’s position relative to the compute and memory roofs indicates its bottleneck and remaining optimization headroom. section 3.1.5

Read the two limits together:

  • The compute roof is a horizontal ceiling set by peak arithmetic throughput. section 3.1.5
  • The memory roof rises with arithmetic intensity; its slope is peak memory bandwidth. section 3.1.5

A naive FP32 matrix multiplication

For large K, the naive kernel performs about 0.25 FLOP per byte. On the stated H200 specifications, that places its calculated bandwidth-limited throughput at 1.2 TFLOP/s, versus an FP32 ridge point near 14 FLOP/byte. These are theoretical estimates, not measured results. section 5.5.3

Figure 5.11 The theoretical roofline position of the naive FP32 matmul kernel on an NVIDIA H200 SXM. The 0.25 FLOP/byte arithmetic intensity intersects the listed 4.8 TB/s memory-bandwidth roof at 1.2 TFLOP/s. The FP32 ridge is approximately 14 FLOP/byte. Both positions are calculated from published specifications and are not measured kernel results.
CH05 F11 Fartale

See the calculated roofline position for the naive FP32 matmul and the H200’s stated limits. section 5.5.3

How GPU Programming with Triton covers it

How deep it goes
Defines the two roofs and arithmetic-intensity plot, then works through a numerical matrix-multiplication estimate.
Examples
Uses naive FP32 matrix multiplication on H200, estimating 0.25 FLOP/byte and 1.2 TFLOP/s.
What you can do after
Evaluate theoretical performance ceilings and identify whether memory bandwidth or compute throughput limits a kernel.
How it connects
Connects arithmetic intensity, compute throughput, and memory bandwidth to GPU-kernel performance.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.