liveBook · Manning

Throughput

As explained in GPU Programming with Triton

Throughput is the amount of computation or data processing completed per unit of time.

GPU throughput measures how quickly work is completed: compute throughput is measured in floating-point operations per second, while memory bandwidth measures bytes transferred per second. A kernel’s arithmetic intensity helps show which resource limits its performance. section 3.1.3 section 3.1.5

The roofline model compares a kernel’s performance with compute and memory ceilings. Low arithmetic intensity points to a memory-bound kernel; high arithmetic intensity points to a compute-bound kernel. Distance from a ceiling indicates remaining optimization headroom. section 3.1.5

Figure 3.2 Visualisation of three kernels as Compute Bound, Memory Bound and Optimal through the roofline model
CH03 F02 Fartale

Compare the memory-bound, compute-bound, and near-optimal kernel positions against the two hardware ceilings. section 3.1.5

H100 arithmetic intensity

The cited H100 peak rates correspond to roughly 590 FLOPs per byte. A workload doing far fewer operations per byte may be limited by memory bandwidth rather than compute. section 3.1.3

How GPU Programming with Triton covers it

How deep it goes
Defines compute and memory ceilings, then classifies kernels by arithmetic intensity and distance from those ceilings.
Examples
Uses H100 peak-rate estimates, representative roofline kernels, and matrix multiplication in transformer workloads.
What you can do after
Evaluate whether a kernel is memory-bound or compute-bound and identify its distance from the corresponding performance ceiling.
How it connects
Connects throughput to arithmetic intensity, memory bandwidth, the roofline model, and matrix multiplication.

Also explained in

Compare how 32 books explain throughput →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.