liveBook · Manning

Compute-bound

As explained in GPU Programming with Triton

A property of a computation whose execution rate is limited primarily by the speed of arithmetic calculation rather than memory transfer.

A GPU kernel is compute-bound when arithmetic throughput, rather than memory delivery, chiefly determines its execution time. High arithmetic intensity—many operations per byte transferred—can keep arithmetic units busy, especially when data is reused in SRAM and HBM accesses are minimized. section 3.1.4

\[\text{Arithmetic Intensity} = \frac{N_{\text{op}}}{N_{\text{byte}}}.\]

Arithmetic intensity is operations performed per byte moved; it helps distinguish arithmetic-limited work from memory-limited work. section 3.1.3

Compute-bound workloads typically have high arithmetic intensity:

  • A kernel performing about 600 FLOPs on one loaded byte can be compute-bound when memory access is efficient. section 3.1.4
  • Matrix multiplication and convolution are examples; element-wise operations, standalone softmax, and normalization are often memory-bound instead. section 3.1.4
Figure 3.2 Visualisation of three kernels as Compute Bound, Memory Bound and Optimal through the roofline model
CH03 F02 Fartale

The roofline plot compares a kernel’s arithmetic intensity and performance with the hardware’s memory-bandwidth and compute-throughput ceilings; its distance from a ceiling indicates optimization headroom. section 3.1.5

How GPU Programming with Triton covers it

How deep it goes
Defines compute-bound execution through arithmetic-unit saturation and high arithmetic intensity, then locates its ceiling in a roofline model.
Examples
Uses high-operations-per-byte work, matrix multiplication, and convolution as compute-bound examples.
What you can do after
Evaluate whether arithmetic throughput or memory bandwidth limits a kernel by relating its arithmetic intensity to hardware ceilings.
How it connects
Connects arithmetic intensity, memory-bound kernels, SRAM reuse, GPU arithmetic units, and the roofline model.

Also explained in

Compare how 9 books explain compute-bound →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.