liveBook · Manning

Compute-bound

As explained in Grokking GPU Programming

Compute-bound describes a computation whose performance is limited primarily by the processor's computational capacity.

A computation is compute-bound when its arithmetic, rather than moving data, limits performance. Arithmetic intensity measures floating-point operations per byte transferred: an increment kernel performs just 1 addition over 8 bytes, while a 50-add kernel performs 50. Even the 50-add kernel remains memory-bound on the example GPU, so doing more math does not automatically make a kernel compute-bound. section 10.3

The Roofline model compares arithmetic intensity with a hardware-specific ridge point: peak FLOPS divided by peak memory bandwidth. Below the ridge, bandwidth limits performance; above it, compute capacity is the limit. The example GPU’s ridge is about 11.1 operations per byte, but the threshold changes across GPUs. section 10.3

Figure 10.2 The Roofline model for our example GPU (10 TFLOPS, 900 GB/s). Attainable performance follows the memory roof up to the ridge point at about 11.1 ops/byte, then flattens along the compute roof. The increment kernel (0.125 ops/byte) and the 50-add kernel (6.25 ops/byte) both land on the memory roof, so both are memory-bound. Both axes are logarithmic, so that intensities from 0.1 to 100 ops/byte fit on one graph.

The Roofline diagram shows where the memory limit gives way to the compute limit; both example kernels fall on the memory-bound side. section 10.3

Compute-bound kernels spend most of their time doing arithmetic and can perform well at lower occupancy. Occupancy alone does not determine performance; identify the bottleneck first. section 14.7

How Grokking GPU Programming covers it

How deep it goes
Defines arithmetic intensity and uses the Roofline ridge point to classify bottlenecks, while noting hardware-specific thresholds.
Examples
Compares increment and 50-add kernels, then contrasts naive matrix multiplication with high-intensity matrix multiplication.
What it assumes
Introduces arithmetic intensity as floating-point operations per byte, without assuming prior knowledge of bottleneck classification.
What you can do after
Evaluate a kernel's likely bottleneck by comparing its arithmetic intensity with the GPU's peak-FLOPS-to-bandwidth ridge point.
How it connects
Connects arithmetic intensity and the Roofline model to memory-bound kernels, matrix multiplication, and occupancy.

Also explained in

Compare how 9 books explain compute-bound →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.