A GPU kernel can do plenty of math and still spend most of its time waiting for data. Arithmetic intensity measures how much work it gets from each memory trip: floating-point operations performed divided by bytes transferred. Count both reads and writes. The result is expressed in operations per byte (ops/byte). section 10.3 section 10.12
One addition versus 50
An increment reads one 4-byte float, adds 1, and writes one 4-byte float. Its intensity is 1 ÷ 8 = 0.125 ops/byte. If it instead performs 50 additions on the loaded value before writing it, the intensity rises to 50 ÷ 8 = 6.25 ops/byte. Both examples are memory-bound on the example GPU, despite the extra arithmetic. section 10.3
To judge an intensity, compare it with the GPU’s ridge point: peak floating-point operations per second divided by peak memory bandwidth. Below it, memory bandwidth sets the pace; above it, peak compute throughput is the limit. For a GPU with 10 TFLOPS and 900 GB/s, the ridge point is about 11.1 ops/byte. Other GPUs have different thresholds. section 10.3 section 14.7
Figure 10.2 The Roofline model for our example GPU (10 TFLOPS, 900 GB/s). Attainable performance follows the memory roof up to the ridge point at about 11.1 ops/byte, then flattens along the compute roof. The increment kernel (0.125 ops/byte) and the 50-add kernel (6.25 ops/byte) both land on the memory roof, so both are memory-bound. Both axes are logarithmic, so that intensities from 0.1 to 100 ops/byte fit on one graph.

The Roofline graph places the two example kernels below the ridge point, where the memory-bandwidth slope limits attainable performance. section 10.3
The ridge point based on peak bandwidth assumes good memory access. Strided or scattered accesses can waste transferred bytes and reduce effective bandwidth. A kernel with 100 operations per 8 bytes has an intensity of 12.5 ops/byte, just above the example GPU’s peak-bandwidth ridge point. But at 12.5% bandwidth efficiency, the effective ridge point rises to about 89 ops/byte, putting that same kernel on the memory-bound side. Profile bandwidth rather than trusting the operation count alone. section 10.12 section 10.14
Reuse data in matrix multiplication
In a naive matrix multiplication, each thread repeatedly requests a row of one input and a column of the other. Counting input reads, one thread performs 2,048 operations for 8,192 bytes, or 0.25 ops/byte. A tiled version loads chunks into shared memory for reuse: in the worked example, requested input traffic falls from about 4 GB to 128 MB with 32-by-32 tiles, while the arithmetic stays the same. These are requested loads; caches can make actual DRAM traffic lower. section 12.1 section 12.8
Use intensity as a starting point, not a promise of speed. If a kernel appears memory-bound, inspect its access pattern and measured bandwidth. More occupancy may help hide memory latency, but it will not necessarily help if bandwidth is already saturated or accesses are inefficient. section 10.12 section 14.7



