Imagine a GPU with plenty of arithmetic capacity, but a narrow gate through which data must pass. If the compute units wait for data faster than memory can supply it, the kernel is memory-bound. The limiting resource is memory data transfer, not the speed of the arithmetic itself. section 10.2
A useful test is arithmetic intensity: the number of floating-point operations performed per byte transferred. Low intensity means each byte supports little computation. In the Roofline model, attainable performance on the memory-bound side is limited by bandwidth multiplied by arithmetic intensity. section 10.3
An increment kernel
For output[i] = input[i] + 1.0f, each element requires one addition, a 4-byte read, and a 4-byte write. Its arithmetic intensity is 1 ÷ 8 = 0.125 operations per byte. On an example GPU with 10 TFLOPS of peak compute and 900 GB/s of bandwidth, the ridge point is approximately 11.1 operations per byte. The increment falls well below it, on the memory-bound side. section 10.3
Figure 10.2 The Roofline model for our example GPU (10 TFLOPS, 900 GB/s). Attainable performance follows the memory roof up to the ridge point at about 11.1 ops/byte, then flattens along the compute roof. The increment kernel (0.125 ops/byte) and the 50-add kernel (6.25 ops/byte) both land on the memory roof, so both are memory-bound. Both axes are logarithmic, so that intensities from 0.1 to 100 ops/byte fit on one graph.

The Roofline figure places the increment and a 50-add kernel on the bandwidth-limited slope, below the example GPU’s ridge point. section 10.3
The ridge point is peak compute throughput divided by peak memory bandwidth. Above it, the model’s compute roof limits performance; below it, the memory roof does. The threshold differs by GPU, so calculate it for the hardware in use rather than treating 11.1 operations per byte as universal. section 10.3
Repeated reads in matrix multiplication
In a naive matrix multiplication, each output thread reads a row of A and a column of B, repeating requests for values other threads also need. For the example with rows and columns of length 1,024, one thread requests 8,192 input bytes and performs 2,048 operations: 0.25 operations per byte. Tiling reduces repeated requests and raises the example’s intensity to 8 operations per byte. A cache may already satisfy some requests, so requested traffic is not necessarily the same as traffic crossing from DRAM. section 12.1
Improving a memory-bound kernel can mean using bandwidth more efficiently or reusing data to reduce transfers. Higher occupancy can also help by letting the GPU schedule other warps while some wait for data. But more occupancy is not automatically faster: bandwidth may already be saturated, or the access pattern may be the real constraint. section 10.2 section 14.7

















