A floating-point operation is an arithmetic action on floating-point values, such as addition or multiplication. FLOPs counts operations; FLOPS measures how many operations happen per second. section 10.2 section 10.3
Operation counts help show how much computation a kernel performs relative to the data it moves. Arithmetic intensity is FLOPs divided by transferred bytes, including both reads and writes. section 10.3
Counting operations in kernels
An increment kernel performs one addition, reads four bytes, and writes four, for an arithmetic intensity of 0.125 ops per byte. A multiply-add counts as two operations: a naive matrix-multiplication thread doing 1,024 multiply-adds performs 2,048 FLOPs while moving 8,192 bytes, or 0.25 ops per byte. section 10.3 section 12.1
Figure 10.1 The GPU memory bottleneck. Computation throughput (10 TFLOPS) is the 8-lane highway on the left, wide enough to move many operations in parallel. Memory bandwidth (900 GB/s) is the single toll gate on the right: every memory access has to pass through it. The arrow marks the bottleneck: no matter how many lanes feed in, the rate at which cars exit is capped by the gate. This imbalance between compute capacity and memory bandwidth is the central problem this chapter addresses.

Compare compute throughput with memory bandwidth to see why a high operations-per-second capacity does not guarantee a kernel can stay compute-bound. section 10.2



