FLOPS measures how many floating-point operations a processor performs each second. Floating-point numbers include decimals such as 3.14; additions and multiplications are examples of operations. FLOPS is a rate, unlike FLOPs, which counts operations. A trillion operations per second is one TFLOPS. section 10.2 section 10.3
FLOPS describes compute capacity, but it does not alone determine kernel speed: data must also reach the processor. For an example GPU rated at 10 TFLOPS with 900 GB/s of memory bandwidth, a kernel reading one float for each operation would need 40 TB/s to keep computation fully supplied—far more than available. section 10.2
Figure 10.1 The GPU memory bottleneck. Computation throughput (10 TFLOPS) is the 8-lane highway on the left, wide enough to move many operations in parallel. Memory bandwidth (900 GB/s) is the single toll gate on the right: every memory access has to pass through it. The arrow marks the bottleneck: no matter how many lanes feed in, the rate at which cars exit is capped by the gate. This imbalance between compute capacity and memory bandwidth is the central problem this chapter addresses.

The highway and toll-gate analogy illustrates why high compute throughput can be limited by memory bandwidth. section 10.2
A rate is not a kernel-speed guarantee
An increment kernel performs one addition while reading and writing a float. Its arithmetic intensity is 0.125 operations per byte; a kernel performing 50 additions per float reaches 6.25. Both remain memory-bound on the example GPU, despite the extra computation. section 10.3


