liveBook · Manning

Floating-point operation count

As explained in Grokking GPU Programming

A floating-point operation is an arithmetic operation performed on floating-point values.

A floating-point operation is an arithmetic action on floating-point values, such as addition or multiplication. FLOPs counts operations; FLOPS measures how many operations happen per second. section 10.2 section 10.3

Operation counts help show how much computation a kernel performs relative to the data it moves. Arithmetic intensity is FLOPs divided by transferred bytes, including both reads and writes. section 10.3

Counting operations in kernels

An increment kernel performs one addition, reads four bytes, and writes four, for an arithmetic intensity of 0.125 ops per byte. A multiply-add counts as two operations: a naive matrix-multiplication thread doing 1,024 multiply-adds performs 2,048 FLOPs while moving 8,192 bytes, or 0.25 ops per byte. section 10.3 section 12.1

Figure 10.1 The GPU memory bottleneck. Computation throughput (10 TFLOPS) is the 8-lane highway on the left, wide enough to move many operations in parallel. Memory bandwidth (900 GB/s) is the single toll gate on the right: every memory access has to pass through it. The arrow marks the bottleneck: no matter how many lanes feed in, the rate at which cars exit is capped by the gate. This imbalance between compute capacity and memory bandwidth is the central problem this chapter addresses.

Compare compute throughput with memory bandwidth to see why a high operations-per-second capacity does not guarantee a kernel can stay compute-bound. section 10.2

How Grokking GPU Programming covers it

How deep it goes
Defines FLOPs, distinguishes them from FLOPS, and works through operation-per-byte calculations for simple and matrix kernels.
Examples
Counts an increment as one addition and a matrix thread’s 1,024 multiply-adds as 2,048 operations.
What you can do after
Evaluate whether a kernel is memory-bound by relating its floating-point operation count to bytes transferred.
How it connects
Connects arithmetic intensity to memory bandwidth, compute-bound and memory-bound kernels, and the Roofline model.

Also explained in

Compare how 4 books explain floating-point operation count →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.