liveBook · Manning

Floating-point operation count

As explained in GPU Programming with Triton

A floating-point operation count is the total number of floating-point operations required by an algorithm or program.

A floating-point operation count measures computational work, not elapsed time. One addition or multiplication counts as one operation; the total count for an algorithm or program is often written FLOPs. FLOP/s, by contrast, measures how quickly that work is performed. section 3.1.2

For matrix multiplication with dimensions M × K and K × N, the total is 2MKN operations: each of the MN output elements requires K multiplications and K additions. Given runtime t in seconds, achieved throughput is 2MKN / (t × 10¹²) TFLOP/s. section 5.5

Counting work

A loop with 1,000 floating-point additions performs 1,000 operations regardless of how long it takes. A 2 × 3 matrix multiplied by a 3 × 2 matrix requires 2 × 2 × 3 × 3 = 36 operations under the multiply-and-add convention. section 3.1.2 section 5.5

How GPU Programming with Triton covers it

How deep it goes
Defines individual operations and derives matrix-multiplication work, without detailing operation counts for arbitrary kernels.
Examples
Counts a loop of 1,000 additions and derives work for a matrix product.
What you can do after
Evaluate computational work by counting operations and calculate achieved TFLOP/s from runtime.
How it connects
Connects operation counts to matrix multiplication and achieved floating-point throughput.

Also explained in

Compare how 4 books explain floating-point operation count →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.