liveBook · Manning

Floating-point operations per second

Also called flops, tflops

Floating-point operations per second (FLOP/s) is a measure of computational throughput expressed as the number of floating-point operations performed per second.

How 3 Manning books explain it

Grokking GPU Programming

FLOPS measures the rate of floating-point operations performed per second.

How it is used
Compares GPU compute capacity with memory bandwidth to explain performance limits and classify kernels.
How deep it goes
Defines the rate and distinguishes it from operation counts, then applies it to bandwidth comparisons and the Roofline model.
Examples
Uses an increment kernel, a 50-add kernel, and an example GPU rated at 10 TFLOPS and 900 GB/s.
What you can do after
Evaluate GPU compute capacity alongside memory bandwidth and recognize when data delivery limits kernel performance.

Read how Grokking GPU Programming explains floating-point operations per second →

GPU Programming with Triton

Floating-point operations per second measures the rate at which a processor or system performs floating-point arithmetic.

How it is used
Measures floating-point throughput as a performance metric for processors and computing systems.
How deep it goes
Defines individual operations, total operation counts, and rates, then works through a simple timing calculation.
Examples
Uses a 1,000-add loop and a GPU rated at 10 TFLOPS to illustrate measured and peak throughput.
What you can do after
Evaluate throughput by dividing a floating-point operation count by elapsed time.

Read how GPU Programming with Triton explains floating-point operations per second →

CUDA for LLMs

TFLOPS measures computational throughput in trillions of floating-point operations per second.

How it is used
Uses TFLOPS to assess compute-bound GEMM performance alongside compute-unit throughput.
How deep it goes
Interprets measured TFLOPS through a kernel-optimization ladder, from naive access patterns to vectorized tiling.
Examples
Compares naive and vectorized 4096 × 4096 FP16 GEMM kernels on an RTX 3090.
What you can do after
Evaluate GEMM kernels by comparing TFLOPS with compute and DRAM throughput at relevant workload sizes.

Read how CUDA for LLMs explains floating-point operations per second →

Also covered in

What to know first

Related concepts

Questions about floating-point operations per second

What is floating-point operations per second?

Floating-point operations per second (FLOP/s) is a measure of computational throughput expressed as the number of floating-point operations performed per second.

Which Manning books explain floating-point operations per second?

Grokking GPU Programming, GPU Programming with Triton and CUDA for LLMs.

What should I know before learning floating-point operations per second?

Manning books that teach floating-point operations per second usually introduce caching and float16 first.

What is floating-point operations per second a kind of?

Floating-point operations per second is a kind of operations per second. Operations per second is a throughput metric measuring the number of completed operations during one second.

What is floating-point operations per second also called?

Floating-point operations per second is also called flops or tflops.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.