liveBook · Manning

Arithmetic intensity

Arithmetic intensity is the ratio of computational operations performed to bytes of data transferred by a computation.

How 4 Manning books explain it

GPU Programming with Triton

Arithmetic intensity is the ratio of floating-point operations performed to bytes transferred between a kernel and global memory.

How it is used
Uses arithmetic intensity to identify GPU kernel bottlenecks and motivate reducing global-memory traffic.
How deep it goes
Derives per-element traffic and FLOP estimates, compares hardware ridge points, and relates the estimates to roofline ceilings and implementation choices.
Examples
Works through naive FP32 matrix multiplication and fused SiLU multiplication, with a Triton SwiGLU kernel.
What you can do after
Diagnose memory-bound kernels with arithmetic-intensity estimates and evaluate fusion or tiling as ways to reduce traffic per operation.

Read how GPU Programming with Triton explains arithmetic intensity →

Grokking GPU Programming

Arithmetic intensity is the number of floating-point operations performed per byte transferred to and from memory.

How it is used
Uses arithmetic intensity to identify likely GPU kernel bottlenecks and guide memory-access optimization.
How deep it goes
Derives the ratio and ridge point, works through numeric kernels, and distinguishes requested traffic from actual DRAM traffic.
Examples
Calculates intensities for increment and 50-add kernels, then compares naive and tiled matrix multiplication traffic.
What you can do after
Diagnose kernel bottlenecks by calculating intensity, comparing it with an effective ridge point, and profiling bandwidth.

Read how Grokking GPU Programming explains arithmetic intensity →

CUDA for LLMs

Computational intensity is the ratio of floating-point operations performed to bytes transferred.

How it is used
Guides GPU kernel optimization by distinguishing memory-bandwidth bottlenecks from opportunities to improve compute utilization.
How deep it goes
Calculates FLOPs per byte and per-load FMA reuse, tracing GEMM from basic tiling through register-blocked micro-tiles.
Examples
Compares FP16 4096 × 4096 GEMM with GEMV and contrasts 1D and 2D block-tiling load counts.
What you can do after
Diagnose GPU kernel bottlenecks by estimating arithmetic intensity and selecting bandwidth-focused or reuse-focused optimizations.

Read how CUDA for LLMs explains arithmetic intensity →

Domain-Specific Small Language Models

The number of compute operations divided by the number of bytes moved to and from memory for an algorithm or model layer.

How it is used
Diagnoses whether inference performance is limited by memory transfers or available compute.
How deep it goes
Defines the operations-per-byte ratio and works through an attention calculation to identify a bottleneck.
Examples
Uses Transformer attention and GPT-Neo's 30.29 ops/byte result against a T4 threshold of 216.6 ops/byte.
What you can do after
Diagnose whether inference is memory-bound or compute-bound by comparing workload intensity with the hardware threshold.

Read how Domain-Specific Small Language Models explains arithmetic intensity →

Also covered in

What to know first

Related concepts

Questions about arithmetic intensity

What is arithmetic intensity?

Arithmetic intensity is the ratio of computational operations performed to bytes of data transferred by a computation.

Which Manning books explain arithmetic intensity?

GPU Programming with Triton, Grokking GPU Programming, CUDA for LLMs and Domain-Specific Small Language Models.

What should I know before learning arithmetic intensity?

Manning books that teach arithmetic intensity usually introduce hardware register, GPU memory, shared memory and tiling first.

What is the difference between arithmetic intensity and ridge point?

Arithmetic intensity is the ratio of computational operations performed to bytes of data transferred by a computation. A ridge point is the arithmetic intensity at which a roofline model transitions from memory-bandwidth-limited to compute-throughput-limited performance.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.