Arithmetic intensity
Arithmetic intensity is the ratio of computational operations performed to bytes of data transferred by a computation.
GPU Programming with Triton
Arithmetic intensity is the ratio of floating-point operations performed to bytes transferred between a kernel and global memory.
- How it is used
- Uses arithmetic intensity to identify GPU kernel bottlenecks and motivate reducing global-memory traffic.
- How deep it goes
- Derives per-element traffic and FLOP estimates, compares hardware ridge points, and relates the estimates to roofline ceilings and implementation choices.
- Examples
- Works through naive FP32 matrix multiplication and fused SiLU multiplication, with a Triton SwiGLU kernel.
- What you can do after
- Diagnose memory-bound kernels with arithmetic-intensity estimates and evaluate fusion or tiling as ways to reduce traffic per operation.
Read how GPU Programming with Triton explains arithmetic intensity →
Grokking GPU Programming
Arithmetic intensity is the number of floating-point operations performed per byte transferred to and from memory.
- How it is used
- Uses arithmetic intensity to identify likely GPU kernel bottlenecks and guide memory-access optimization.
- How deep it goes
- Derives the ratio and ridge point, works through numeric kernels, and distinguishes requested traffic from actual DRAM traffic.
- Examples
- Calculates intensities for increment and 50-add kernels, then compares naive and tiled matrix multiplication traffic.
- What you can do after
- Diagnose kernel bottlenecks by calculating intensity, comparing it with an effective ridge point, and profiling bandwidth.
Read how Grokking GPU Programming explains arithmetic intensity →
CUDA for LLMs
Computational intensity is the ratio of floating-point operations performed to bytes transferred.
- How it is used
- Guides GPU kernel optimization by distinguishing memory-bandwidth bottlenecks from opportunities to improve compute utilization.
- How deep it goes
- Calculates FLOPs per byte and per-load FMA reuse, tracing GEMM from basic tiling through register-blocked micro-tiles.
- Examples
- Compares FP16 4096 × 4096 GEMM with GEMV and contrasts 1D and 2D block-tiling load counts.
- What you can do after
- Diagnose GPU kernel bottlenecks by estimating arithmetic intensity and selecting bandwidth-focused or reuse-focused optimizations.
Domain-Specific Small Language Models
The number of compute operations divided by the number of bytes moved to and from memory for an algorithm or model layer.
- How it is used
- Diagnoses whether inference performance is limited by memory transfers or available compute.
- How deep it goes
- Defines the operations-per-byte ratio and works through an attention calculation to identify a bottleneck.
- Examples
- Uses Transformer attention and GPT-Neo's 30.29 ops/byte result against a T4 threshold of 216.6 ops/byte.
- What you can do after
- Diagnose whether inference is memory-bound or compute-bound by comparing workload intensity with the hardware threshold.
Read how Domain-Specific Small Language Models explains arithmetic intensity →
Also covered in
What to know first
- hardware registerA hardware register is a small, rapidly accessible storage location within a processor or processing device that holds data, addresses, or control information during execution.
- GPU memoryGPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
- shared memoryShared memory is a memory region accessible to multiple processes, threads, or other concurrent execution contexts for sharing data.
- tilingTiling is an optimization technique that partitions data or computation into smaller blocks to improve locality, reuse, and memory efficiency.
Related concepts
- roofline modelused by · The roofline model estimates attainable computational performance by bounding throughput according to arithmetic intensity, peak compute throughput, and peak memory bandwidth.
- matrix multiplication kernelproduced by · A matrix multiplication kernel is a GPU kernel that computes the product of matrices.
- ridge pointcontrasts with · A ridge point is the arithmetic intensity at which a roofline model transitions from memory-bandwidth-limited to compute-throughput-limited performance.
- compute-boundcontrasts with · A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
- floating-point operation countmeasures · A floating-point operation count is the number of arithmetic operations performed on values represented in floating-point format by a computation.
- memory boundcontrasts with · A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput.
Questions about arithmetic intensity
What is arithmetic intensity?
Arithmetic intensity is the ratio of computational operations performed to bytes of data transferred by a computation.
Which Manning books explain arithmetic intensity?
GPU Programming with Triton, Grokking GPU Programming, CUDA for LLMs and Domain-Specific Small Language Models.
What should I know before learning arithmetic intensity?
Manning books that teach arithmetic intensity usually introduce hardware register, GPU memory, shared memory and tiling first.
What is the difference between arithmetic intensity and ridge point?
Arithmetic intensity is the ratio of computational operations performed to bytes of data transferred by a computation. A ridge point is the arithmetic intensity at which a roofline model transitions from memory-bandwidth-limited to compute-throughput-limited performance.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



