liveBook · Manning

Memory traffic

As explained in GPU Programming with Triton

Global memory traffic is the movement of data between GPU computation and global memory.

Memory traffic is data moved between GPU computation and global memory. Repeatedly writing intermediate results to global memory and loading them again increases traffic, leaving the GPU waiting on memory instead of computing. section 3.3

Kernel fusion reduces these transfers by combining chained operations in one pass and keeping intermediate values on chip, often in registers. The arithmetic can stay the same while memory traffic falls. section 3.0 section 3.3

SiLU followed by multiplication

For each element, an unfused SiLU-then-multiply chain makes five memory accesses: load x, store and reload the intermediate, load y, and store the output. With float32 values, that is 20 bytes per element. A fused kernel loads x and y once and stores the output once: three accesses, or 12 bytes per element. section 3.3

Figure 3.4 A fused kernel loads both inputs once, performs the full SiLU and multiplication in registers, and writes the output once, eliminating the intermediate round-trip through global memory.
CH03 F04 Fartale

Shows how fusion avoids the intermediate global-memory round trip by keeping the computation in registers. section 3.3

How GPU Programming with Triton covers it

How deep it goes
Quantifies per-element memory accesses and float32 bytes for fused and unfused operations.
Examples
Counts transfers for an unfused SiLU-and-multiplication chain and its fused equivalent.
What you can do after
Evaluate memory costs by counting per-element loads and stores in fused and unfused kernels.
How it connects
Links memory traffic to kernel fusion, arithmetic intensity, and global memory.

Also explained in

Compare how 2 books explain memory traffic →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.