Memory traffic is data moved between GPU computation and global memory. Repeatedly writing intermediate results to global memory and loading them again increases traffic, leaving the GPU waiting on memory instead of computing. section 3.3
Kernel fusion reduces these transfers by combining chained operations in one pass and keeping intermediate values on chip, often in registers. The arithmetic can stay the same while memory traffic falls. section 3.0 section 3.3
SiLU followed by multiplication
For each element, an unfused SiLU-then-multiply chain makes five memory accesses: load x, store and reload the intermediate, load y, and store the output. With float32 values, that is 20 bytes per element. A fused kernel loads x and y once and stores the output once: three accesses, or 12 bytes per element. section 3.3
Figure 3.4 A fused kernel loads both inputs once, performs the full SiLU and multiplication in registers, and writes the output once, eliminating the intermediate round-trip through global memory.
Shows how fusion avoids the intermediate global-memory round trip by keeping the computation in registers. section 3.3

