liveBook · Manning

Cache flushing

As explained in GPU Programming with Triton

Cache flushing is the deliberate eviction of cached data so subsequent measurements begin without data retained from earlier executions.

GPU benchmarks can report misleading times when inputs remain in cache between runs. Cache flushing deliberately evicts cached data so each measurement starts from a cold state and reflects the kernel’s memory traffic rather than data retained from earlier executions. section 2.2.10

A practical approach is to zero a buffer larger than the GPU’s L2 cache before each iteration. The buffer must exceed the cache capacity to evict it completely; the benchmark then measures repeated kernel executions under consistent cold-cache conditions. section 2.2.10

How GPU Programming with Triton covers it

How deep it goes
Explains cold-cache rationale and eviction with a buffer larger than L2, without detailing cache internals.
Examples
Uses repeated vector-addition benchmarks across input sizes to compare Python, PyTorch, and Triton.
What you can do after
Evaluate GPU-kernel timings with cache effects controlled by evicting cached data between measurements.
How it connects
Connects GPU benchmarking with L2 cache behavior and measured memory bandwidth.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.