liveBook · Manning

Occupancy

As explained in GPU Programming with Triton

GPU occupancy is the proportion of a processor's available concurrent threads or warps that are active for execution.

GPU occupancy measures how fully an SM is populated with concurrent warps: active warps divided by the SM’s maximum supported warps. If an SM supports 64 warps but register usage permits 32, occupancy is 50%. Higher occupancy gives the scheduler more warps to switch between while others wait on memory, helping hide latency. section 3.2.1

In Triton, local variables are assigned to registers automatically. Increasing register usage—especially in fused kernels that keep more intermediates alive—can cross an occupancy threshold, reduce concurrent warps, and expose memory latency. Lower occupancy is not automatically worse: sufficient registers can prevent spills, while performance also depends on instruction-level parallelism, memory coalescing, and Tensor Core utilization. section 3.2.1 section 3.7

Register pressure in a fused kernel

Fusing operations can reduce global-memory traffic by keeping intermediate values in registers, but those additional live values may lower occupancy. The practical target is a balance: retain enough concurrent warps to hide latency without forcing register spills or starving each thread of working registers. section 3.2.1 section 3.7

How GPU Programming with Triton covers it

How deep it goes
Defines occupancy mathematically and explains register-driven thresholds, but stops short of detailed hardware occupancy calculations.
Examples
Uses a 64-warp SM limited to 32 active warps and relates fused kernels to register pressure and reduced global-memory traffic.
What it assumes
Assumes basic awareness of GPU threads, warps, SMs, registers, and memory latency while defining occupancy explicitly.
What you can do after
Diagnose/debug with it by relating register usage and fusion choices to concurrent warps, latency hiding, spills, and performance.
How it connects
Connects occupancy with SMs, warps, registers, memory latency, spilling, instruction-level parallelism, memory coalescing, and Tensor Cores.

Also explained in

Compare how 3 books explain occupancy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.