A GPU kernel can have thousands of threads but still leave room for more work on each streaming multiprocessor (SM). Occupancy measures how full that room is: it is the number of active warps divided by the maximum number of warps the SM can hold. On NVIDIA GPUs, a warp contains 32 threads. section 14.5 section 14.1
Counting occupied slots
If an SM can hold 64 warps and has 16 active warps, occupancy is 16 ÷ 64, or 25%. With 48 active warps, it is 75%. The maximum depends on the GPU architecture, so check the device’s compute capability table. section 14.5
Occupancy matters because an SM can switch to another resident warp when one stalls waiting for memory. More resident warps can help hide that wait, but a full SM is not automatically a fast one. section 14.12 section 14.7
Three constraints determine how many warps can reside on an SM:
- Registers per thread: each resident thread needs a share of the SM’s register budget. Higher register use can leave room for fewer blocks. section 14.6
- Shared memory per block: each resident block reserves part of the SM’s shared memory budget. Larger reservations can reduce the number of blocks that fit. section 14.6
- Block size and block slots: a block contains whole warps, and an SM can host only a limited number of blocks. Very small blocks can exhaust those slots before filling the available warp capacity. section 14.6
Find the tightest constraint
Suppose an SM has a 64-warp maximum, 65,536 registers, 48 KB of shared memory, and 16 block slots. A kernel uses 256 threads, 48 registers per thread, and 4 KB of shared memory per block. Registers fit five whole blocks, or 40 warps; shared memory and block slots permit more. Registers therefore limit occupancy to 40 ÷ 64, or 62.5%. Actual allocations use architecture-specific chunks, so a hand calculation can overestimate occupancy. section 14.8
A common mistake is to force maximum occupancy and expect a speedup. Reducing the register budget can cause variables to spill into slower local memory, making each thread slower. A memory-bound kernel may benefit from more warps, but it may already have enough to saturate memory bandwidth. Try different block sizes, register budgets, or shared memory sizes, and keep the configuration that actually runs fastest. section 14.11 section 14.7
On a machine with an NVIDIA GPU, Nsight Compute can report achieved occupancy for each kernel. For example, run: ncu --metrics sm__warps_active.avg.pct_of_peak_sustained_active ./your_program. Compare that report and the kernel’s performance across configurations; do not optimize the occupancy percentage alone. section 14.7


