Occupancy
GPU occupancy is the proportion of a processor’s supported concurrent execution capacity occupied by active threads or warps.
Grokking GPU Programming
GPU occupancy is the ratio of active warps on a streaming multiprocessor to the maximum number of warps it can support.
- How it is used
- Uses occupancy to reason about GPU utilization and kernel performance.
- How deep it goes
- Calculates occupancy from resource constraints, discusses allocation rounding, and explains why higher occupancy can hurt performance.
- Examples
- Uses a train analogy, numerical occupancy examples, and a worked calculation with register and shared-memory limits.
- What you can do after
- Diagnose GPU occupancy limits and compare kernel configurations using Nsight Compute and performance measurements.
CUDA for LLMs
Occupancy is the ratio of active warps on a GPU streaming multiprocessor to the maximum supported number of active warps.
- How it is used
- Use occupancy alongside throughput and runtime to assess GPU kernel utilization and guide optimization.
- How deep it goes
- Compares achieved occupancy across kernels and relates it to bandwidth, parallelism, and sequential bottlenecks.
- Examples
- Compares naive and warp-parallel softmax, LayerNorm, and top-k kernels, plus vectorized GEMV and cuBLAS.
- What you can do after
- Diagnose kernel utilization with Nsight Compute and judge occupancy against bandwidth, compute, and runtime measurements.
GPU Programming with Triton
GPU occupancy is the proportion of a processor's available concurrent threads or warps that are active for execution.
- How it is used
- Diagnoses GPU performance effects of register allocation, fused-kernel design, and the ability to hide memory latency.
- How deep it goes
- Defines occupancy mathematically and explains register-driven thresholds, but stops short of detailed hardware occupancy calculations.
- Examples
- Uses a 64-warp SM limited to 32 active warps and relates fused kernels to register pressure and reduced global-memory traffic.
- What you can do after
- Diagnose/debug with it by relating register usage and fusion choices to concurrent warps, latency hiding, spills, and performance.
What to know first
- threadA thread is a distinct sequence of execution with its own control flow and state, managed or scheduled by a processor, runtime, or operating system.
- hardware registerA hardware register is a small, rapidly accessible storage location within a processor or processing device that holds data, addresses, or control information during execution.
- forward passA forward pass is a computation that propagates inputs through a model or dependency structure in the forward direction to calculate outputs or intermediate values.
- GPU memoryGPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
Related concepts
- Nsight Computemeasured by · Nsight Compute is NVIDIA’s interactive profiler for analyzing and optimizing CUDA kernel performance and resource use.
- warpmeasures · A warp is a group of GPU threads scheduled and issued together for execution, typically processing the same instruction on different data.
- hardware registerconfigured by · A hardware register is a small, rapidly accessible storage location within a processor or processing device that holds data, addresses, or control information during execution.
- __launch_bounds__configured by · __launch_bounds__ is an NVIDIA-specific CUDA compiler attribute that specifies a kernel's maximum threads per block and optionally a minimum number of blocks per multiprocessor.
- compute-boundcontrasts with · A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
- memory boundused by · A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput.
Questions about occupancy
What is occupancy?
GPU occupancy is the proportion of a processor’s supported concurrent execution capacity occupied by active threads or warps.
Which Manning books explain occupancy?
Grokking GPU Programming, CUDA for LLMs and GPU Programming with Triton.
What should I know before learning occupancy?
Manning books that teach occupancy usually introduce thread, hardware register, forward pass and GPU memory first.
What is the difference between occupancy and compute-bound?
GPU occupancy is the proportion of a processor’s supported concurrent execution capacity occupied by active threads or warps. A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


