liveBook · Manning

Chapter 14: Warps and occupancy

What you will learn in chapter 14 of Grokking GPU Programming

Introduces warp-based thread execution and its performance implications when threads take different paths.

After this chapter you will be able to

What it builds on

Section by section

Introduction

Introduces warp-based thread execution and its performance implications when threads take different paths.

Key terms: warps, single instruction, multiple threads, SIMT, branch divergence, warp

14.1 The marching band inside your GPU

This section explains how NVIDIA warps execute thread code, synchronize safely, and use scheduling to hide memory latency.

Key terms: warp, block, streaming multiprocessor, SM, SIMT, SIMD, branch divergence, lane, bounds check, lockstep

14.2 When the band splits: branch divergence

This section explains how branch divergence serializes paths within a warp and how to estimate its cost.

Key terms: warp, branch divergence, masking, Golden Rule of Divergence, serialized model, utilization, slowdown, global index, threadIdx.x, idx % 2

14.3 Three strategies for managing divergence

This section explains three strategies for managing warp divergence and illustrates warp-uniform work with tree reduction.

Key terms: predication, select op, branchless code, divergence, warp-uniform, blockIdx.x, threadIdx.x, tree reduction, stride-based pairing, neighbor pairing

14.4 Which strategy should you choose?

This section explains how to choose a divergence-management strategy and why divergence also reduces memory efficiency.

Key terms: predication, branchless select, warp-uniform branches, blockIdx, threadIdx, data-dependent, sort first, separate kernels per type, divergence, divergent warp

14.5 Filling the stage: occupancy

This section defines SM occupancy using a train analogy and shows how to express it as a percentage of the maximum warps.

Key terms: occupancy, SM, warp, max warps, compute capability table

14.6 What limits occupancy: three resources

This section explains how registers, shared memory, and block slots limit occupancy on a GPU’s streaming multiprocessors.

Key terms: occupancy, SM, registers, shared memory, block slots, register file, registers per thread, resident threads, register spilling, local memory

14.7 Occupancy is a means, not a goal

This section explains why occupancy is a performance aid rather than a target and presents ways to measure and tune it.

Key terms: occupancy, registers, local memory, throughput, memory-bound, compute-bound, arithmetic intensity, ridge point, resident warps, achieved occupancy

14.8 Worked example: checking all three constraints

This section works through an occupancy calculation, identifies the limiting resource, and explains practical qualifications and trade-offs.

Key terms: occupancy, registers, shared memory, block limit, warps, __launch_bounds__, spilling, profile, register allocation unit, occupancy API

14.9 Common mistakes

This section identifies mistaken assumptions about warp execution order and occupancy that can cause failures or slower kernels.

Key terms: warps, block, execution order, resident warps, occupancy

14.10 Mistake 1: Assuming cross-warp execution order

This section explains why unsynchronized communication between warps can produce a race and how a block-wide barrier fixes it.

Key terms: warp, cross-warp race, stale value, __syncthreads(), barrier, memory fence, block

14.11 Mistake 2: Blindly maximizing occupancy

This section explains why maximizing occupancy can slow a kernel and recommends measuring configurations instead.

Key terms: occupancy, registers, spill, local memory, warps, configuration

14.12 Summary

Concepts explained in this chapter

← Chapter 13: Read-only data optimization

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.