liveBook · Manning

Warp divergence

Warp divergence occurs when threads in a GPU warp follow different control-flow paths, requiring the paths to be executed sequentially.

How 2 Manning books explain it

Grokking GPU Programming

Branch divergence is a GPU execution inefficiency that occurs when threads in the same warp take different control-flow paths.

How it is used
Explains a hidden GPU kernel slowdown through the execution of threads in warps.
How deep it goes
Explains masking and serialized branch paths, with a simple additive timing model.
Examples
Uses an even–odd branch and a four-way branch to illustrate active and masked threads.
What you can do after
Diagnose/debug potential divergence by checking whether threads within the same warp take different paths.

Read how Grokking GPU Programming explains warp divergence →

GPU Programming with Triton

Warp divergence is the execution of different conditional branches by threads in the same warp, requiring the branches to be processed sequentially.

How it is used
Diagnoses a GPU performance cost caused by lane-varying branches and motivates Triton’s per-lane selection.
How deep it goes
Explains branch serialization, inactive lanes, throughput loss, and tl.where’s tradeoffs.
Examples
Uses ReLU’s negative-versus-positive branches and a 10-cycle-per-branch illustration.
What you can do after
Apply Triton’s tl.where to select values per lane while accounting for candidate computations.

Read how GPU Programming with Triton explains warp divergence →

What to know first

Related concepts

Questions about warp divergence

What is warp divergence?

Warp divergence occurs when threads in a GPU warp follow different control-flow paths, requiring the paths to be executed sequentially.

Which Manning books explain warp divergence?

Grokking GPU Programming and GPU Programming with Triton.

What should I know before learning warp divergence?

Manning books that teach warp divergence usually introduce parallelism, CPU, memory latency and warp first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.