Warp divergence occurs when threads in the same warp take different branches of a conditional. In NVIDIA’s 32-thread warp model, threads execute in lockstep, so the hardware processes each branch in turn and disables lanes that do not match it. section 3.4 section 3.4.1
A divergent ReLU branch
For `if x < 0: output = 0; else: output = x`, negative and positive inputs can send lanes down different paths. The warp executes both branches serially; if each takes 10 cycles, the divergent warp takes 20 cycles rather than 10. A split between branches can roughly halve throughput, while lanes taking the same branch avoid this penalty. section 3.4.1
Figure 3.6 When threads in the same warp evaluate an if/else condition differently, the GPU must execute both branches serially. During each branch, only a subset of lanes is active while the others are idle, reducing effective parallelism and throughput.
The figure shows how serial branch execution leaves some lanes inactive during each path. section 3.4.1
For element-wise conditions in Triton, `tl.where` selects a value per lane while keeping the warp on one control-flow path. Both candidate values are evaluated, however, so expensive candidate computations can still cost time. An `if` remains suitable when its condition is shared by every lane in the block. section 3.4.2
output = tl.where(x > 0, x, tl.zeros_like(x))
This ReLU expression uses `tl.where` to select between the input and zero for each lane. section 3.4.2

