liveBook · Manning

Warp scheduler

As explained in Grokking GPU Programming

Warp scheduling is the GPU hardware process of selecting warps for execution on available processing resources.

A warp scheduler keeps an SM busy by issuing work from a ready warp when another warp is waiting for memory. Think of a barista taking another order while milk steams: the waiting warp’s registers are already resident, so switching requires no save-and-restore step. This helps hide memory latency. section 14.1

Taking turns during a memory wait

If a warp computes for 25 cycles and then waits 400 cycles for memory, roughly 16 warps can take turns computing during that wait. With too few warps, the SM may sit idle; with more, a warp is more likely to be ready. This is a simplified single-pipe picture; real SMs have several warp schedulers. section 14.1

How Grokking GPU Programming covers it

How deep it goes
Works through resident registers and a 25-cycle compute, 400-cycle memory-wait example, then notes the single-pipe simplification.
Examples
Uses a barista serving another customer during a wait and estimates how 16 warps can cover a memory stall.
What you can do after
Understand how ready warps can keep an SM working while another warp waits for memory.
How it connects
Connects warp scheduling to streaming multiprocessors, occupancy, memory latency, and CPU context switching.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.