liveBook · Manning

Register spilling

As explained in Grokking GPU Programming

Register spilling is the compiler's placement of register values in slower memory when register demand exceeds the available or permitted registers.

A GPU kernel may need more registers per thread than it is allowed. The compiler spills excess values into local memory: private to that thread, but stored in slow off-chip memory—not a fast CPU-style stack. section 14.6

Spilling can follow an attempt to raise occupancy. For example, asking __launch_bounds__(256, 8) may limit register use to 32 per thread; reducing a kernel from 48 to 32 registers can cause spilling. More resident warps may not make up for the extra memory cost. section 14.8 section 14.12

When deciding whether to push occupancy:

  • A spilled value costs roughly 28–50 cycles on an L1 hit, or 400–600 cycles if it comes from DRAM; register access costs no extra cycles. section 14.6
  • Profile the trade-off rather than assuming higher occupancy is faster; keeping lower occupancy may be best if each warp already runs fast enough. section 14.12 section 14.8

How Grokking GPU Programming covers it

How deep it goes
Defines spilling, locates values in off-chip local memory, quantifies access costs, and explains the occupancy trade-off.
Examples
Uses register counts of 32, 48, and 64 per thread and a launch-bounds example to illustrate occupancy and spill risk.
What you can do after
Evaluate register-limit changes by profiling occupancy gains against possible spill costs.
How it connects
Connects register spilling to GPU occupancy, registers, local memory, and CUDA launch bounds.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.