liveBook · Manning

Register file

As explained in GPU Programming with Triton

A register file is the hardware storage array containing the registers available to a processor execution unit.

A register file is the fast, per-SM storage holding values that GPU threads are actively using. Each thread’s registers supply arithmetic operands and receive results; those values are private to the thread. section 3.2.1

Register use affects both data access and concurrency:

  • More registers per thread can mean fewer concurrent threads on an SM, reducing the GPU’s ability to hide memory latency. section 3.2.1
  • If register demand exceeds physical capacity, the compiler spills values to local memory. Spilled data is backed by global DRAM and may be cached, but accesses remain much slower than register access. section 3.2.1
  • In Triton, register management is largely hidden: programmers use local variables while the compiler allocates registers. section 3.2.1

H100 allocation trade-off

An H100 SM has 65,536 32-bit registers. At 32 registers per thread, the example fits 2,048 threads; at 64, it fits 1,024, assuming the SM supports that many concurrent threads. section 3.2.1

Figure 3.3 Memory system of an NVIDIA H100 GPU, showing per-SM register file, unified L1/shared memory, constant caches, the chip-level L2 cache, and HBM3 global memory (with PCIe/NVLink connectivity).
CH03 F03 Fartale

Locate the per-SM register file in the GPU memory hierarchy alongside caches and global memory. section 3.2

How GPU Programming with Triton covers it

How deep it goes
Defines per-thread use, H100 capacity, concurrency trade-offs, and spilling to local memory.
Examples
Compares H100 occupancy examples using 32 or 64 registers per thread.
What you can do after
Understand how register demand can limit concurrent threads or cause slower spilled-memory accesses.
How it connects
Connects registers to GPU memory hierarchy, arithmetic logic units, thread concurrency, and register spilling.

Also explained in

Compare how 2 books explain register file →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.