A streaming multiprocessor (SM) is an NVIDIA GPU computational building block. Multiple SMs provide physical resources for parallel work, while each SM executes groups of threads and supplies resources such as registers and on-chip memory. section 1.3.3 section 3.2.1 section 1.3.1
A GPU launch requests a grid of blocks. The hardware scheduler distributes those blocks across SMs as resources become available. An SM can run one or more blocks at a time, then take on more until the grid is processed. In Triton, the programmer writes a kernel and launches a grid of program instances; the compiler manages the underlying CUDA threads, warps, and blocks. section 1.3.3 section 1.4
Figure 1.7 The grid contains every block launched for the kernel, and the GPU schedules those blocks across its Streaming Multiprocessors. Each SM runs one or more blocks at a time depending on available resources and continues pulling new blocks until the grid is fully processed.
Follow how a grid’s blocks are scheduled across SMs and processed as resources become free. section 1.3.3
An SM’s resources limit how much work can be resident at once. Registers hold each thread’s active values, including intermediate results. More registers per thread can leave room for fewer concurrent threads, reducing the ability to hide memory latency by switching between threads. If values spill from registers, they go to local memory, which is backed by global DRAM but cached in L1 and L2. section 3.2.1
Register pressure on an H100
An H100 SM has 65,536 32-bit registers. At 32 registers per thread, the register file can accommodate 2,048 threads; at 64, it can accommodate 1,024. The first calculation also assumes the SM supports 2,048 concurrent threads. section 3.2.1
Hardware details vary, so check the target GPU rather than relying on H100 numbers. Useful properties include its SM or compute-unit count, cores per unit, memory bandwidth, and architecture family. NVIDIA’s SM corresponds to an AMD compute unit or an Intel Xe-Core/subslice in the cross-vendor terminology. section 3.6.1 section 1.3.1

