liveBook · Manning

Streaming multiprocessor

As explained in GPU Programming with Triton

A streaming multiprocessor is NVIDIA's basic GPU computational building block that executes groups of threads and provides computational resources.

A streaming multiprocessor (SM) is an NVIDIA GPU computational building block. Multiple SMs provide physical resources for parallel work, while each SM executes groups of threads and supplies resources such as registers and on-chip memory. section 1.3.3 section 3.2.1 section 1.3.1

A GPU launch requests a grid of blocks. The hardware scheduler distributes those blocks across SMs as resources become available. An SM can run one or more blocks at a time, then take on more until the grid is processed. In Triton, the programmer writes a kernel and launches a grid of program instances; the compiler manages the underlying CUDA threads, warps, and blocks. section 1.3.3 section 1.4

Figure 1.7 The grid contains every block launched for the kernel, and the GPU schedules those blocks across its Streaming Multiprocessors. Each SM runs one or more blocks at a time depending on available resources and continues pulling new blocks until the grid is fully processed.
Block scheduling on GPU Streaming Multiprocessors

Follow how a grid’s blocks are scheduled across SMs and processed as resources become free. section 1.3.3

An SM’s resources limit how much work can be resident at once. Registers hold each thread’s active values, including intermediate results. More registers per thread can leave room for fewer concurrent threads, reducing the ability to hide memory latency by switching between threads. If values spill from registers, they go to local memory, which is backed by global DRAM but cached in L1 and L2. section 3.2.1

Register pressure on an H100

An H100 SM has 65,536 32-bit registers. At 32 registers per thread, the register file can accommodate 2,048 threads; at 64, it can accommodate 1,024. The first calculation also assumes the SM supports 2,048 concurrent threads. section 3.2.1

Hardware details vary, so check the target GPU rather than relying on H100 numbers. Useful properties include its SM or compute-unit count, cores per unit, memory bandwidth, and architecture family. NVIDIA’s SM corresponds to an AMD compute unit or an Intel Xe-Core/subslice in the cross-vendor terminology. section 3.6.1 section 1.3.1

How GPU Programming with Triton covers it

How deep it goes
Explains block scheduling, register allocation, concurrency limits, and spilling, while deferring detailed memory-hierarchy treatment.
Examples
Calculates H100 register-limited thread counts and asks readers to inspect their own GPU’s compute units and memory properties.
What you can do after
Evaluate how SM scheduling and register use affect kernel concurrency and performance.
How it connects
Connects SMs to thread blocks, warps, register pressure, memory latency, and parallelism.

Also explained in

Compare how 2 books explain streaming multiprocessor →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.