liveBook · Manning

Thread

As explained in GPU Programming with Triton

A CPU thread is an independently schedulable sequence of instructions executed by a CPU.

A thread is a sequence of instructions executed by a processor. A CPU thread typically follows its own instruction sequence and branches, and the operating system schedules it independently. A GPU thread can be thought of as a worker operating on a slice of data, such as a tensor element or pixel. section 1.3.3

GPU threads normally execute in groups. On NVIDIA GPUs, a warp contains 32 threads that execute the same instruction on their own data. Warps form blocks, and blocks form the grid launched for a kernel. Triton exposes work at the block level as program instances; its compiler manages the individual threads. section 1.3.3 section 1.5

Figure 1.6 The relationship between the software execution model (threads, blocks, and grids) and the underlying GPU hardware (cores, streaming multiprocessors, and device). Each level of the software model corresponds to a specific layer of GPU hardware responsible for executing it..
Growth in training data

The execution hierarchy connects threads, blocks, and grids to the GPU hardware that runs them. section 1.3.3

Many concurrent threads help a GPU hide memory latency: while some wait for data, others can compute. Each thread also has private registers for active values such as operands and intermediate results. Register use matters because the registers available on a streaming multiprocessor must be shared among its threads; higher per-thread usage can reduce concurrency and latency hiding. section 1.4 section 3.2.1

Branching within a warp

For ReLU, threads processing negative and positive values may take different branches. A divergent warp executes the branches serially, enabling only the threads that need each branch. If all threads take the same branch, this divergence penalty does not occur. section 3.4.1

Registers and concurrency

An NVIDIA H100 streaming multiprocessor has 65,536 32-bit registers. At 32 registers per thread, that register budget accommodates 2,048 threads; at 64 registers per thread, it accommodates 1,024. Other hardware limits also constrain concurrency. section 3.2.1

How GPU Programming with Triton covers it

How deep it goes
Explains thread scheduling, warp divergence, latency hiding, and register-concurrency trade-offs without requiring explicit per-thread coordination in Triton.
Examples
Uses ReLU branching and an H100 register budget to illustrate divergence and thread concurrency.
What you can do after
Evaluate and understand how divergence and per-thread register use can affect GPU throughput.
How it connects
Connects threads to warps, blocks, grids, SIMT, registers, and latency hiding.

Also explained in

Compare how 17 books explain thread →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.