A thread is a sequence of instructions executed by a processor. A CPU thread typically follows its own instruction sequence and branches, and the operating system schedules it independently. A GPU thread can be thought of as a worker operating on a slice of data, such as a tensor element or pixel. section 1.3.3
GPU threads normally execute in groups. On NVIDIA GPUs, a warp contains 32 threads that execute the same instruction on their own data. Warps form blocks, and blocks form the grid launched for a kernel. Triton exposes work at the block level as program instances; its compiler manages the individual threads. section 1.3.3 section 1.5
Figure 1.6 The relationship between the software execution model (threads, blocks, and grids) and the underlying GPU hardware (cores, streaming multiprocessors, and device). Each level of the software model corresponds to a specific layer of GPU hardware responsible for executing it..
The execution hierarchy connects threads, blocks, and grids to the GPU hardware that runs them. section 1.3.3
Many concurrent threads help a GPU hide memory latency: while some wait for data, others can compute. Each thread also has private registers for active values such as operands and intermediate results. Register use matters because the registers available on a streaming multiprocessor must be shared among its threads; higher per-thread usage can reduce concurrency and latency hiding. section 1.4 section 3.2.1
Branching within a warp
For ReLU, threads processing negative and positive values may take different branches. A divergent warp executes the branches serially, enabling only the threads that need each branch. If all threads take the same branch, this divergence penalty does not occur. section 3.4.1
Registers and concurrency
An NVIDIA H100 streaming multiprocessor has 65,536 32-bit registers. At 32 registers per thread, that register budget accommodates 2,048 threads; at 64 registers per thread, it accommodates 1,024. Other hardware limits also constrain concurrency. section 3.2.1
















