liveBook · Manning

Thread block

As explained in GPU Programming with Triton

A GPU block is a group of threads scheduled as a unit that can cooperate through shared memory.

A thread block is a group of GPU threads organized to work together. Warps form blocks, and all blocks launched for a kernel form a grid that covers the computation. The GPU schedules blocks across its streaming multiprocessors (SMs), with each SM running one or more blocks at a time as resources permit. section 1.3.3 section 1.5

Figure 1.7 The grid contains every block launched for the kernel, and the GPU schedules those blocks across its Streaming Multiprocessors. Each SM runs one or more blocks at a time depending on available resources and continues pulling new blocks until the grid is fully processed.
Block scheduling on GPU Streaming Multiprocessors

The grid supplies blocks to SMs until all blocks have been processed. section 1.3.3

Threads in a block can cooperate through shared memory: one thread can place data there for another thread in the same block to use. Registers, by contrast, are private to each thread. Shared memory is explicitly managed and is much faster than global memory, though slower than registers. section 1.4 section 1.4

In Triton, a program instance is the unit of work corresponding to a CUDA block. You write the kernel from the perspective of one instance, computing a tile of data and choosing its size; the compiler manages the underlying threads and their coordination. section 1.3.3 section 1.5

Sizing a grid

For a vector of 2,000,000 elements processed at 512 elements per block, the grid needs 3,907 blocks: 3,906 full blocks and one block for the remaining 128 elements. Separately, a 128-thread block contains four 32-thread NVIDIA warps; a grid of 400 such blocks contains 1,600 warps. section 1.6

How GPU Programming with Triton covers it

How deep it goes
Explains the execution hierarchy, SM scheduling, and shared-memory visibility without implementing a complete block-level kernel.
Examples
Uses vector grid-sizing and thread-to-warp counting exercises to illustrate block arithmetic.
What it assumes
Introduces GPU threads and execution-model terminology from first principles.
What you can do after
Evaluate/understand block scheduling and cooperation, and calculate grid and warp counts for simple workloads.
How it connects
Connects thread blocks to warps, grids, streaming multiprocessors, shared memory, and Triton program instances.

Also explained in

Compare how 4 books explain thread block →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.