liveBook · Manning

SIMT

As explained in Grokking GPU Programming

SIMT is a GPU execution model in which groups of threads execute instructions together while operating on separate data.

A CUDA kernel can launch thousands of threads, but the GPU does not normally run each one independently. It groups threads into warps of 32. Threads in a warp normally execute the same instruction together, each on its own data. This execution model is called single instruction, multiple threads (SIMT). section 14.0 section 2.8

Think of a warp as a marching band: the musicians follow the same score, but each plays their own part. A streaming multiprocessor (SM) runs many warps. Warps stay within block boundaries, and a block with fewer than 32 threads still occupies a whole warp, leaving some lanes idle. section 14.1 section 2.8

Figure 14.1 A block of 256 threads is divided into eight warps of 32 consecutive threads each: warp 0 holds threads 0–31, warp 1 holds threads 32–63, and so on up to warp 7 (threads 224–255). The grouping is fixed: a thread never moves to another warp.

A 256-thread block forms eight fixed warps of 32 consecutive threads. section 14.1

Doubling an array

Each thread calculates its own array index, checks that the index is less than the array length, and doubles that element. The code describes one thread’s work; the hardware issues instructions across the warp. section 14.1

__global__ void doubleKernel(int* data, int n) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    if (idx < n) {
        data[idx] = data[idx] * 2;
    }
}

The kernel uses block and thread indices to give each thread its own element, with a bounds check before the write. section 14.1

Threads can encounter different paths through an if statement. When they disagree, the paths take turns and some lanes sit idle; this is branch divergence. Even when threads follow the same path, do not rely on warp lockstep for synchronization. If threads depend on seeing one another’s writes, use __syncthreads(), or __syncwarp() for warp-local algorithms without cross-warp data dependencies. section 14.0 section 14.1

SIMT describes how hardware runs threads; SPMD describes writing one program for threads that work on different data. SIMD is another comparison: its parallel operations use explicit instructions, while SIMT code can look like ordinary per-thread code with branches and loops. section 14.1 section 2.8

How Grokking GPU Programming covers it

How deep it goes
Traces warp grouping and array execution, then explains divergence and the need for explicit synchronization.
Examples
Uses a marching-band analogy, an array-doubling kernel, and a warp-count exercise.
What you can do after
Diagnose warp divergence and avoid relying on lockstep when threads exchange data.
How it connects
Connects SIMT to warps, streaming multiprocessors, branch divergence, synchronization, SPMD, and SIMD.

Also explained in

Compare how 3 books explain SIMT →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.