A CUDA kernel can launch thousands of threads, but the GPU does not normally run each one independently. It groups threads into warps of 32. Threads in a warp normally execute the same instruction together, each on its own data. This execution model is called single instruction, multiple threads (SIMT). section 14.0 section 2.8
Think of a warp as a marching band: the musicians follow the same score, but each plays their own part. A streaming multiprocessor (SM) runs many warps. Warps stay within block boundaries, and a block with fewer than 32 threads still occupies a whole warp, leaving some lanes idle. section 14.1 section 2.8
Figure 14.1 A block of 256 threads is divided into eight warps of 32 consecutive threads each: warp 0 holds threads 0–31, warp 1 holds threads 32–63, and so on up to warp 7 (threads 224–255). The grouping is fixed: a thread never moves to another warp.

A 256-thread block forms eight fixed warps of 32 consecutive threads. section 14.1
Doubling an array
Each thread calculates its own array index, checks that the index is less than the array length, and doubles that element. The code describes one thread’s work; the hardware issues instructions across the warp. section 14.1
__global__ void doubleKernel(int* data, int n) {
int idx = blockIdx.x * blockDim.x + threadIdx.x;
if (idx < n) {
data[idx] = data[idx] * 2;
}
}The kernel uses block and thread indices to give each thread its own element, with a bounds check before the write. section 14.1
Threads can encounter different paths through an if statement. When they disagree, the paths take turns and some lanes sit idle; this is branch divergence. Even when threads follow the same path, do not rely on warp lockstep for synchronization. If threads depend on seeing one another’s writes, use __syncthreads(), or __syncwarp() for warp-local algorithms without cross-warp data dependencies. section 14.0 section 14.1
SIMT describes how hardware runs threads; SPMD describes writing one program for threads that work on different data. SIMD is another comparison: its parallel operations use explicit instructions, while SIMT code can look like ordinary per-thread code with branches and loops. section 14.1 section 2.8


