liveBook · Manning

Block size

As explained in Grokking GPU Programming

Threads per block is the configured number of GPU threads assigned to each CUDA thread block in a kernel launch.

When an array is too large for one group of GPU threads, CUDA can spread the work across many blocks. Block size is the number of threads assigned to each block. In a one-dimensional launch such as kernel<<<40, 256>>>(...), the first number is the block count and the second is the threads per block. section 2.4 section 3.0

A practical starting point is 256 threads per block. GPU threads execute in groups called warps, each containing 32 threads. A block of 256 fills eight warps; a block of 250 leaves six slots unused in its last warp. Start with correct code, then use profiling to decide whether another size works better. section 3.10

Figure 3.8 Block size matters: warp utilization from wasteful to optimal. A block size of 1 still occupies a full warp, leaving 31 of its 32 hardware threads idle, about 97% waste. A block size of 250 is better but still wastes 6 threads in the last warp. A block size of 256 (a multiple of 32) fills every warp completely, leaving no thread in a warp idle.

Compare how block sizes of 1, 250, and 256 fill their allocated warps. section 3.10

Covering 10,000 elements

With 256 threads per block, calculate the block count using ceiling division: (10000 + 255) / 256 = 40. The launch has 10,240 threads. Inside the kernel, compute idx = blockIdx.x * blockDim.x + threadIdx.x, then work only if idx < 10000. The extra 240 threads fail the bounds check, so they do not access the array. section 3.10 section 3.16

A block cannot exceed 1,024 threads on CUDA-supported NVIDIA GPUs at the time of writing. Exceeding the limit makes the launch fail: no threads run, and the data may appear unchanged. Call cudaGetLastError() immediately after the launch to check for the error; use more blocks rather than an oversized block. section 2.4 section 3.16

Block size can also be a correctness requirement, not just a performance choice. In a tiled matrix multiplication kernel where each thread loads one tile element, a 32 × 32 tile needs a matching 32 × 32 block. A 16 × 16 block leaves most tile slots unloaded; extra threads in a block larger than its tile can corrupt shared memory. Different mappings require corresponding changes to the kernel. section 12.4

For tuning, try several valid block sizes and measure runtime with CUDA events or a profiler such as Nsight Compute. Block size affects occupancy alongside registers and shared memory, but the highest occupancy does not necessarily produce the fastest kernel. section 14.7

How Grokking GPU Programming covers it

How deep it goes
Explains launch sizing, warp utilization, bounds checks, and a tile-matching constraint, then recommends measuring alternative sizes.
Examples
Uses a 10,000-element launch and a 32 × 32 tiled matrix multiplication kernel to illustrate sizing.
What it assumes
Assumes familiarity with kernel launches and per-thread indexing before extending array processing to multiple blocks.
What you can do after
Configure CUDA launches with a starting block size, ceiling-divided block count, and bounds check, then measure alternatives.
How it connects
Connects block size to warps, global indexing, occupancy, shared memory, and tiled matrix multiplication.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.