liveBook · Manning

Thread limit

As explained in Grokking GPU Programming

A thread limit is the maximum number of threads that can be assigned to one CUDA block.

A thread limit is the maximum number of threads in one CUDA block: 1,024 on modern GPUs. Think of a block as a classroom with limited room; larger workloads need more blocks, not one oversized block. section 3.0 section 3.1

Exceeding the limit has a practical consequence:

  • A launch such as <<<1, 10000>>> is rejected, so no threads run and the data remains unchanged. section 3.1
  • Use multiple blocks instead, such as <<<40, 250>>> for 10,000 threads. Check a rejected launch with cudaGetLastError() and inspect the launch dimensions. section 3.1

Matrix-multiplication tiles

A square tile uses TILE_SIZE × TILE_SIZE threads per block. TILE_SIZE 32 uses exactly 1,024 threads; TILE_SIZE 64 would use 4,096 and exceed the limit. section 12.8

How Grokking GPU Programming covers it

How deep it goes
Defines the 1,024-thread ceiling, explains its SM constraint, and notes that older GPUs allowed 512.
Examples
Uses a 10,000-thread launch and square matrix-multiplication tiles to illustrate legal and oversized blocks.
What it assumes
Assumes familiarity with kernel launches and threadIdx.x, then connects thread and block organization to the limit.
What you can do after
Diagnose rejected launches by checking their dimensions and distribute work across multiple blocks.
How it connects
Connects thread limits to CUDA blocks, grids, streaming multiprocessors, and shared memory.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.