liveBook · Manning

CUDA execution hierarchy

As explained in Grokking GPU Programming

The CUDA execution hierarchy consisting of grids, blocks, and threads.

When one block cannot cover a large array, CUDA spreads the work across multiple blocks. Picture a school: classrooms handle separate stacks of exams, while students within each classroom do the individual work. section 3.0 section 3.2

A kernel launch organizes work into three levels:

  • Grid: all blocks launched by one kernel call. section 3.2
  • Block: a group of threads; every block in a launch has the same number of threads. section 3.2
  • Thread: the smallest unit that does the work. section 3.2
    myKernel<<<4, 256>>>(); // 4 classrooms, 256 students each

This launch creates a grid of 4 blocks, each with 256 threads: 1,024 threads total. section 3.2

Each block runs on one streaming multiprocessor (SM), which runs it from start to finish. Blocks may execute in any order, so independent work is safe, but code must not rely on one block finishing before another. Extra blocks wait until an SM is available. section 3.2

How Grokking GPU Programming covers it

How deep it goes
Explains grid, block, and thread roles plus block scheduling and ordering, without detailing lower-level GPU execution internals.
Examples
Uses a school-and-classroom analogy, grading 10,000 exams, and a launch with four blocks of 256 threads.
What it assumes
Assumes familiarity with kernel launches and threadIdx.x, then defines the grid, block, and thread levels.
What you can do after
Understand how blocks distribute work and why correctness cannot depend on block execution order.
How it connects
Connects kernel launches, streaming multiprocessors, and block synchronization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.