liveBook · Manning

Chapter 3: Scaling up: from one block to many

What you will learn in chapter 3 of Grokking GPU Programming

This introduction explains scaling array processing beyond one block by spreading threads across multiple blocks.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains scaling array processing beyond one block by spreading threads across multiple blocks.

Key terms: block, thread, scaling up, kernel

3.1 What’s wrong with one block?

This section explains why a single CUDA block cannot scale to large workloads, how multiple blocks address the limit, and how to detect a rejected launch.

Key terms: block, threads, launch, streaming multiprocessor, SM, shared memory, synchronize, cudaGetLastError(), cudaErrorInvalidConfiguration, launch dimensions

3.2 The three-level hierarchy: grids, blocks, and threads

This section explains CUDA’s grid-block-thread hierarchy, block scheduling, and the implications for ordering and combining work across blocks.

Key terms: grid, block, thread, numBlocks, threadsPerBlock, SM, portable, blocks, independent work, barrier

3.3 Why the thread index alone isn’t enough

This section explains why block-local thread indices overlap and introduces the information needed to give threads distinct array positions.

Key terms: threadIdx.x, index overlap problem, global index, blockIdx.x, blockDim.x, race condition, atomic operations

3.4 The global index formula

This section introduces the one-dimensional global thread index and the chapter’s one-thread-per-element model.

Key terms: global index, blockIdx.x, blockDim.x, threadIdx.x, one thread per element, map threads to data

3.5 The global index as a student ID

This section explains global thread indexing through a student ID analogy and connects the same grouping formula to flattening a two-dimensional array.

Key terms: global index, blockIdx.x, blockDim.x, threadIdx.x, row-major flattening, flat array, row, width, column

3.6 Tracing through the numbers

The section traces global thread indices across blocks and introduces the GPU memory hierarchy.

Key terms: global index, blockIdx.x, blockDim.x, threadIdx.x, registers, shared memory, global memory, memory hierarchy

3.7 Your first multi-block kernel

This section applies global indexing and a bounds check to a multi-block array-doubling kernel, showing how to cover an array safely when blocks do not divide its size evenly.

Key terms: global index, bounds check, if (idx < n), out-of-bounds access, block size, blocks, threads, idle threads, warp, branch divergence

3.8 Launching the right number of threads

Introduces the upcoming calculation of block count and block size for a kernel launch.

3.9 Part A: How many blocks do I need?

This section explains how to calculate the number of CUDA blocks using ceiling division and handle empty inputs safely.

Key terms: ceiling division, numBlocks, threadsPerBlock, integer division, n = 0, 0 blocks, skip the launch, alternative formula, signed int, unsigned type

3.10 Part B: How big should each block be?

This section recommends a practical block size and explains its relationship to warp utilization and bounds checking.

Key terms: threadsPerBlock, warps, block size, profiling, threads per block, blocks, bounds check

3.11 The multi-block kernel recipe

This section brings together the multi-block recipe for processing arrays and explains the role of explicit synchronization.

Key terms: block size, ceiling division, global index, bounds check, threadsPerBlock, numBlocks, threadIdx.x, blockIdx.x, blockDim.x, cudaDeviceSynchronize()

3.12 Common mistakes

This section previews three multi-block addressing mistakes and a symptom-first troubleshooting checklist.

3.13 Mistake 1: Using threadIdx directly

This section diagnoses overlapping array accesses caused by using a block-local thread index and explains how a global index fixes them.

Key terms: threadIdx.x, blockDim.x, block offset, global index, index overlap

3.14 Mistake 2: Truncating instead of rounding up

This section explains why truncating division can leave the tail of an array unprocessed and how ceiling division fixes the block count.

Key terms: integer division, ceiling division, threadsPerBlock, block size

3.15 Mistake 3: Swapping the grid and block sizes

This section explains how swapping CUDA launch dimensions can cause failures or inefficient execution and how to correct their order.

Key terms: number of blocks, threads per block, cudaGetLastError(), warp

3.16 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 2: Your first parallel program Chapter 4: The Map pattern →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.