3 Scaling up: from one block to many
This chapter covers
- Why a single block can’t handle large problems (the 1,024 thread limit)
- CUDA’s three-level hierarchy: Grid, Block, Thread
- The global index formula that gives every thread a unique ID
- How to choose block sizes and compute the number of blocks
- Why bounds checking becomes critical with multiple blocks
- The four-step recipe that turns any 1D array problem into a correct kernel launch
In Chapter 2, you learned how to launch a kernel, a function that runs on the GPU, and how each thread knows its own position using threadIdx.x. You processed arrays by giving each thread one element. That worked great for small arrays. But what happens when your arrays get big? In this chapter, “scaling up” means one specific move: going from a single block to many, so the same kernel can handle arrays of any size.
Imagine you’re a teacher who just gave an exam to 10,000 students. You need to grade every single exam. If you sit down and grade them one by one, it’s going to be a very long weekend.
But what if you could open 40 classrooms, put 250 graders in each room, hand each room a stack of exams, and say “Go!”? They could all work at the same time, and the job would be done in a fraction of the time.