chapter two

2 Your first CUDA program

 

This chapter covers

  • Deconstructing your first CUDA kernel, vectorAdd, line by line
  • Understanding the core CUDA programming model: host versus device and data transfers
  • Mastering the GPU’s threading hierarchy: grids, blocks, and threads
  • Introducing essential C concepts like pointers as they appear in the code
  • Compiling and running your program with the nvcc compiler

We are going to take a vectorAdd kernel, put it under a microscope, and dissect it piece by piece. We won’t just look at the kernel itself; we’ll look at the entire program required to launch it: the “host” code that runs on your CPU. By the end of this chapter, you will have written, compiled, and run your very first complete CUDA program. You will understand not just what the code does, but why it’s structured the way it is.

2.1 The “why”: Throughput vs. latency

We’re going to peek into some code in a second, but first, the reason to use a GPU over a faster CPU comes down to two concepts throughput and latency:

2.2 Your first run

2.3 Anatomy of a CUDA program: The vecadd.cu file

2.4 Where the parallelism comes in

2.5 Scaling up: Grids, blocks, and boundary checks

2.6 Expanding to 3D arrays

2.6.1 3D tensor addition: CPU implementation

2.6.2 The CUDA kernel for 3D tensor addition

2.6.3 3D kernel launch configuration

2.6.4 Real-world example: Batch image processing

2.7 The universal pattern: From nD to 1D

Summary