2 Your first parallel program
This chapter covers
- The structure of a parallel CUDA program
- How to write a kernel function with __global__
- How to launch parallel work with <<<…>>>
- How threads use threadIdx to identify themselves
- How the hardware runs your threads: warps, SMs, and CUDA cores
- How to transfer data between the host (CPU) and the device (GPU)
You met the grading room in Chapter 1: one teacher, a stack of 1,000 worksheets, and 1,000 students who could each grade one. In this chapter the analogy stops being a picture and becomes a program. By the end, you’ll have written the “grade the sheet in front of you” instruction in CUDA and submitted your first solutions to swforces.com. All you need on hand is your swforces.com account from Chapter 1 and basic C: functions, loops, and a little comfort with pointers (malloc and free will make a brief appearance).
Hello, Parallel World!
Every programmer starts with “Hello, World.” Ours has a twist: instead of printing the message once, we’ll have eight threads print it at the same time. But what is a thread? Picture those 1,000 students again: each student is one thread, a single worker that runs your code on its own small slice of the work. A parallel program launches many threads at once, and every one of them runs the same code independently.