liveBook · Manning

Chapter 2: Your first parallel program

What you will learn in chapter 2 of Grokking GPU Programming

This introduction previews the CUDA concepts covered in the chapter and the grading-room analogy that will guide the first program.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction previews the CUDA concepts covered in the chapter and the grading-room analogy that will guide the first program.

2.1 Hello, Parallel World!

Introduces a CUDA Hello World program in which eight threads print independently, along with basic CUDA terminology and the limits of the online judge.

Key terms: thread, parallel, printf, host, device, kernel, __global__, <<<blocks, threads>>>, cudaDeviceSynchronize(), nvcc

2.2 The headers: plugging in the toolkit

This section explains the roles of the example’s headers and why to include them explicitly outside the online judge.

Key terms: <stdio.h>, printf, <cuda_runtime.h>, CUDA runtime functions, cudaDeviceSynchronize(), swforces.com, nvcc, portability

2.3 The kernel function: a function that runs in parallel

This section explains how a CUDA kernel runs the same code across threads and how __global__ identifies a function called from the CPU to execute on the GPU.

Key terms: kernel, threads, launch, printf, __global__, host, device

2.4 The launch syntax: starting those threads

This section explains CUDA kernel launch syntax, how its configuration determines the threads launched, and the per-block thread limit.

Key terms: launch syntax, blocks, threads per block, kernel, triple angle brackets, launch marker, nvcc, cudaLaunchKernel, launch configuration, thread index

2.5 Synchronization: you have to wait

This section explains asynchronous CUDA launches, host-device synchronization, and related error reporting, then checks kernel return values and launch sizes.

Key terms: asynchronous, non-blocking, cudaDeviceSynchronize(), __syncthreads(), host, device, barrier, cudaGetLastError(), launch configuration, void

2.6 Each thread has its own job

This section explains how thread indices assign different data to threads running the same code and how thread layouts and block boundaries affect those indices.

Key terms: threadIdx.x, thread index, 1D layout, blockIdx.x, thread block, layout, 1D, 2D, 3D, dim3

2.7 Meet threadIdx

This section uses thread identity to filter kernel work, introduces SPMD and execution-order uncertainty, and describes multidimensional thread coordinates.

Key terms: threadIdx.x, kernel, thread index, if statement, thread execution order, warp, single program, multiple data, SPMD, data parallelism, task parallelism

2.8 How the hardware runs your threads: warps and SMs

This section introduces GPU execution in warps on streaming multiprocessors, distinguishes the programming and execution models, and revisits thread indexing.

Key terms: streaming multiprocessor, SM, warp, SIMT, lane, SPMD, threadIdx, threadIdx.x

2.9 Squaring an array

This section introduces parallel array squaring by assigning different data elements to threads using their indices.

Key terms: thread, index, integer array, kernel

2.10 The function signature

The section identifies the squaring kernel’s arguments and explains why its array pointer must refer to GPU memory.

Key terms: data, n, device memory, host memory, cudaMalloc, malloc

2.11 Thread index

This section explains how a thread selects its array element using threadIdx.x and why that index suffices for a single-block launch.

Key terms: threadIdx.x, idx, thread index, blockIdx.x, blockDim.x, single block

2.12 Bounds check

This section explains why GPU threads need a bounds check before accessing an array and what can happen without one.

Key terms: bounds check, if (idx < n), out-of-bounds access

2.13 The actual work

This section explains how threads square individual array elements and how a bounds check prevents invalid accesses.

Key terms: thread, idx, squareKernel, bounds check, if (idx < n), out of bounds, main function, array pointer, GPU

2.14 Moving data: CPU to GPU and back

This section explains the CPU–GPU data lifecycle for the squaring program and its costs and theoretical performance.

Key terms: host, device, host memory, device memory, CUDA data lifecycle, cudaMemcpyHostToDevice, cudaMemcpyDeviceToHost, h_data, d_data, overhead

2.15 Step 1: Allocate device memory (set up desks in the grading room)

This section explains how cudaMalloc reserves device memory and reports the allocation’s result.

Key terms: cudaMalloc, device memory, host data, pointer to a pointer, device address, error code, synchronous, cudaErrorMemoryAllocation

2.16 Step 2: Copy host to device (carry the exams to the students)

This section explains how to copy an array from host memory to reserved device memory using cudaMemcpy and the correct copy direction.

Key terms: cudaMemcpy, h_data, d_data, cudaMemcpyHostToDevice, cudaMemcpyDeviceToHost, undefined, error code

2.17 Step 3: Launch kernel (let the students grade)

This section launches the array-squaring kernel on device data and explains why explicit synchronization is useful but not required before a blocking device-to-host copy.

Key terms: kernel, launch syntax, squareKernel, thread, device memory, device pointer, host pointer, illegal memory access, cudaDeviceSynchronize(), asynchronous

2.18 Step 4: Copy device to host (collect the graded exams)

This section shows how to copy computed results from GPU device memory back to host memory.

Key terms: cudaMemcpy, host, device memory, cudaMemcpyDeviceToHost

2.19 Step 5: Free memory (clear the desks out of the room)

This section completes the GPU memory lifecycle with cleanup and reinforces the five-step pattern through application and debugging exercises.

Key terms: cudaFree, device memory, GPU memory, host, allocate, copy in, compute, copy out, free, doubleKernel

2.20 Common mistakes

This section introduces a quick-reference checklist of common mistakes and their remedies.

2.21 Mistake 1: Forgetting to synchronize

This section explains how asynchronous kernel launches can cause missing output, how to wait for completion, and why swforces.com can hide the mistake.

Key terms: asynchronous, cudaDeviceSynchronize(), blocking device-to-host cudaMemcpy, swforces.com, asynchronously

2.22 Mistake 2: Copying data the wrong way

This section explains how to recognize and fix a cudaMemcpy direction mismatch when sending data to the GPU.

Key terms: cudaMemcpy, cudaMemcpyHostToDevice, cudaMemcpyDeviceToHost, destination, source

2.23 Mistake 3: Missing bounds check

This section explains how to prevent and diagnose out-of-bounds errors in CUDA programs, including undersized host buffers and incorrect kernel bounds checks.

Key terms: bounds check, thread ID, array index, kernel parameter, host arrays, malloc, cudaMalloc, launch, heap, Wrong Answer

2.24 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 1: Why parallel programming? Chapter 3: Scaling up: from one block to many →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.