Chapter 2: Your first parallel program
This introduction previews the CUDA concepts covered in the chapter and the grading-room analogy that will guide the first program.
After this chapter you will be able to
- Independent parallel threads. Explain how eight threads produce the program's output and what is not guaranteed about their execution order.
- Host and device roles. Distinguish the host and device in a CUDA program.
- CUDA execution building blocks. Describe the basic roles of __global__, the launch syntax, and cudaDeviceSynchronize().
- CUDA program header roles. Explain what <stdio.h> and <cuda_runtime.h> declare in the example program.
- Parallel kernel execution. Explain how launching a kernel repeats its code across threads rather than using a loop in the kernel body. Apply this idea to the eight-thread greeting example.
- Kernel function qualifier. Explain what __global__ tells the compiler about where a CUDA function is typically called and where it executes.
- Kernel launch configuration. Explain the two launch parameters and contrast launching a kernel with calling an ordinary C function.
- Asynchronous launches and waiting. Explain how a kernel launch overlaps CPU and GPU work and why the CPU may need to synchronize before using results or ending the program.
- Host and thread synchronization. Distinguish cudaDeviceSynchronize() from the block-level thread barrier __syncthreads().
- Thread indices and individual work. Explain how threads running the same code can work on different elements using threadIdx.x in a one-dimensional block.
- Index uniqueness within blocks. Explain where threadIdx.x is unique and what happens to its values when there are multiple blocks.
- Thread identity and filtering. Trace how four threads run the same kernel while only thread 2 prints, using threadIdx.x and a condition.
- SPMD and data parallelism. Explain SPMD and relate it to data parallelism, distinguishing it from task parallelism.
- GPU execution hierarchy. Describe how SMs and warps execute threads and relate that behavior to SIMT.
- Automatic warp packing. Explain how launched threads are packed into warps and what happens when only eight threads are launched in the example.
- Indexed parallel array squaring. Explain how threads can use their indices to work on different elements while squaring an integer array in parallel.
- Kernel arguments and device memory. Describe the kernel’s two arguments—an integer-array pointer named data and an array size named n—and explain why data must point to device memory rather than host memory.
- Thread index and assignment. Explain how each thread uses threadIdx.x to identify the array element it handles.
- Single-block index simplification. Explain why threadIdx.x alone gives the full index for the single-block launch shown here.
- Thread bounds check. Explain how checking if (idx < n) before computation prevents extra threads from accessing nonexistent array elements, and describe the consequences of omitting it.
- Per-thread array computation. Explain how the same kernel statement squares a different array element in each thread, and describe what changing the expression to data[idx] + 1 would do.
- Bounds check protection. Explain what can happen when extra threads access an array without a bounds check, and trace the guarded empty-array case.
- Host and device memory. Explain why data must be moved between host and device memory before GPU threads can process it and results can return to the CPU.
- CUDA data lifecycle. Trace the five-step CUDA data lifecycle in order, distinguishing the two copy directions.
- GPU overhead and payoff. Explain when GPU parallelism can outweigh the costs of allocation and data movement, and how retaining data on the GPU can reduce repeated transfers.
- Device memory allocation. Explain what cudaMalloc(&d_data, n * sizeof(int)) reserves and what state the memory is in afterward.
- Pointer and return code. Explain why the example passes &d_data to cudaMalloc rather than assigning its return value to d_data.
- Host to device copy. Explain how the example uses cudaMemcpy to give the GPU its own copy of the host array.
- Parallel kernel launch. Describe the squareKernel launch and what the GPU threads do with the data.
- Synchronization and blocking copy. Explain why cudaDeviceSynchronize() is used after the asynchronous launch and why it is not strictly required before the following blocking copy.
- Device to host copy. Explain how the example uses cudaMemcpy to bring the computed array back from device memory to host memory.
- Device memory cleanup. Explain when and why to release device memory with cudaFree.
- Five-step GPU lifecycle. Describe the five steps of the host-side GPU workflow in order and how the pattern can be reused for a different per-element computation.
- Kernel completion and synchronization. Explain why a host program can miss kernel output and how to wait for the kernel before relying on its completion.
- Copy direction and destination. Explain how to match a cudaMemcpy direction to its destination and source, and describe what can happen when the direction is wrong.
- Kernel array bounds checks. Explain why extra threads can access an array out of bounds and how to guard their accesses using the actual element count.
- Host and device sizing. Explain how undersized host arrays can corrupt the heap and how host allocation, device allocation, and launch size must cover the input.
What it builds on
- Your lab: swforces.com (chapter 1) · Introduces the swforces.com lab.
- A family of parallel approaches (chapter 1) · The distinction between data and task parallelism.
- Sequential vs. parallel thinking (chapter 1) · The distinction between sequential and parallel thinking.
Section by section
Introduction
This introduction previews the CUDA concepts covered in the chapter and the grading-room analogy that will guide the first program.
2.1 Hello, Parallel World!
Introduces a CUDA Hello World program in which eight threads print independently, along with basic CUDA terminology and the limits of the online judge.
- Independent parallel threads: Explain how eight threads produce the program's output and what is not guaranteed about their execution order.
Why it matters: This is the basic shift from one sequential execution to many independent workers. - Host and device roles: Distinguish the host and device in a CUDA program.
Why it matters: The distinction lets you describe which part of a CUDA program runs on which processor. - CUDA execution building blocks: Describe the basic roles of __global__, the launch syntax, and cudaDeviceSynchronize().
Why it matters: These are the basic pieces that identify, start, and wait for the parallel work. - CUDA tools and judging: Describe what nvcc does and distinguish an online judge pass from running on a physical GPU.
Why it matters: A successful logic check is useful, but it is not a GPU performance test. - Production error checking: State what error checks the book's short examples omit and what real code should check.
Why it matters: Short teaching examples should not be mistaken for a complete production error-handling practice.
Key terms: thread, parallel, printf, host, device, kernel, __global__, <<<blocks, threads>>>, cudaDeviceSynchronize(), nvcc
2.2 The headers: plugging in the toolkit
This section explains the roles of the example’s headers and why to include them explicitly outside the online judge.
- CUDA program header roles: Explain what <stdio.h> and <cuda_runtime.h> declare in the example program.
Why it matters: The headers make the functions used by the example available to the program. - Explicit header inclusion portability: Explain why the book explicitly includes both headers even though some environments add them automatically.
Why it matters: Explicit includes make complete programs less dependent on conveniences of a particular environment.
Key terms: <stdio.h>, printf, <cuda_runtime.h>, CUDA runtime functions, cudaDeviceSynchronize(), swforces.com, nvcc, portability
2.3 The kernel function: a function that runs in parallel
This section explains how a CUDA kernel runs the same code across threads and how __global__ identifies a function called from the CPU to execute on the GPU.
- Parallel kernel execution: Explain how launching a kernel repeats its code across threads rather than using a loop in the kernel body. Apply this idea to the eight-thread greeting example.
Why it matters: This is the basic shift from repeating work in a loop to running the same instructions across GPU threads. - Kernel function qualifier: Explain what __global__ tells the compiler about where a CUDA function is typically called and where it executes.
Why it matters: The qualifier identifies a function as GPU-executed code typically invoked from the host.
Key terms: kernel, threads, launch, printf, __global__, host, device
2.4 The launch syntax: starting those threads
This section explains CUDA kernel launch syntax, how its configuration determines the threads launched, and the per-block thread limit.
- Kernel launch configuration: Explain the two launch parameters and contrast launching a kernel with calling an ordinary C function.
Why it matters: Reading a launch configuration tells you how many threads run the kernel. - Launch marker compiler handling: Describe what the triple angle brackets mean to the CUDA compiler.
Why it matters: This separates the syntax marker from the configuration values inside it. - Launch configuration and work: Explain how launch configuration relates to distributing work across data.
Why it matters: The launch size and thread indices connect parallel execution to the data being processed. - Threads per block limit: State the per-block thread limit given in the section and explain its implication for larger workloads.
Why it matters: The block limit motivates moving beyond a single block for larger workloads.
Key terms: launch syntax, blocks, threads per block, kernel, triple angle brackets, launch marker, nvcc, cudaLaunchKernel, launch configuration, thread index
2.5 Synchronization: you have to wait
This section explains asynchronous CUDA launches, host-device synchronization, and related error reporting, then checks kernel return values and launch sizes.
- Asynchronous launches and waiting: Explain how a kernel launch overlaps CPU and GPU work and why the CPU may need to synchronize before using results or ending the program.
Why it matters: Waiting at the right point keeps the CPU from moving past work the GPU has not finished. - Host and thread synchronization: Distinguish cudaDeviceSynchronize() from the block-level thread barrier __syncthreads().
Why it matters: The two synchronization calls coordinate different participants and are not interchangeable. - Asynchronous error reporting: Distinguish when a kernel execution error and an outright launch failure are reported.
Why it matters: Knowing when errors surface helps you identify whether a launch or its execution failed. - Kernel void return rationale: Explain why a kernel returns void and where it writes results instead.
Why it matters: This explains why kernel results are written to memory rather than returned like a single ordinary function value. - Launch configuration thread count: Apply the launch dimensions in the quick check to explain why both configurations produce eight output lines.
Why it matters: Counting threads across blocks helps you predict how much work a launch starts.
Key terms: asynchronous, non-blocking, cudaDeviceSynchronize(), __syncthreads(), host, device, barrier, cudaGetLastError(), launch configuration, void
2.6 Each thread has its own job
This section explains how thread indices assign different data to threads running the same code and how thread layouts and block boundaries affect those indices.
- Thread indices and individual work: Explain how threads running the same code can work on different elements using threadIdx.x in a one-dimensional block.
Why it matters: This is how identical kernel code can give each thread a different piece of work. - Index uniqueness within blocks: Explain where threadIdx.x is unique and what happens to its values when there are multiple blocks.
Why it matters: Knowing the scope of a thread index prevents you from mistaking threads in different blocks for the same thread. - Thread layout and dimensions: Describe thread block layouts, how they are chosen at launch, and how threadIdx components relate to their dimensions.
Why it matters: A suitable layout helps relate the arrangement of threads to the shape of the data.
Key terms: threadIdx.x, thread index, 1D layout, blockIdx.x, thread block, layout, 1D, 2D, 3D, dim3
2.7 Meet threadIdx
This section uses thread identity to filter kernel work, introduces SPMD and execution-order uncertainty, and describes multidimensional thread coordinates.
- Thread identity and filtering: Trace how four threads run the same kernel while only thread 2 prints, using threadIdx.x and a condition.
Why it matters: Thread-specific indices let a shared kernel assign different work to different threads. - Thread execution order: Explain why output order should not be assumed when all threads print, and contrast real hardware with the section's swforces.com behavior.
Why it matters: A consistent-looking printout is not evidence that CUDA guarantees that execution order. - SPMD and data parallelism: Explain SPMD and relate it to data parallelism, distinguishing it from task parallelism.
Why it matters: This model underlies the data-parallel GPU programs developed throughout the book. - Multidimensional thread coordinates: Describe how 2D and 3D block layouts give threads coordinates suited to the shape of their data.
Why it matters: Matching thread coordinates to data dimensions makes images, matrices, and volumes easier to address.
Key terms: threadIdx.x, kernel, thread index, if statement, thread execution order, warp, single program, multiple data, SPMD, data parallelism, task parallelism
2.8 How the hardware runs your threads: warps and SMs
This section introduces GPU execution in warps on streaming multiprocessors, distinguishes the programming and execution models, and revisits thread indexing.
- GPU execution hierarchy: Describe how SMs and warps execute threads and relate that behavior to SIMT.
Why it matters: This hierarchy explains how the GPU executes the threads you launch. - Automatic warp packing: Explain how launched threads are packed into warps and what happens when only eight threads are launched in the example.
Why it matters: Warp packing shows why launching fewer than 32 threads can leave execution slots unused. - Programming and execution models: Distinguish SPMD from SIMT as presented in the section.
Why it matters: The distinction keeps the program's organization separate from the hardware's execution model. - Thread index components: Explain why the example uses threadIdx.x rather than threadIdx alone.
Why it matters: Selecting a component gives the example a usable index from the block's thread layout.
Key terms: streaming multiprocessor, SM, warp, SIMT, lane, SPMD, threadIdx, threadIdx.x
2.9 Squaring an array
This section introduces parallel array squaring by assigning different data elements to threads using their indices.
- Indexed parallel array squaring: Explain how threads can use their indices to work on different elements while squaring an integer array in parallel.
Why it matters: Using thread indices to divide data is a foundation for parallel array operations.
Key terms: thread, index, integer array, kernel
2.10 The function signature
The section identifies the squaring kernel’s arguments and explains why its array pointer must refer to GPU memory.
- Kernel arguments and device memory: Describe the kernel’s two arguments—an integer-array pointer named data and an array size named n—and explain why data must point to device memory rather than host memory.
Why it matters: It connects the kernel’s inputs to the distinction between CPU and GPU memory.
Key terms: data, n, device memory, host memory, cudaMalloc, malloc
2.11 Thread index
This section explains how a thread selects its array element using threadIdx.x and why that index suffices for a single-block launch.
- Thread index and assignment: Explain how each thread uses threadIdx.x to identify the array element it handles.
Why it matters: This connects each parallel thread to its own piece of the array. - Single-block index simplification: Explain why threadIdx.x alone gives the full index for the single-block launch shown here.
Why it matters: This explains why the shorter indexing expression works in the current example.
Key terms: threadIdx.x, idx, thread index, blockIdx.x, blockDim.x, single block
2.12 Bounds check
This section explains why GPU threads need a bounds check before accessing an array and what can happen without one.
- Thread bounds check: Explain how checking if (idx < n) before computation prevents extra threads from accessing nonexistent array elements, and describe the consequences of omitting it.
Why it matters: A bounds check keeps extra threads from accessing memory outside the array.
Key terms: bounds check, if (idx < n), out-of-bounds access
2.13 The actual work
This section explains how threads square individual array elements and how a bounds check prevents invalid accesses.
- Per-thread array computation: Explain how the same kernel statement squares a different array element in each thread, and describe what changing the expression to data[idx] + 1 would do.
Why it matters: This shows how identical thread code can perform work on different pieces of data. - Bounds check protection: Explain what can happen when extra threads access an array without a bounds check, and trace the guarded empty-array case.
Why it matters: The guard prevents threads without a matching element from accessing memory outside the array. - Array availability on GPU: State what the main function must do before this array kernel can touch the array.
Why it matters: The kernel cannot operate on the array until the array is available on the GPU.
Key terms: thread, idx, squareKernel, bounds check, if (idx < n), out of bounds, main function, array pointer, GPU
2.14 Moving data: CPU to GPU and back
This section explains the CPU–GPU data lifecycle for the squaring program and its costs and theoretical performance.
- Host and device memory: Explain why data must be moved between host and device memory before GPU threads can process it and results can return to the CPU.
Why it matters: The memory separation explains why a GPU program needs more than a kernel. - CUDA data lifecycle: Trace the five-step CUDA data lifecycle in order, distinguishing the two copy directions.
Why it matters: This sequence provides the overall structure for a CUDA program that processes data. - GPU overhead and payoff: Explain when GPU parallelism can outweigh the costs of allocation and data movement, and how retaining data on the GPU can reduce repeated transfers.
Why it matters: A fast kernel alone does not guarantee that the whole GPU program will be faster. - Ideal parallel squaring complexity: Distinguish the ideal parallel step count for squaring an array from the costs and limits of a real GPU program.
Why it matters: The ideal step count highlights parallelism without implying that the entire real program takes constant time. - Single-block launch limits: State the valid range of n for the example launch <<<1, n>>> and explain why larger arrays need a different launch arrangement.
Why it matters: The example launch cannot handle every array size as written.
Key terms: host, device, host memory, device memory, CUDA data lifecycle, cudaMemcpyHostToDevice, cudaMemcpyDeviceToHost, h_data, d_data, overhead
2.15 Step 1: Allocate device memory (set up desks in the grading room)
This section explains how cudaMalloc reserves device memory and reports the allocation’s result.
- Device memory allocation: Explain what cudaMalloc(&d_data, n * sizeof(int)) reserves and what state the memory is in afterward.
Why it matters: Reserving space is the first step before putting data on the GPU. - Pointer and return code: Explain why the example passes &d_data to cudaMalloc rather than assigning its return value to d_data.
Why it matters: Understanding the output pointer keeps the device address distinct from the call’s return code. - Allocation completion and failure: Describe when cudaMalloc returns and what it reports if the device runs out of memory.
Why it matters: The call’s completion and error status tell you whether the allocation succeeded.
Key terms: cudaMalloc, device memory, host data, pointer to a pointer, device address, error code, synchronous, cudaErrorMemoryAllocation
2.16 Step 2: Copy host to device (carry the exams to the students)
This section explains how to copy an array from host memory to reserved device memory using cudaMemcpy and the correct copy direction.
- Host to device copy: Explain how the example uses cudaMemcpy to give the GPU its own copy of the host array.
Why it matters: The kernel needs its input in device memory before the GPU can work on it. - Copy direction and errors: Explain why the cudaMemcpy direction must match the source and destination, and describe what can happen if it does not.
Why it matters: A wrong direction can leave the destination without the data you expected.
Key terms: cudaMemcpy, h_data, d_data, cudaMemcpyHostToDevice, cudaMemcpyDeviceToHost, undefined, error code
2.17 Step 3: Launch kernel (let the students grade)
This section launches the array-squaring kernel on device data and explains why explicit synchronization is useful but not required before a blocking device-to-host copy.
- Parallel kernel launch: Describe the squareKernel launch and what the GPU threads do with the data.
Why it matters: The launch connects the CPU's instruction to the parallel work performed on the GPU. - Device pointer kernel argument: Explain why the kernel receives d_data rather than h_data and what can happen if the wrong pointer is passed.
Why it matters: Passing the right pointer lets the kernel work on the data placed on the device. - Synchronization and blocking copy: Explain why cudaDeviceSynchronize() is used after the asynchronous launch and why it is not strictly required before the following blocking copy.
Why it matters: Understanding both wait points prevents unnecessary synchronization while making errors easier to locate.
Key terms: kernel, launch syntax, squareKernel, thread, device memory, device pointer, host pointer, illegal memory access, cudaDeviceSynchronize(), asynchronous
2.18 Step 4: Copy device to host (collect the graded exams)
This section shows how to copy computed results from GPU device memory back to host memory.
- Device to host copy: Explain how the example uses cudaMemcpy to bring the computed array back from device memory to host memory.
Why it matters: Copying the results back makes the GPU's computed values available on the host.
Key terms: cudaMemcpy, host, device memory, cudaMemcpyDeviceToHost
2.19 Step 5: Free memory (clear the desks out of the room)
This section completes the GPU memory lifecycle with cleanup and reinforces the five-step pattern through application and debugging exercises.
- Device memory cleanup: Explain when and why to release device memory with cudaFree.
Why it matters: Freeing device memory lets later programs use it instead of leaving it allocated unnecessarily. - Five-step GPU lifecycle: Describe the five steps of the host-side GPU workflow in order and how the pattern can be reused for a different per-element computation.
Why it matters: The five-step pattern gives you a reusable structure for GPU programs. - Missing input transfer diagnosis: Explain how a missing host-to-device copy can produce wrong output without a crash and how to investigate the lifecycle.
Why it matters: Checking the entire lifecycle can reveal transfer mistakes even when a program runs without crashing. - Separate host and device memory: Explain why a kernel using a device pointer cannot simply read an ordinary malloc-allocated host array in this example.
Why it matters: Understanding the separate memory spaces explains why the input transfer is necessary.
Key terms: cudaFree, device memory, GPU memory, host, allocate, copy in, compute, copy out, free, doubleKernel
2.20 Common mistakes
This section introduces a quick-reference checklist of common mistakes and their remedies.
2.21 Mistake 1: Forgetting to synchronize
This section explains how asynchronous kernel launches can cause missing output, how to wait for completion, and why swforces.com can hide the mistake.
- Kernel completion and synchronization: Explain why a host program can miss kernel output and how to wait for the kernel before relying on its completion.
Why it matters: Waiting at the right point prevents the host from moving on before GPU work is complete. - Swforces execution caveat: Explain why successful output without explicit synchronization on swforces.com does not establish that the program is safe on a real GPU.
Why it matters: A successful test on the learning platform can hide a synchronization bug that appears on real hardware.
Key terms: asynchronous, cudaDeviceSynchronize(), blocking device-to-host cudaMemcpy, swforces.com, asynchronously
2.22 Mistake 2: Copying data the wrong way
This section explains how to recognize and fix a cudaMemcpy direction mismatch when sending data to the GPU.
- Copy direction and destination: Explain how to match a cudaMemcpy direction to its destination and source, and describe what can happen when the direction is wrong.
Why it matters: Matching the direction to the pointers helps prevent a copy that appears to run but leaves your program using wrong data.
Key terms: cudaMemcpy, cudaMemcpyHostToDevice, cudaMemcpyDeviceToHost, destination, source
2.23 Mistake 3: Missing bounds check
This section explains how to prevent and diagnose out-of-bounds errors in CUDA programs, including undersized host buffers and incorrect kernel bounds checks.
- Kernel array bounds checks: Explain why extra threads can access an array out of bounds and how to guard their accesses using the actual element count.
Why it matters: A correct bounds check prevents extra threads from corrupting memory without silently skipping valid elements. - Host and device sizing: Explain how undersized host arrays can corrupt the heap and how host allocation, device allocation, and launch size must cover the input.
Why it matters: Correct GPU-side sizes cannot protect a host buffer that is too small for the input. - Submission error messages: Distinguish the meanings of Wrong Answer, Runtime Error, and Time Limit Exceeded on swforces.com.
Why it matters: The reported failure helps you decide what kind of problem to investigate. - Independent thread mental model: Describe the independent-thread model for this chapter and its limitation.
Why it matters: The one-thread-per-element analogy fits these examples but should not be mistaken for a universal rule.
Key terms: bounds check, thread ID, array index, kernel parameter, host arrays, malloc, cudaMalloc, launch, heap, Wrong Answer
2.24 Summary
Where this is used later
- One thread, one element (chapter 4) · A kernel is a function that runs in parallel on the GPU.
- The three-level hierarchy: grids, blocks, and threads (chapter 3) · The hierarchy expands the relationship between blocks and threads.
- What’s wrong with one block? (chapter 3) · The limitations of a single block motivate scaling to multiple blocks.
- The multi-block kernel recipe (chapter 3) · Synchronization establishes a point at which GPU work has completed.
- The barrier: __syncthreads() (chapter 5) · Uses the distinction when introducing the thread barrier.
- Where you’ve seen Map before (chapter 4) · Each thread has its own job.
- Why the thread index alone isn’t enough (chapter 3) · Explains why a thread index alone is insufficient with multiple blocks.
- The global index formula (chapter 3) · Uses thread and block information to form a global index.
Concepts explained in this chapter
- cudaDeviceSynchronizecudaDeviceSynchronize is a CUDA runtime API function that blocks the host until previously issued work on the device has completed.
- threadIdx.xthreadIdx.x is a CUDA-provided per-thread coordinate identifying a thread's position along the x dimension within its block.
- CUDA data lifecycleThe CUDA data lifecycle is the five-stage GPU workflow of allocating device memory, copying data in, computing, copying results out, and freeing device memory.
- CUDA kernelA CUDA kernel is a function launched on a GPU to execute concurrently across many GPU threads.
- global function qualifier__global__ is a CUDA function qualifier that declares a function callable from the host and executable on the device as a kernel.
- threadA thread is a sequence of instructions that performs a unit of computation.
- data parallelismData parallelism performs the same operation concurrently on multiple pieces of data.
- bounds checkingA bounds check is a conditional test that verifies an index lies within a data structure's valid range before access or computation.
- CUDA kernel launch syntaxCUDA launch syntax specifies the execution configuration used to launch a kernel, including its blocks and threads.
- SPMDSPMD is a programming model in which one program is executed by many parallel threads, with each thread operating on different data.
- streaming multiprocessorA streaming multiprocessor is a GPU hardware unit that schedules and executes groups of GPU threads using its execution resources.
- warpA warp is a GPU execution group whose threads typically execute the same instruction concurrently on different data.
- thread layoutA thread layout is the arrangement of threads in a CUDA block along one, two, or three axes.
- cudaMemcpycudaMemcpy is a CUDA runtime API function that copies data between host and device memory or between device memory regions.
- block sizeThreads per block is the configured number of GPU threads assigned to each CUDA thread block in a kernel launch.
- GPU memoryDevice memory is memory located on a GPU and accessible to GPU computations.
- synchronizationSynchronization is the coordination of concurrent computations so that their execution observes required ordering or completion constraints.
- cudaMalloccudaMalloc is a CUDA runtime API function that allocates memory on a GPU device.
- CUDACUDA is NVIDIA's parallel computing platform and programming model for executing general-purpose computations on GPUs.
- nvccnvcc is NVIDIA's CUDA compiler driver for compiling CUDA source files into executable programs.
- thread execution orderThread execution order is the sequence in which concurrently running threads reach and execute operations.
- two-dimensional thread layoutA 2D thread layout arranges the threads of a CUDA block along two dimensions, giving each thread two coordinates.
- SIMTSIMT is a GPU execution model in which groups of threads execute instructions together while operating on separate data.
- thread blockA block is a CUDA launch hierarchy unit whose threads are grouped and scheduled for execution on GPU hardware.
- asynchronous programmingAsynchronous execution allows an operation to proceed without requiring the caller to wait for its completion before continuing.
- kernel launchA kernel launch starts execution of a GPU kernel across a specified number of threads.
- out-of-bounds array accessAn out-of-bounds access is an attempt to read or write outside the valid range of a data structure or memory allocation.
- CPUA CPU is a general-purpose processor that executes program instructions.
- GPUA GPU is a processor designed to execute many operations concurrently, especially data-parallel operations.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
