liveBook · Manning

CUDA

As explained in Grokking GPU Programming

CUDA is NVIDIA's parallel computing platform and programming model for executing general-purpose computations on GPUs.

CUDA (Compute Unified Device Architecture) is NVIDIA’s platform for writing general-purpose code that runs on GPUs. It makes GPU computing available for tasks beyond graphics, including image processing, matrix math, and AI. section 1.6 section 1.10

A CUDA program uses the CPU as the host and the GPU as the device: the CPU runs the main program, while the GPU takes on large parallel jobs. A kernel is a function launched to run that work on the GPU; threads use indices to identify their work, and data may need to move between host and device memory. section 1.0 section 2.0

A GPU is useful when a task offers many independent, similar operations. Small datasets, branch-heavy logic, or step-by-step dependencies may favor the CPU instead. Keeping data on the GPU can also matter: unnecessary CPU–GPU transfers can slow a workload. section 1.10 section 1.7

Grade the exams in parallel

Imagine 1,000 students each grading one exam. A CUDA kernel can express the instruction for grading the exam in front of each worker, illustrating how a large job can be divided into parallel work. section 2.0

Table 1.2 Contrasting CPU and GPU design philosophies.
 
CPU (Master Chef)
GPU (Thousand Cooks)
Cores
6–24 on typical desktops
Thousands of simple processing units
Design Goal
Low Latency (Finish each task fast)
High Throughput (Finish many tasks at once)
Analogy
A chef preparing a complex 7-course meal
1,000 workers rolling 1,000 burritos at once
Best For
OS, Web Servers, Complex Logic, and any single-threaded task where one fast core wins
Matrix Math, Image Processing, AI, and many more parallel patterns you’ll discover in this book

Compare the CPU’s low-latency design with the GPU’s high-throughput design and the kinds of work each suits. section 1.6

How Grokking GPU Programming covers it

How deep it goes
Defines CUDA and outlines kernels, launches, thread indexing, and data transfers, while only previewing detailed execution mechanics.
Examples
Uses exam grading, matrix math, image processing, AI training, and an image pipeline to illustrate parallel work and transfer costs.
What it assumes
Assumes basic C functions and loops, with some pointer comfort helpful; introduces needed pointer techniques and requires no GPU.
What you can do after
Build it from scratch by writing and launching a CUDA kernel, identifying threads, and transferring data between CPU and GPU.
How it connects
Connects GPU kernels with CPU host-device work, memory transfers, parallel patterns, PyTorch, memory hierarchy, and synchronization.

Also explained in

Compare how 7 books explain CUDA →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.