CUDA (Compute Unified Device Architecture) is NVIDIA’s platform for writing general-purpose code that runs on GPUs. It makes GPU computing available for tasks beyond graphics, including image processing, matrix math, and AI. section 1.6 section 1.10
A CUDA program uses the CPU as the host and the GPU as the device: the CPU runs the main program, while the GPU takes on large parallel jobs. A kernel is a function launched to run that work on the GPU; threads use indices to identify their work, and data may need to move between host and device memory. section 1.0 section 2.0
A GPU is useful when a task offers many independent, similar operations. Small datasets, branch-heavy logic, or step-by-step dependencies may favor the CPU instead. Keeping data on the GPU can also matter: unnecessary CPU–GPU transfers can slow a workload. section 1.10 section 1.7
Grade the exams in parallel
Imagine 1,000 students each grading one exam. A CUDA kernel can express the instruction for grading the exam in front of each worker, illustrating how a large job can be divided into parallel work. section 2.0
Table 1.2 Contrasting CPU and GPU design philosophies.
|
|
CPU (Master
Chef)
|
GPU
(Thousand Cooks)
|
|
Cores
|
6–24 on
typical desktops
|
Thousands
of simple processing units
|
|
Design Goal
|
Low Latency
(Finish each task fast)
|
High
Throughput (Finish many tasks at once)
|
|
Analogy
|
A chef
preparing a complex 7-course meal
|
1,000
workers rolling 1,000 burritos at once
|
|
Best For
|
OS, Web
Servers, Complex Logic, and any single-threaded task where one fast core wins
|
Matrix
Math, Image Processing, AI, and many more parallel patterns you’ll discover
in this book
|
Compare the CPU’s low-latency design with the GPU’s high-throughput design and the kinds of work each suits. section 1.6






