Latency is the time one job takes from start to finish. Throughput measures how much work gets done per unit of time. CPUs are designed for low latency; GPUs accept higher latency per task to achieve high throughput across many tasks. section 1.6
Table 1.2 Contrasting CPU and GPU design philosophies.
|
|
CPU (Master
Chef)
|
GPU
(Thousand Cooks)
|
|
Cores
|
6–24 on
typical desktops
|
Thousands
of simple processing units
|
|
Design Goal
|
Low Latency
(Finish each task fast)
|
High
Throughput (Finish many tasks at once)
|
|
Analogy
|
A chef
preparing a complex 7-course meal
|
1,000
workers rolling 1,000 burritos at once
|
|
Best For
|
OS, Web
Servers, Complex Logic, and any single-threaded task where one fast core wins
|
Matrix
Math, Image Processing, AI, and many more parallel patterns you’ll discover
in this book
|
Compare the CPU’s focus on finishing one task quickly with the GPU’s focus on doing many tasks at once. section 1.6











