liveBook · Manning

Latency

As explained in Grokking GPU Programming

Latency is the time required to complete one operation or job.

Latency is the time one job takes from start to finish. Throughput measures how much work gets done per unit of time. CPUs are designed for low latency; GPUs accept higher latency per task to achieve high throughput across many tasks. section 1.6

Table 1.2 Contrasting CPU and GPU design philosophies.
 
CPU (Master Chef)
GPU (Thousand Cooks)
Cores
6–24 on typical desktops
Thousands of simple processing units
Design Goal
Low Latency (Finish each task fast)
High Throughput (Finish many tasks at once)
Analogy
A chef preparing a complex 7-course meal
1,000 workers rolling 1,000 burritos at once
Best For
OS, Web Servers, Complex Logic, and any single-threaded task where one fast core wins
Matrix Math, Image Processing, AI, and many more parallel patterns you’ll discover in this book

Compare the CPU’s focus on finishing one task quickly with the GPU’s focus on doing many tasks at once. section 1.6

How Grokking GPU Programming covers it

How deep it goes
Defines latency and contrasts it with throughput, then relates both measures to CPU and GPU design goals.
Examples
Contrasts a CPU handling chess-engine logic with a GPU applying a filter to many pixels, and likens their designs to chefs and burrito makers.
What you can do after
Evaluate whether small, branch-heavy, or dependent work favors a CPU or independent, regular parallel work favors a GPU.
How it connects
Connects latency to throughput, CPUs, GPUs, and the host–device relationship.

Also explained in

Compare how 12 books explain latency →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.