liveBook · Manning

Memory bandwidth

As explained in Grokking GPU Programming

Memory bandwidth is the rate at which data can be transferred between a processor and memory.

Memory bandwidth is the rate at which data moves between a processor and memory. In GPU kernels, it can become the bottleneck: how threads access global memory may matter more than how much computation they perform. section 10.0

When more threads do not help

A kernel launched with 10,000 threads is correct and well organized, yet barely faster than its CPU counterpart. The missing speedup points to how its threads read memory. section 10.0

How Grokking GPU Programming covers it

How deep it goes
Connects bandwidth limits to global-memory access patterns and previews arithmetic intensity without providing worked calculations.
Examples
Uses a correct 10,000-thread kernel that barely outperforms a CPU version to illustrate the bottleneck.
What you can do after
Understand why a highly parallel GPU kernel can still run slowly when its threads access memory poorly.
How it connects
Connects memory bandwidth to arithmetic intensity, global-memory access patterns, and data layout.

Also explained in

Compare how 4 books explain memory bandwidth →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.