liveBook · Manning

L2 cache

As explained in Grokking GPU Programming

An L2 cache is a hardware-managed cache that stores copies of data to reduce the cost of memory access.

When a GPU kernel reads global memory, the request passes through the L2 cache before reaching DRAM. A cache hit avoids that trip; a miss can cost about 400–600 cycles. section 11.1

L2 is a hardware-managed cache shared by all streaming multiprocessors (SMs). It sits below the L1 cache in the memory hierarchy: example latencies are about 100–200 cycles for L2 and 28–50 for L1. Large L2 caches can also partially absorb scattered accesses to small arrays. section 10.7 section 11.1

Figure 11.1 The GPU memory hierarchy as a mountain. Each level up is faster but smaller: global memory at the base offers gigabytes at ~400–600 cycles, while registers at the peak respond in ~1 cycle but hold only a handful of values per thread. Everything above the dashed line lives on-chip, close to the cores; global memory sits off-chip in DRAM.

Use the hierarchy diagram to compare L2 with the faster, smaller levels above it and global memory below. section 11.1

Scattered reads

If 32 threads read unrelated locations, they may touch 32 different cache lines. A large L2 can partly help when the scattered data fits there, but the access pattern can still require many transactions. section 10.7

How Grokking GPU Programming covers it

How deep it goes
Places L2 in the memory hierarchy, gives latency and sharing details, and contrasts hits with slower DRAM misses.
Examples
Uses a warp of 32 threads reading unrelated locations to illustrate scattered accesses and cache-line traffic.
What you can do after
Diagnose why global-memory misses can make a GPU kernel slow by comparing L2 and DRAM access costs.
How it connects
Connects L2 cache with global memory, DRAM, L1 cache, shared memory, and streaming multiprocessors.

Also explained in

Compare how 2 books explain L2 cache →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.