When a GPU kernel reads global memory, the request passes through the L2 cache before reaching DRAM. A cache hit avoids that trip; a miss can cost about 400–600 cycles. section 11.1
L2 is a hardware-managed cache shared by all streaming multiprocessors (SMs). It sits below the L1 cache in the memory hierarchy: example latencies are about 100–200 cycles for L2 and 28–50 for L1. Large L2 caches can also partially absorb scattered accesses to small arrays. section 10.7 section 11.1
Figure 11.1 The GPU memory hierarchy as a mountain. Each level up is faster but smaller: global memory at the base offers gigabytes at ~400–600 cycles, while registers at the peak respond in ~1 cycle but hold only a handful of values per thread. Everything above the dashed line lives on-chip, close to the cores; global memory sits off-chip in DRAM.

Use the hierarchy diagram to compare L2 with the faster, smaller levels above it and global memory below. section 11.1
Scattered reads
If 32 threads read unrelated locations, they may touch 32 different cache lines. A large L2 can partly help when the scattered data fits there, but the access pattern can still require many transactions. section 10.7

