L2 cache
An L2 cache is a second-level hardware cache that stores copies of data to reduce memory-access costs and is typically larger and slower than L1 cache.
GPU Programming with Triton
An L2 cache is a hardware-managed GPU cache that is generally larger and farther from compute units than the L1 cache.
- How it is used
- Uses L2 to explain GPU memory traffic and data reuse in performance-sensitive kernels.
- How deep it goes
- Explains hardware-managed caching, shared placement and traffic paths, with H100 capacity and bandwidth figures.
- Examples
- Illustrates L2 placement on the H100 and uses grouped program IDs in matrix multiplication to encourage reuse.
- What you can do after
- Evaluate kernel memory behavior by considering cache reuse and L2 bandwidth alongside DRAM traffic.
Grokking GPU Programming
An L2 cache is a hardware-managed cache that stores copies of data to reduce the cost of memory access.
- How it is used
- Explains GPU memory-access costs by showing how L2 hits and misses affect trips to DRAM.
- How deep it goes
- Places L2 in the memory hierarchy, gives latency and sharing details, and contrasts hits with slower DRAM misses.
- Examples
- Uses a warp of 32 threads reading unrelated locations to illustrate scattered accesses and cache-line traffic.
- What you can do after
- Diagnose why global-memory misses can make a GPU kernel slow by comparing L2 and DRAM access costs.
Also covered in
What to know first
- threadA thread is a distinct sequence of execution with its own control flow and state, managed or scheduled by a processor, runtime, or operating system.
- float16float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats.
- GPU memoryGPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
Related concepts
- CPU cachekind of · A CPU cache is small, fast memory that stores copies of data or instructions for quicker access by a processor.
- memory hierarchypart of · A memory hierarchy is an arrangement of storage levels with differing capacities, access speeds, latencies, costs, or persistence.
- GPU memoryuses · GPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
- scattered memory access patternaddresses · A scattered memory access pattern occurs when threads access memory locations that are not close together, often requiring separate memory transactions.
Questions about L2 cache
What is L2 cache?
An L2 cache is a second-level hardware cache that stores copies of data to reduce memory-access costs and is typically larger and slower than L1 cache.
Which Manning books explain L2 cache?
GPU Programming with Triton and Grokking GPU Programming.
What should I know before learning L2 cache?
Manning books that teach L2 cache usually introduce thread, float16, GPU memory and mean first.
What is L2 cache a kind of?
L2 cache is a kind of CPU cache. A CPU cache is small, fast memory that stores copies of data or instructions for quicker access by a processor.
What is L2 cache part of?
L2 cache is part of memory hierarchy. A memory hierarchy is an arrangement of storage levels with differing capacities, access speeds, latencies, costs, or persistence.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

