The last-level cache (LLC) is the largest, slowest level of a CPU’s cache hierarchy before main memory. It matters because an access that misses the LLC may have to reach DRAM. Approximate access times are 1–2 nanoseconds for L1, 10 nanoseconds or more for the LLC, and up to 100 nanoseconds for DRAM. Those differences make data placement a low-latency concern even within one machine. section 2.3.2 section 3.1
Caches closer to the CPU are faster and smaller. An LLC can hold megabytes or tens of megabytes, but whether that helps a particular application depends on its working set: the data it needs during execution. Keeping that set comfortably within LLC capacity is a useful goal, not a guarantee of consistently fast accesses. Threads can compete for caches, and misses at faster levels still introduce latency variation. section 3.4 section 2.3.2
Use the LLC as a practical sizing and latency check:
- Estimate the application’s active working set and compare it with the machine’s LLC capacity. section 3.4
- When latency varies, consider whether data or instruction misses are forcing fetches from a slower cache level or DRAM. section 2.3.2
- Treat cache capacity as one part of placement: CPU and memory topology also affect how far an access must travel. section 3.1 section 3.4
A rough latency comparison
Suppose an access is served by the LLC at roughly 10 nanoseconds rather than by DRAM at roughly 100 nanoseconds. The difference is about an order of magnitude for that access. It does not translate directly into a 10× application speedup: the result depends on which accesses miss and how they affect execution. section 3.1 section 2.3.2
