An L1 cache is the smallest, fastest cache level closest to a CPU core. It avoids slower cache levels and DRAM, typically providing access in about 1–2 nanoseconds. section 3.4 section 2.3.2
Its speed comes with limited capacity: modern L1 caches hold roughly 100–200 KiB, while the LLC reaches tens of MiB. When a working set exceeds L1, data or instructions may arrive from slower levels, creating latency variance and worsening tail latency. section 3.4 section 2.3.2
Practical implications include:
- Keep latency-sensitive working sets compact, while recognizing that fitting within the larger LLC may be the more realistic optimization target. section 3.4
- Account for instruction-cache misses as well as data-cache misses; stalled instruction fetches can significantly delay execution. section 2.3.2
- Treat cache behavior as a trade-off rather than an absolute guarantee: speculative execution, SMT contention, and shared cache resources can alter latency. section 2.3.2
Latency-sensitive access
A memory access may take about 1 nanosecond from a fast cache, around 10 nanoseconds from the LLC, or up to 100 nanoseconds from DRAM, depending on cache state. section 2.3.2
Figure 3.4 UMA and NUMA architectures. The left side of the figure represents a UMA architecture where all CPU cores have the same access latency to DRAM. The right side represents a NUMA architecture with two nodes. CPUs that are in the same node can access NUMA node local memory faster than memory in a remote node.
Use the UMA/NUMA diagram to place L1-cache behavior within the broader physical data-placement and locality picture. section 3.4
