Cache miss

As explained in Latency

A cache miss occurs when requested data or instructions are absent from the current cache level and must be fetched from slower storage.

A cache miss occurs when a lookup cannot find the requested value at the current cache level. The request must then reach a slower source. For an application cache, that source is usually a backing store; for a CPU cache, it may be another cache level or DRAM. A hit avoids that slower access. section 2.3.2 section 6.2

Misses matter because access times differ sharply. L1 cache access can take 1–2 nanoseconds, last-level cache access 10 nanoseconds or more, and DRAM access up to 100 nanoseconds. When a working set does not fit in L1, misses introduce latency variance that can affect tail latency. Instruction misses can also stall execution while the CPU waits for the fetch. section 2.3.2

For an application cache, inspect both the miss path and how often it runs:

• Measure the hit-to-miss ratio. A high miss ratio means more slow lookups or computations; cache lookup overhead can even make the application slower than it would be without caching. section 6.2
• Decide who handles a miss. With read-through caching, the cache fetches the key from the backing store, updates itself, and returns the value; the miss is transparent to the application. section 6.3.2
• Account for capacity. Once a cache is full, adding data on a miss requires eviction. A replacement policy chooses what to remove, and its effect on hit ratio depends on the workload. section 6.6

Caching an absent result

A customer-search API may repeatedly receive filters that match no records. Without a cached answer, each lookup repeats the expensive search. Negative caching stores the fact that no result exists, so a later request can avoid that work. section 6.2

Figure 6.2 With read-through caching, the client looks up a key from the cache. Unlike with cache-aside caching, the read-through cache queries the database and updates itself on a cache miss.

Follow a read-through miss from the client's key lookup through the cache's database read and cache update. section 6.3.2

Read-through caching simplifies application-side cache management, but moves backing-store access and data-format conversion into the cache. Its miss path still has tail-latency implications; refresh-ahead can update values asynchronously before they expire. section 6.3.2

How Latency covers it

How deep it goes

Defines misses and explains CPU latency, read-through handling, and eviction consequences without specifying a measurement implementation.

Examples

Uses customer searches with no results and a key lookup that triggers a read-through database fetch.

What it assumes

Introduces cache hits and misses from a key lookup while assuming familiarity with application data stores and basic CPU memory terminology.

What you can do after

Evaluate/understand cache effectiveness through the hit-to-miss ratio and account for miss handling and eviction when choosing a caching strategy.

How it connects

Connects cache misses to cache hits, tail latency, read-through caching, negative caching, and cache replacement.

liveBookmanning.com

Read Latency on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  100 Go Mistakes and How to Avoid ThemA cache miss occurs when requested data is not present in the relevant cache and must be obtained from a lower-level memory source.

Compare how 2 books explain cache miss →

Related concepts

• cachingcontrasts with · Caching is the practice of storing reusable data or computation results so later access can avoid repeating retrieval or computation.
• compulsory missincludes · A compulsory miss is a cache miss caused by the first access to a memory block that has not yet been loaded into the cache.
• heapproduced by · A runtime memory region from which dynamically allocated data is obtained and whose storage is managed independently of individual function call frames.
• databaseuses · A database is an organized collection of persistent data managed by software for storage, retrieval, querying, and modification.

This explainer was generated by Manning's knowledge graph from the text of Latency. The section links point to the parts of the book it draws on.