A cache hit occurs when a lookup finds the requested value in the cache; a miss requires retrieval from the backing store. Hit rate, or cache hit ratio, is hits divided by total lookups: hits / (hits + misses). It measures how much of a particular workload the cache serves. At 100%, every lookup hits; at 0%, every lookup misses. section 6.2 section 6.5
A higher hit rate usually reduces latency because it avoids slower access to primary storage. It is not a speed guarantee: each cache lookup has overhead, and a workload with many misses can run slower than it would without caching. The useful question is therefore how the ratio behaves for the workload being served, not whether a cache exists. section 6.2 section 6.5
Translate hit rate into latency
Suppose 80% of lookups hit in 0.1 seconds and 20% miss in 1.0 seconds. The weighted average is 0.8 × 0.1 + 0.2 × 1.0 = 0.28 seconds. The average falls as the hit rate rises toward 100%, but the remaining misses still form a slower part of the latency distribution. section 6.5
To use hit rate when tuning a cache:
- Count hits and misses over the workload of interest, then calculate hits / (hits + misses). Check latency as well: improving the ratio is valuable when it avoids costly backing-store accesses. section 6.5
- Examine capacity and replacement policy. A cache smaller than its dataset must evict entries; the best policy depends on access patterns. LRU can retain useful entries when recently accessed data is likely to be reused, but low locality can make its overhead counterproductive. section 6.6 section 6.6.1
- Check expiration alongside freshness. A short time-to-live can cause avoidable misses; a long one can leave stale data after the backing store changes. section 6.7
Figure 6.6 A cache latency distribution when the hit ratio is 80% with a cache hit latency of 0.1 seconds and a cache miss latency of 1.0 seconds
The latency distribution makes the consequence of an 80% hit rate visible: fast hits coexist with slower misses. section 6.5
