Caching speeds up data retrieval by keeping temporary copies closer to where they are accessed. A backing store, such as a database, retains the primary copy. A lookup by cache key is a hit when the copy is available; on a miss, the value must come from the backing store. Reusing results avoids some of that retrieval delay, but the copies consume storage and may become stale. section 6.0 section 6.2
Consider caching when a key–value lookup is enough, changing or replicating the existing system is difficult, or storing the whole dataset near its users is impractical. User profiles, session state, and selected product-catalog entries can fit this pattern; workloads needing complex queries may not. An in-memory client cache can also avoid a network hop, at the cost of more application memory. section 6.1 section 6.3.5
Choose how the cache interacts with its backing store, then account for the limits of the copy:
- Select a read/write strategy—cache-aside, read-through, write-through, or write-behind—according to who fetches a missing value and how writes propagate. The choices differ in latency and complexity. section 6.3 section 6.11
- Track the hit ratio: a cache that frequently misses still pays for backing-store access. When capacity is full, an eviction policy such as LRU, LFU, or FIFO determines what leaves; the best choice depends on the workload. section 6.2 section 6.6
- Set a time-to-live (TTL) with care. A short TTL can lower the hit ratio; a long one can serve stale data. Multiple cache copies also raise coherence questions unless the application tolerates incoherence. section 6.11
Cache a database-backed API response
A REST API can check an in-application Moka cache before querying PostgreSQL. On a hit, it returns the cached result. On a miss, it queries the database, inserts the result, and returns it. A five-second TTL bounds each entry’s validity. Comparing latency eCDFs with and without caching shows lower latency overall, but database misses leave a heavy tail. section 6.10
Listing 6.3 HTTP request handler
const CACHE_KEY: &str = "RESULT_KEY";
async fn say_hello(
data: web::Data<Arc<AppState>>,
) -> anyhow::Result<String, Error> {
if let Some(result) = data.cache.get(CACHE_KEY) {
Ok(result)
} else {
let result = fetch_value(&data.pool).await?;
data.cache.insert(CACHE_KEY.to_owned(), result.to_owned());
Ok(result)
}
}
The request handler makes the cache-aside path concrete: look up a key, fetch from PostgreSQL on a miss, then insert the result. section 6.10
Keep persistence and placement in mind. An in-memory cache normally loses its entries on restart and must warm up again; a persistent cache can avoid that cold start but adds complexity and possible consistency issues. A separate cache server requires a network trip, while a client-side cache sits in the application. Measure the resulting latency rather than assuming every cache placement helps equally. section 6.2 section 6.3.5
