liveBook · Manning

Semantic caching

Also called semantic cache

Semantic caching is a technique that stores and reuses results for requests judged sufficiently similar in meaning, rather than requiring identical text or keys.

How 3 Manning books explain it

Build an Advanced RAG Application (From Scratch)

A semantic cache is a store of prior query results indexed so that semantically similar incoming queries can retrieve and reuse them.

How it is used
Avoids repeated RAG processing for semantically equivalent enterprise queries, reducing latency, cost, and rate-limit pressure.
How deep it goes
Builds a FAISS-and-JSON cache through filtering, lookup, persistence, thresholds, and unified-pipeline integration; defers approximate indexes until larger scale.
Examples
Uses paraphrased Uber 2021 revenue questions to demonstrate a miss followed by a cache hit and measured speedup.
What you can do after
Build it from scratch as a persisted FAISS semantic cache with conservative freshness handling inside an Enterprise RAG pipeline.

Read how Build an Advanced RAG Application (From Scratch) explains semantic caching →

Building Reliable AI Systems

A caching technique that reuses results for queries with similar meanings, rather than requiring exact textual matches.

How it is used
Reduces redundant LLM calls for repeated customer-support questions, especially during traffic surges.
How deep it goes
Explains embedding, vector search, threshold-based retrieval, and cache-miss storage, then sketches integration into a service.
Examples
Uses store-hours and refund-policy questions, plus an e-commerce support service with queued requests.
What you can do after
Apply it via GPTCache in an LLM service with a configured embedding model and similarity threshold.

Read how Building Reliable AI Systems explains semantic caching →

Machine Learning Platform Engineering

Semantic caching is a caching method that reuses results for inputs judged equivalent or sufficiently similar in meaning rather than identical in text.

How it is used
Reduces repeated LLM calls, latency, and token costs for predictable question patterns in production systems.
How deep it goes
Explains embeddings, threshold trade-offs, freshness, semantic drift, and lookup overhead, then demonstrates vector-similarity implementation.
Examples
Uses paraphrased API-key questions and a RedisVL vector-similarity cache with configurable thresholds.
What you can do after
Build semantic lookup with embeddings and RedisVL, while tuning thresholds and managing response freshness.

Read how Machine Learning Platform Engineering explains semantic caching →

Also covered in

What to know first

Related concepts

Questions about semantic caching

What is semantic caching?

Semantic caching is a technique that stores and reuses results for requests judged sufficiently similar in meaning, rather than requiring identical text or keys.

Which Manning books explain semantic caching?

Build an Advanced RAG Application (From Scratch), Building Reliable AI Systems and Machine Learning Platform Engineering.

What should I know before learning semantic caching?

Manning books that teach semantic caching usually introduce embedding, grounding, hallucination and hit rate first.

What is the difference between semantic caching and exact-match caching?

Semantic caching is a technique that stores and reuses results for requests judged sufficiently similar in meaning, rather than requiring identical text or keys. Exact-match caching is a caching method that reuses a stored result only when a new input exactly matches the associated cached input.

What is semantic caching also called?

Semantic caching is also called semantic cache.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.