Semantic caching
Semantic caching is a technique that stores and reuses results for requests judged sufficiently similar in meaning, rather than requiring identical text or keys.
Build an Advanced RAG Application (From Scratch)
A semantic cache is a store of prior query results indexed so that semantically similar incoming queries can retrieve and reuse them.
- How it is used
- Avoids repeated RAG processing for semantically equivalent enterprise queries, reducing latency, cost, and rate-limit pressure.
- How deep it goes
- Builds a FAISS-and-JSON cache through filtering, lookup, persistence, thresholds, and unified-pipeline integration; defers approximate indexes until larger scale.
- Examples
- Uses paraphrased Uber 2021 revenue questions to demonstrate a miss followed by a cache hit and measured speedup.
- What you can do after
- Build it from scratch as a persisted FAISS semantic cache with conservative freshness handling inside an Enterprise RAG pipeline.
Read how Build an Advanced RAG Application (From Scratch) explains semantic caching →
Building Reliable AI Systems
A caching technique that reuses results for queries with similar meanings, rather than requiring exact textual matches.
- How it is used
- Reduces redundant LLM calls for repeated customer-support questions, especially during traffic surges.
- How deep it goes
- Explains embedding, vector search, threshold-based retrieval, and cache-miss storage, then sketches integration into a service.
- Examples
- Uses store-hours and refund-policy questions, plus an e-commerce support service with queued requests.
- What you can do after
- Apply it via GPTCache in an LLM service with a configured embedding model and similarity threshold.
Read how Building Reliable AI Systems explains semantic caching →
Machine Learning Platform Engineering
Semantic caching is a caching method that reuses results for inputs judged equivalent or sufficiently similar in meaning rather than identical in text.
- How it is used
- Reduces repeated LLM calls, latency, and token costs for predictable question patterns in production systems.
- How deep it goes
- Explains embeddings, threshold trade-offs, freshness, semantic drift, and lookup overhead, then demonstrates vector-similarity implementation.
- Examples
- Uses paraphrased API-key questions and a RedisVL vector-similarity cache with configurable thresholds.
- What you can do after
- Build semantic lookup with embeddings and RedisVL, while tuning thresholds and managing response freshness.
Read how Machine Learning Platform Engineering explains semantic caching →
Also covered in
- Designing AI SystemsIn early access (MEAP)
What to know first
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- groundingGrounding is the practice of anchoring or constraining an AI system’s outputs with relevant information, context, or evidence.
- hallucinationA hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
- hit rateCache hit rate is the proportion of cache lookups satisfied by a usable result already stored in the cache.
Related concepts
- embeddinguses · An embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- FAISSuses · FAISS is an open-source software library for efficient similarity search and clustering of dense vectors.
- GPTCacheuses · GPTCache is an open-source library that caches large-language-model responses and can reuse semantically similar results.
- similarity searchuses · Similarity search is a retrieval technique that finds or ranks items by their similarity to a query, often by comparing feature or vector representations under a similarity or distance measure.
Questions about semantic caching
What is semantic caching?
Semantic caching is a technique that stores and reuses results for requests judged sufficiently similar in meaning, rather than requiring identical text or keys.
Which Manning books explain semantic caching?
Build an Advanced RAG Application (From Scratch), Building Reliable AI Systems and Machine Learning Platform Engineering.
What should I know before learning semantic caching?
Manning books that teach semantic caching usually introduce embedding, grounding, hallucination and hit rate first.
What is the difference between semantic caching and exact-match caching?
Semantic caching is a technique that stores and reuses results for requests judged sufficiently similar in meaning, rather than requiring identical text or keys. Exact-match caching is a caching method that reuses a stored result only when a new input exactly matches the associated cached input.
What is semantic caching also called?
Semantic caching is also called semantic cache.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


