GPTCache wraps requests to GPT-based endpoints and reuses answers for exact or semantically similar prompts. A sufficiently similar cached request returns its stored answer; a miss calls the model and caches the result. This can reduce LLM calls, cost, and response time for repeated questions. section 9.2.3
import gptcache #1
from gptcache import cache #2
from gptcache.embedding import Onnx as DefaultEmbedding #3
from gptcache.adapter import openai as gptcache_openai #4
# 1. Initialize GPTCache
cache.init_embeddings(DefaultEmbedding()) #5
cache.set_similarity_threshold(0.75) #6
# 2. Wrap LLM calls
def get_cached_answer_gptcache(prompt): #7
response = gptcache_openai.ChatCompletion.create(
model="gpt-5.4-mini", #8
messages=[{"role": "user", "content": prompt}] #9
)
return response["choices"][0]["message"]["content"] #10
# 3. Use this in your code
answer = get_cached_answer_gptcache("When do you open?") #11
print(answer) #12
The Python example initializes ONNX embeddings, sets a similarity threshold of 0.75, and routes a ChatCompletion request through GPTCache’s OpenAI adapter. section 9.2.3
Choose an embedding function, similarity threshold, and local or remote storage. A close match can serve an existing answer; a poor match should reach the model or prompt for clarification. A cache hit saves a call, but similarity alone does not establish that the stored answer is appropriate. section 9.2.3
