Augmentation adds retrieved information to a model’s input before generation. In a retrieval-augmented generation (RAG) pipeline, a prompt template combines the user’s question with selected evidence, giving the model external context for its answer. section 3.2.2 section 3.10.1
For document retrieval, the template receives the original text of the highest-scoring chunks, often with metadata such as document names, sections, or URLs. Vectors serve as a matching mechanism; they do not enter the prompt. Retrieval top-K controls how much content is passed into the context window, balancing coverage against irrelevant noise. A reranker can reorder candidates before augmentation. section 3.2.2
Minimal document RAG
A TF-IDF vectorizer transforms a document collection and a user question into sparse vectors. Cosine similarity identifies the highest-scoring documents; their text is inserted into a prompt that instructs the model to answer only from the supplied context. The augmented prompt is then sent to the model. section 3.10.1
Listing 3.1 Minimal RAG in Python: TF-IDF indexing, similarity-based retrieval, and augmented generation
corpus = [doc["text"] for doc in documents] #1
vectorizer = TfidfVectorizer(stop_words="english")
doc_vectors = vectorizer.fit_transform(corpus) #2
def retrieve(query: str, k: int = 3) -> List[Dict[str, Any]]: #3
query_vec = vectorizer.transform([query])
similarities = cosine_similarity(query_vec, doc_vectors)[0]
top_indices = np.argsort(similarities)[::-1][:k]
results: List[Dict[str, Any]] = []
for idx in top_indices:
results.append({
"id": documents[idx]["id"],
"title": documents[idx]["title"],
"score": float(similarities[idx]),
"text": documents[idx]["text"],
})
return results
def query_model_with_rag(question: str,
k: int = 3,
model: str = "gpt-4o-mini") -> str: #4
retrieved_docs = retrieve(question, k=k)
context = build_context(retrieved_docs)
prompt = (
"You are a helpful assistant ...\n"
"Documentation:\n"
f"{context}\n\n"
f"User question: {question}"
)
# Query model (see full source code in GitHub)
Inspect the Python indexing and similarity-based retrieval steps that supply context for augmented generation. section 3.10.1
In Graph RAG, augmentation converts a retrieved subgraph into model-readable facts, often triples. For a dependency question, a triple might identify a company and the product it manufactures. Facts must be ranked and pruned to fit the context window, while preserving clear entities and relationships. The resulting context supports multi-hop reasoning, but reliability depends on graph quality, extraction accuracy, and query formulation. section 3.3.2
Keep context construction explicit enough to inspect, test, and debug. Layering prompt templates, retrieval, memory, and tools behind multiple abstractions can make it difficult to identify the source of a wrong answer. section 10.3
