liveBook · Manning

Retrieval-augmented generation

As explained in Context Engineering

A retrieval-augmented generation pipeline sequences retrieval and generation operations to supply retrieved information to a generative model.

Retrieval-augmented generation (RAG) supplies an LLM with external knowledge retrieved at query time. The model generates an answer using selected passages in its context alongside its parametric memory. This allows it to address changing, domain-specific, or proprietary information without retraining. section 3.2

A RAG system has two workflows:

  • Indexing runs ahead of requests, preparing external data and building a searchable index. section 3.2
  • Generation runs on the request path: it searches the index, adds selected material to the prompt, and sends the augmented prompt to the LLM. section 3.2

Minimal keyword-based RAG

A Python implementation indexes documents as sparse TF-IDF vectors. For each question, it uses the same vectorizer, ranks documents by cosine similarity, and inserts the highest-scoring matches into a prompt instructing the model to answer only from that context. The augmented prompt is then sent to an OpenAI model. section 3.10.1

Listing 3.1 Minimal RAG in Python: TF-IDF indexing, similarity-based retrieval, and augmented generation
corpus = [doc["text"] for doc in documents] #1
vectorizer = TfidfVectorizer(stop_words="english")
doc_vectors = vectorizer.fit_transform(corpus) #2

def retrieve(query: str, k: int = 3) -> List[Dict[str, Any]]: #3
    query_vec = vectorizer.transform([query])
    similarities = cosine_similarity(query_vec, doc_vectors)[0]
    top_indices = np.argsort(similarities)[::-1][:k]
    results: List[Dict[str, Any]] = []
    for idx in top_indices:
        results.append({
            "id": documents[idx]["id"],
            "title": documents[idx]["title"],
            "score": float(similarities[idx]),
            "text": documents[idx]["text"],
        })
    return results

def query_model_with_rag(question: str,
                k: int = 3,
                model: str = "gpt-4o-mini") -> str: #4
    retrieved_docs = retrieve(question, k=k)
    context = build_context(retrieved_docs)
    prompt = (
        "You are a helpful assistant ...\n"
        "Documentation:\n"
        f"{context}\n\n"
        f"User question: {question}"
    )
    # Query model (see full source code in GitHub)

Inspect the TF-IDF indexing and similarity-based retrieval code for the minimal pipeline. section 3.10.1

Grounding means factual claims are supported by material placed in the context; it does not guarantee accuracy or establish that the sources are trustworthy. Reliability also depends on source freshness and authority, chunking, ranking, permission filters, and prompt assembly. section 3.1 section 3.2

Retrieval selects passages dynamically, unlike cache-augmented generation, which preloads reusable context, or direct injection, which loads complete documents into the context window. Agentic RAG extends retrieval with an agent that can reformulate queries, select tools, and iterate over intermediate results. section 3.0 section 3.1

Local RAG stack

Ollama can run the LLM and embedding model locally, Qdrant can store and query embeddings, and Agno can orchestrate the pipeline. One configuration ingests a PDF manual into Qdrant, but fetches that manual from a remote URL at startup, so it is not fully offline as written. section 3.10.3

Listing 3.3 RAG system implemented in Python, using an Ollama model, Qdrant as a vector database, and Agno as an agent framework
from agno.agent import Agent
from agno.models.ollama import Ollama
from agno.knowledge.knowledge import Knowledge
from agno.vectordb.qdrant import Qdrant
from agno.knowledge.embedder.ollama import OllamaEmbedder

vector_db = Qdrant( #1
    collection="my-vector-db",
    url="http://localhost:6333",
    embedder=OllamaEmbedder(id="nomic-embed-text"),
)
knowledge_base = Knowledge(vector_db=vector_db) #2
agent = Agent( #3
    name="Local RAG",
    model=Ollama(id="llama3.2:1b"),
    knowledge=knowledge_base,
)

Inspect the configuration connecting the local model, embedder, vector database, and agent. section 3.10.3

How Context Engineering covers it

How deep it goes
Explains offline indexing and online generation, then implements sparse retrieval and configures a local vector-backed pipeline.
Examples
Uses TF-IDF document retrieval and a locally ingested PDF manual to illustrate alternative pipelines.
What you can do after
Apply it using scikit-learn and the OpenAI API, or configure a local pipeline with Ollama, Qdrant, and Agno.
How it connects
Connects external knowledge, parametric memory, grounding, and agentic retrieval.

Also explained in

Compare how 46 books explain retrieval-augmented generation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.