liveBook · Manning

Retrieve-then-generate pipeline

As explained in Retrieval Augmented Generation, The Foundational Ideas

Single-pass RAG is a retrieval-augmented generation architecture that performs one retrieval pass and then generates an answer without iterative planning or correction.

A retrieve-then-generate pipeline answers a question by fetching relevant passages and giving them to a language model as context. Its simplest form is single-pass: one retrieval, one generation, and no feedback loop. It is a practical baseline when questions closely match a well-structured knowledge base, but a wrong retrieval can become a wrong answer. section 1.5.1 section 11.1

The basic workflow separates preparation from answering:

  • Prepare the knowledge base: split documents into chunks, create embeddings with a pretrained model, and store their vectors. section 1.5.1
  • At query time, retrieve the most similar chunks and pass their text to a language model to generate an answer. section 1.5.1

When a larger top-k changes the answer

A pipeline built over a PDF split into 155 chunks retrieved three passages for a question about which retriever the paper used. The relevant passage was absent, so it could not answer; retrieving five passages worked. Increasing top-k may help, but it does not establish that the approach will scale or reliably select valid evidence. section 1.5.1

Listing 11.1 The retrieval trace.
{
  "trace_id": "req_5592_compliance",  #A
  "query": "Can we onboard a PEP from the EU using expedited KYC?",
  "retrieval_step": {
    "method": "Dense Vector Search (Cosine Similarity)",
    "top_k": 3,
    "results": [
      {
        "rank": 1,
        "doc_id": "internal_policy_2020",
        "score": 0.94,  #B
        "content": "Global KYC Policy 2020... For PEPs, "
                   "expedited KYC is permitted...",
        "status": "SELECTED",  #C
        "error_type": "FP1: Staleness (Superseded by 2024 Directive)"
      },
      {
        "rank": 2,
        "doc_id": "hr_holiday_policy",
        "score": 0.76,
        "content": "Employees traveling to the EU...",
        "status": "IGNORED",
        "error_type": "Noise"
      },
      {
        "rank": 3,
        "doc_id": "eu_directive_2024",
        "score": 0.72,  #D
        "content": "EU Directive 2024... prohibits simplified "
                   "due diligence for PEPs...",
        "status": "IGNORED (Cutoff)",
        "error_type": "FP2: Missed Top Rank (Semantic gap)"
      }
    ]
  },
  "generation_step": {
    "input_context": "Global KYC Policy 2020...",
    "output": "Yes, expedited KYC is permitted for PEPs "
              "under the Global Policy.",
    "compliance_verdict": "VIOLATION"  #E
  }
}
 

The compliance retrieval trace makes a different failure visible: an outdated 2020 policy scores 0.94 for an expedited-KYC query, while a 2024 EU directive scores 0.72. Similarity alone does not check which document is current or authoritative. section 11.1.2

The pipeline is fast to prototype, relatively inexpensive, and easier to debug because retrieval and generation can be evaluated separately. Its limits are structural: missing or stale content, vocabulary mismatches, low-ranked evidence, and questions requiring facts from several documents may leave the model with inadequate context. Curating the knowledge base and filtering by metadata can help; neither gives a single-pass system a way to assess a failed search and try again. Fixed multi-query retrieval broadens the search but still lacks that adaptive loop. For consequential or multi-part questions, corrective evaluation or planning-first retrieval addresses the control-flow gap. section 1.5.1 section 11.1.3 section 11.1.5 section 11.0

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains indexing and query-time steps, then diagnoses ranking, freshness, and multi-hop failures; defers adaptive correction and planning.
Examples
Uses a 155-chunk PDF retrieval test and a compliance trace where an outdated policy outranks a newer directive.
What it assumes
Builds on document chunking, embeddings, vector search, and language-model generation without requiring an advanced mathematical treatment.
What you can do after
Operate/configure it by adjusting top-k and curating evidence, while evaluating retrieval and generation separately.
How it connects
Connects embeddings and vector search to language-model generation, and contrasts fixed retrieval with CRAG, LevelRAG, and agentic RAG.

Also explained in

Compare how 2 books explain retrieve-then-generate pipeline →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.