A retrieve-then-generate pipeline answers a question by fetching relevant passages and giving them to a language model as context. Its simplest form is single-pass: one retrieval, one generation, and no feedback loop. It is a practical baseline when questions closely match a well-structured knowledge base, but a wrong retrieval can become a wrong answer. section 1.5.1 section 11.1
The basic workflow separates preparation from answering:
- Prepare the knowledge base: split documents into chunks, create embeddings with a pretrained model, and store their vectors. section 1.5.1
- At query time, retrieve the most similar chunks and pass their text to a language model to generate an answer. section 1.5.1
When a larger top-k changes the answer
A pipeline built over a PDF split into 155 chunks retrieved three passages for a question about which retriever the paper used. The relevant passage was absent, so it could not answer; retrieving five passages worked. Increasing top-k may help, but it does not establish that the approach will scale or reliably select valid evidence. section 1.5.1
Listing 11.1 The retrieval trace.
{
"trace_id": "req_5592_compliance", #A
"query": "Can we onboard a PEP from the EU using expedited KYC?",
"retrieval_step": {
"method": "Dense Vector Search (Cosine Similarity)",
"top_k": 3,
"results": [
{
"rank": 1,
"doc_id": "internal_policy_2020",
"score": 0.94, #B
"content": "Global KYC Policy 2020... For PEPs, "
"expedited KYC is permitted...",
"status": "SELECTED", #C
"error_type": "FP1: Staleness (Superseded by 2024 Directive)"
},
{
"rank": 2,
"doc_id": "hr_holiday_policy",
"score": 0.76,
"content": "Employees traveling to the EU...",
"status": "IGNORED",
"error_type": "Noise"
},
{
"rank": 3,
"doc_id": "eu_directive_2024",
"score": 0.72, #D
"content": "EU Directive 2024... prohibits simplified "
"due diligence for PEPs...",
"status": "IGNORED (Cutoff)",
"error_type": "FP2: Missed Top Rank (Semantic gap)"
}
]
},
"generation_step": {
"input_context": "Global KYC Policy 2020...",
"output": "Yes, expedited KYC is permitted for PEPs "
"under the Global Policy.",
"compliance_verdict": "VIOLATION" #E
}
}
The compliance retrieval trace makes a different failure visible: an outdated 2020 policy scores 0.94 for an expedited-KYC query, while a 2024 EU directive scores 0.72. Similarity alone does not check which document is current or authoritative. section 11.1.2
The pipeline is fast to prototype, relatively inexpensive, and easier to debug because retrieval and generation can be evaluated separately. Its limits are structural: missing or stale content, vocabulary mismatches, low-ranked evidence, and questions requiring facts from several documents may leave the model with inadequate context. Curating the knowledge base and filtering by metadata can help; neither gives a single-pass system a way to assess a failed search and try again. Fixed multi-query retrieval broadens the search but still lacks that adaptive loop. For consequential or multi-part questions, corrective evaluation or planning-first retrieval addresses the control-flow gap. section 1.5.1 section 11.1.3 section 11.1.5 section 11.0

