Similarity search retrieves documents that are semantically relevant to a query, even when they use different wording. Dense vector search is useful for conceptual questions and paraphrases, but it can miss precise identifiers such as error codes or contract clause numbers. section 6.2.1 section 6.5
In retrieval-augmented generation (RAG), retrieved documents enter the model’s context so it can answer using external knowledge. Retrieval and generation can fail independently: a model may summarize irrelevant documents faithfully, or hallucinate despite receiving relevant ones. section 6.1 section 6.1.1
Choose and evaluate retrieval methods against the question being asked:
- Use dense vector search for semantic similarity and paraphrases; check whether it misses exact terms or identifiers. section 6.2.1 section 6.5
- Use BM25 for precise terminology and grep or ripgrep for exact strings or regular expressions. Exact matches are not necessarily relevant to the task. section 6.2.1
- Consider hybrid retrieval to combine sparse and dense signals. Check for poor fusion weights, duplicates, and conflicting rankings. section 6.2.1 section 6.5
- Measure precision and recall, then inspect ranking with metrics such as precision@k and nDCG. Relevant documents ranked below the context-window cutoff cannot influence generation. section 6.5 section 6.2.1
Diagnosing a retrieval gap
A financial assistant must answer what drove Q3 revenue growth and what risks could affect Q4. If the retrieved context covers the growth driver but not the risks, a faithful answer may still be incomplete. Check context recall and investigate whether the retrieval method missed relevant documents before changing generation. section 6.0 section 6.5
Table 6.2 Retrieval approaches and their failure modes
|
Retrieval method
|
Strength
|
Failure modes
|
Useful diagnostic signals
|
|
Grep / ripgrep
|
Exact, deterministic matching
|
Lexical variation; exact matches that are not task relevant
|
Recall across term variations; precision of exact matches
|
|
Sparse / TF-IDF / BM25
|
Fast, interpretable, and strong on
terminology
|
Paraphrases and synonyms
|
precision@k, recall@k, AP/MAP
|
|
Dense / vector search
|
Semantic similarity, handles paraphrases
|
Precise identifiers, exact terms
|
recall@k, nDCG with graded relevance, hybrid-vs-dense comparisons
|
|
Hybrid retrieval
|
Combines sparse and dense signals
|
Fusion weights, duplicate results, conflicting rankings
|
precision@k, recall@k, MAP, nDCG
|
Compare retrieval approaches, their failure modes, and useful diagnostic metrics. section 6.2.1




































