BM25 (Best Match 25) is a probabilistic ranking function that scores documents using term frequency, inverse document frequency, and document-length normalization. It is widely used in production search systems and RAG pipelines. section 6.2.1
Use BM25 as a sparse retrieval method when terminology matters:
- Prefer it for fast, interpretable retrieval with strong performance on precise terminology. section 6.2.1
- Expect failures when relevant documents use paraphrases or synonyms rather than the query’s terms. section 6.2.1 section 6.5
- Evaluate retrieval with precision@k, recall@k, and AP or MAP, especially when ranking affects the context-window cutoff. section 6.2.1 section 6.5
Choosing retrieval by query type
An agent can select a BM25 index for a query involving precise terminology, while using dense search for conceptual questions. Comparing the resulting precision and recall helps expose each method’s failure mode. section 6.2.1 section 6.5
Table 6.2 Retrieval approaches and their failure modes
|
Retrieval method
|
Strength
|
Failure modes
|
Useful diagnostic signals
|
|
Grep / ripgrep
|
Exact, deterministic matching
|
Lexical variation; exact matches that are not task relevant
|
Recall across term variations; precision of exact matches
|
|
Sparse / TF-IDF / BM25
|
Fast, interpretable, and strong on
terminology
|
Paraphrases and synonyms
|
precision@k, recall@k, AP/MAP
|
|
Dense / vector search
|
Semantic similarity, handles paraphrases
|
Precise identifiers, exact terms
|
recall@k, nDCG with graded relevance, hybrid-vs-dense comparisons
|
|
Hybrid retrieval
|
Combines sparse and dense signals
|
Fusion weights, duplicate results, conflicting rankings
|
precision@k, recall@k, MAP, nDCG
|
Compare BM25 with exact matching, dense vector search, and hybrid retrieval through their strengths, failure modes, and diagnostic metrics. section 6.2.1






