liveBook · Manning

Hybrid search

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Hybrid retrieval combines multiple retrieval signals, typically sparse lexical matching and dense vector similarity, to retrieve relevant information.

Hybrid search combines sparse lexical retrieval with dense vector retrieval. Sparse methods such as BM25 are strong at matching precise terminology and identifiers; dense retrieval handles paraphrases and conceptual similarity. Combining them helps cover their complementary failure modes. section 6.2.1

A hybrid system can run both retrieval methods and merge their ranked results using Reciprocal Rank Fusion (RRF). Fusion weights, duplicate results, and conflicting rankings can still cause problems, so the combined ranking needs evaluation. section 6.2.1

Identifiers and paraphrases

A query containing a product SKU or error code may be missed by dense retrieval if the embedding model treats the identifier as an arbitrary token. Conversely, sparse retrieval can miss a relevant document that uses different terminology. Hybrid search combines signals suited to both cases. section 6.2.1 section 6.4.1

Diagnose and tune retrieval before judging generation:

  • If context recall is low but faithfulness is high, investigate whether retrieval is missing relevant documents. A dense system missing identifiers is a candidate for hybrid retrieval. section 6.4.1
  • If context precision is low, irrelevant documents may crowd the context window. For a hybrid system, adjust RRF weighting toward the more precise signal. section 6.4.1
  • Use precision@k, recall@k, MAP, or nDCG to assess the retrieved ranking. Relevant documents ranked below the context window cutoff cannot influence generation. section 6.2.1 section 6.5

Hybrid search cannot compensate for every upstream failure. A poorly formulated query can produce poor results regardless of the retrieval method. section 6.2.1

Table 6.2 Retrieval approaches and their failure modes
Retrieval method
Strength
Failure modes
Useful diagnostic signals
Grep / ripgrep
Exact, deterministic matching
Lexical variation; exact matches that are not task relevant
Recall across term variations; precision of exact matches
Sparse / TF-IDF / BM25
Fast, interpretable, and strong on   terminology
Paraphrases and synonyms
precision@k, recall@k, AP/MAP
Dense / vector search
Semantic similarity, handles paraphrases
Precise identifiers, exact terms
recall@k, nDCG with graded relevance, hybrid-vs-dense comparisons
Hybrid retrieval
Combines sparse and dense signals
Fusion weights, duplicate results, conflicting rankings
precision@k, recall@k, MAP, nDCG

Compare retrieval methods, their failure modes, and useful diagnostic signals. section 6.2.1

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains complementary failure modes, ranked-list fusion, and diagnostic tuning without specifying an RRF formula or implementation.
Examples
Contrasts product SKUs and error codes with queries and documents using different terminology.
What you can do after
Diagnose retrieval gaps and adjust fusion weighting using context recall and precision signals.
How it connects
Connects sparse retrieval, dense retrieval, Reciprocal Rank Fusion, context precision, and context recall.

Also explained in

Compare how 15 books explain hybrid search →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.