A retrieval-augmented chatbot can have the right documents and still answer poorly if search fails to surface them. Hybrid search combines retrieval approaches so that one method’s misses need not determine the final candidate set. A common design pairs semantic vector search with keyword search. section 4.0 section 4.3.2
A practical dense–sparse pipeline has four stages:
- Run BM25 over tokenized documents to find exact terms, such as a location or policy wording. section 4.3.2
- Run embedding similarity search to find related meanings even when the query uses different words. section 4.3.2
- Merge the candidate lists and remove duplicates. section 4.3.2
- Optionally rerank the candidates with a cross-encoder. Apply it only to the initial top candidates: scoring query–document pairs jointly is substantially slower than embedding search. section 4.3.4
Employee-policy lookup
For “How much notice do I need before quitting in California?”, embedding search can connect quitting with resignation, while BM25 can prioritize California and notice. In the sample policy corpus, the combined retriever ranks the relevant resignation policy first. Its second result is less relevant, so a plausible ranking is not proof that every returned document is useful. section 4.3.4 section 4.3.2
from rank_bm25 import BM25Okapi #1
from sentence_transformers import SentenceTransformer, CrossEncoder #1
from sklearn.metrics.pairwise import cosine_similarity #1
import numpy as np #1
documents = [ #2
"California employees must give a 30-day notice before resignation.",
"All new hires are eligible for health insurance after 90 days.",
"Termination for cause may lead to loss of severance benefits.",
"Employees may request remote work accommodations under ADA.",
"Maternity and adoption leave are fully paid for 12 weeks."
]
tokenized_corpus = [doc.lower().split() for doc in documents] #3
bm25 = BM25Okapi(tokenized_corpus) #3
dense_model = SentenceTransformer("intfloat/e5-base") #4
dense_embeddings = dense_model.encode(documents, normalize_embeddings=True) #5
cross_encoder = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2") #6
Python setup initializes a BM25 index, precomputes normalized E5 document embeddings, and loads a cross-encoder for final relevance scoring. section 4.3.4
Hybrid search can also combine vector similarity with structured filters, such as date ranges, categories, or document status. Choose the combination around the information users must actually find. For production RAG, evaluate retrieval precision and answer relevance, and keep checking data freshness; adding retrieval methods does not by itself guarantee correct answers. section 4.5.3 section 3.9
