7 HyDE: Imagining the answer before you search
This chapter covers
- Query-document mismatch and why it limits dense retrieval
- The HyDE pipeline and its role in zero-shot retrieval
- Implementing HyDE with an LLM and a contrastive encoder
- Applying HyDE to challenging queries
- Extending HyDE with query-expansion and hybrid techniques
Imagine an engineer builds a RAG system for their company's internal documentation. A new hire asks a reasonable question: "What are the best practices for our deployment pipeline?" The system returns a few scattered snippets about API key rotation and versioning schemes, but completely misses the central architectural document that outlines the entire philosophy and workflow.
The knowledge base had the answer; retrieval could not reach it. That is the query-document mismatch problem, and in the taxonomy from chapter 1 it is two failure points at once: FP1 (Missing Content), where the answer never enters the context because nothing matched, and FP2 (Missed the Top Rank), where the passage that answers the question is retrieved and ranked below documents that merely share its vocabulary. This chapter introduces a clever and counterintuitive solution proposed in 2022 by Luyu Gao and colleagues: Hypothetical Document Embeddings (HyDE; https://arxiv.org/abs/2212.10496). The core idea is that instead of searching for the question the user asked, the system first imagines a perfect answer and then searches for that instead.