7 HyDE: Imagining the answer before you search
This chapter covers
- Query-document mismatch and why it limits dense retrieval
- The HyDE pipeline and its role in zero-shot retrieval
- Implementing HyDE with an LLM and a contrastive encoder
- Applying HyDE to challenging queries
- Extending HyDE with query-expansion and hybrid techniques
Imagine an engineer builds a RAG system for their company's internal documentation. A new hire asks a reasonable question: "What are the best practices for our deployment pipeline?" The system returns a few scattered snippets about API key rotation and versioning schemes, but completely misses the central architectural document that outlines the entire philosophy and workflow.
This failure isn't due to a lack of information in the knowledge base but a failure of retrieval. This scenario illustrates the query-document mismatch problem, one of the most common and frustrating failure modes in RAG systems. This chapter introduces a clever and counterintuitive solution proposed in 2022 by Luyu Gao and colleagues: Hypothetical Document Embeddings (HyDE; https://arxiv.org/abs/2212.10496). The core idea is that instead of searching for the question the user asked, the system first imagines a perfect answer and then searches for that instead.