liveBook · Manning

HypotheticalDocumentEmbedder

As explained in Retrieval Augmented Generation, The Foundational Ideas

HypotheticalDocumentEmbedder is a LangChain component that generates hypothetical text and encodes it into an embedding for retrieval.

HypotheticalDocumentEmbedder implements HyDE retrieval: it asks an LLM to generate hypothetical answer text, encodes that text, and returns one vector for similarity search. This separates query-time generation from vector-store retrieval while hiding the pipeline behind embed_query(query). section 7.3

Startup-growth query

A prompt can ask for a passage answering “What are the best practices for a startup's user growth?” The embedder generates that passage, encodes it, and supplies the resulting vector to a FAISS similarity search. section 7.3

Listing 7.3 HyDE in LangChain.
from providers import get_chat_model, get_embeddings
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_classic.chains.hyde.base import (
    HypotheticalDocumentEmbedder)
llm = get_chat_model(temperature=0)
base_embeddings = get_embeddings()
prompt = PromptTemplate(
    input_variables=["question"],
    template=("Please write a passage to answer the question.\n"
              "Question: {question}\nPassage:")
)  #A
hyde_embedder = HypotheticalDocumentEmbedder(
    llm_chain=prompt | llm | StrOutputParser(),  #B
    base_embeddings=base_embeddings,
)
query = "What are the best practices for a startup's user growth?"
hypothetical_doc_embedding = hyde_embedder.embed_query(query)  #C
 

The listing shows the practical wiring: prompt, chat model, string parser, base embeddings, and a single embed_query call. section 7.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains prompt generation, string parsing, embedding, and vector search, but does not detail embedding-model internals.
Examples
Demonstrates startup user-growth search with a FAISS index and a LangChain Python implementation.
What it assumes
Requires an LLM, embedding model, and vector store; supplies the orchestration rather than teaching those components from zero.
What you can do after
Apply HyDE through LangChain by composing a prompt, LLM chain, parser, embeddings, and embed_query call.
How it connects
Connects hypothetical documents with LLMs, embeddings, dense retrieval, vector stores, and RAG.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.