A search request can fail even when the right information is available. A vague question may give a web search engine too little context; a poorly phrased question may match the wrong chunks in a vector store. Query rewriting puts a transformation step before retrieval: an LLM turns the original request into a clearer search query or several targeted queries. The results then support the answer. section 4.4 section 9.0
One request, three searches
For “How many titles did Michael Jordan win?”, a rewriter can generate “Michael Jordan NBA championships total”, “List of NBA titles won by Michael Jordan”, and “Michael Jordan basketball career championships count”. Searching each phrasing gathers results from several angles rather than relying on the original wording alone. section 4.4
For vector search, start with one rewrite. Prompt the LLM to return a query suited to the vector store, then pass that query—not the raw question—to the retriever. Keep the original question for answer synthesis. This is the Rewrite-Retrieve-Read pattern: rewriting improves the retrieval input without silently changing what the user asked the answer generator to address. section 9.1 section 9.1.4
Listing 9.2 Combined RAG chain with query rewriting
from langchain_core.runnables import RunnablePassthrough
retriever = uk_granular_collection.as_retriever()
rag_prompt_template = """
Given a question and some context, answer the question.
If you do not know the answer, just say I do not know.
Context: {context}
Question: {question}
"""
rag_prompt = ChatPromptTemplate.from_template(
rag_prompt_template)
rewrite_retrieve_read_rag_chain = (
{
"context": {"user_question": RunnablePassthrough()}
| rewriter_chain | retriever, #1
"question": RunnablePassthrough(), #2
}
| rag_prompt
| llm
| StrOutputParser()
)
The RAG chain makes the two paths visible: the rewritten question supplies retrieved context, while the original question passes through to the answer prompt. section 9.1.4
Choose the transformation to fit the retrieval problem:
- Use a single, more precise rewrite when wording is the main obstacle. A prompt can ask for a ChromaDB search query; a LangChain chain connects ChatPromptTemplate, ChatOpenAI, and StrOutputParser. section 9.1.2
- Generate multiple phrasings when one search may miss relevant documents. Parse the LLM’s newline-separated alternatives into a list and retrieve for each; MultiQueryRetriever offers a standard setup. section 9.2.1 section 9.2.3
- For more control over combined results, apply Reciprocal Rank Fusion to rerank documents across query-result lists before answer synthesis. section 10.7.4
More queries are not a free improvement: extra web searches increase response time if run sequentially, though parallel processing can reduce that cost. Rewriting or multiplying questions also may fail to produce the desired results. Test the approach against your own data before treating it as a retrieval fix. section 4.4 section 9.2.3 section 9.6
