Query rewriting is a pre-retrieval step that turns a user’s raw request into a more precise, retrieval-friendly query. It addresses the “what” of retrieval: the router chooses where to search, while rewriting improves the query sent to that destination. It can expand abbreviations, resolve references from recent conversation, and make vague wording specific without changing the document index, embedding model, or retrieval infrastructure. section 7.4 section 7.4.1
Vague or compound wording can yield an embedding that is broadly related to several topics but close to no useful document chunk. For example, “Q3 rev breakdown” may retrieve a noisy mixture of revenue, review, and revision material. Rewriting gives the embedding model clearer terms; decomposition handles multiple information needs by splitting them into focused atomic queries. section 7.4.1 section 7.4
A single-query rewriter should follow these constraints:
- Expand abbreviations and acronyms, such as “Q3” to “third quarter” and “rev” to “revenue.” section 7.4.2
- Use recent conversation history to replace references such as “the company” or “same as last time” with supported specifics; limiting history to three exchanges controls prompt size and token cost. section 7.4.2
- Add domain context only when clearly implied, and preserve the original intent without inventing constraints. section 7.4.2
- Return only the rewritten query so it can pass directly to retrieval or optional decomposition. section 7.4.2
Clarifying a retrieval query
“How’s the Lambda scaling and what about the timeout stuff?” can become “How does AWS Lambda scale under load, and what are the timeout limits?” The rewrite identifies the service, replaces vague wording, and preserves both requested topics. Because this contains two information needs, it can then be decomposed into separate queries for retrieval. section 7.4.2 section 7.4
def rewrite_query(
user_query: str,
conversation_history: list = None
) -> str:
history_context = ""
if conversation_history:
history_context = "\n".join(
f"Q: {q}\nA: {a[:200]}..."
for q, a in conversation_history[-3:] #A
)
rewrite_prompt = f'''
You are a search query optimizer. Rewrite the user's query
to make it more precise and retrieval-friendly.
Rules:
1. Expand abbreviations ("Q3" -> "third quarter",
"rev" -> "revenue")
2. Replace vague references with specific terms using
conversation history
3. Add relevant domain context (year, company name)
when clearly implied
4. Do NOT add constraints the user did not express
5. Return ONLY the rewritten query, no explanation
Conversation history:
{history_context if history_context else "None"}
Original query: {user_query}
Rewritten query:
'''
response = openai_client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "user", "content": rewrite_prompt}
],
max_tokens=200,
temperature=0, #B
)
return response.choices[0].message.content.strip()
The implementation builds a history context from the last three exchanges, prompts an LLM with explicit rewrite rules, and requests only the optimized query. section 7.4.2
In an enterprise RAG pipeline, rewriting runs after intent routing and before vector search. A semantic-cache hit can return immediately; time-sensitive requests bypass the cache for live search. On cache misses, the system records the selected route, rewritten query, sub-queries, and answer, creating an audit trail for debugging, compliance reporting, and improvement. section 7.5 section 7.6






