liveBook · Manning

Query rewriting

As explained in Build an Advanced RAG Application (From Scratch)

A query rewriter is a component that reformulates user queries to clarify references, expand terminology, and add appropriate specificity without inventing constraints.

Query rewriting is a pre-retrieval step that turns a user’s raw request into a more precise, retrieval-friendly query. It addresses the “what” of retrieval: the router chooses where to search, while rewriting improves the query sent to that destination. It can expand abbreviations, resolve references from recent conversation, and make vague wording specific without changing the document index, embedding model, or retrieval infrastructure. section 7.4 section 7.4.1

Vague or compound wording can yield an embedding that is broadly related to several topics but close to no useful document chunk. For example, “Q3 rev breakdown” may retrieve a noisy mixture of revenue, review, and revision material. Rewriting gives the embedding model clearer terms; decomposition handles multiple information needs by splitting them into focused atomic queries. section 7.4.1 section 7.4

A single-query rewriter should follow these constraints:

  • Expand abbreviations and acronyms, such as “Q3” to “third quarter” and “rev” to “revenue.” section 7.4.2
  • Use recent conversation history to replace references such as “the company” or “same as last time” with supported specifics; limiting history to three exchanges controls prompt size and token cost. section 7.4.2
  • Add domain context only when clearly implied, and preserve the original intent without inventing constraints. section 7.4.2
  • Return only the rewritten query so it can pass directly to retrieval or optional decomposition. section 7.4.2

Clarifying a retrieval query

“How’s the Lambda scaling and what about the timeout stuff?” can become “How does AWS Lambda scale under load, and what are the timeout limits?” The rewrite identifies the service, replaces vague wording, and preserves both requested topics. Because this contains two information needs, it can then be decomposed into separate queries for retrieval. section 7.4.2 section 7.4

def rewrite_query(
    user_query: str,
    conversation_history: list = None
) -> str:
    history_context = ""
    if conversation_history:
        history_context = "\n".join(
            f"Q: {q}\nA: {a[:200]}..."
            for q, a in conversation_history[-3:]               #A
        )
    rewrite_prompt = f'''
    You are a search query optimizer. Rewrite the user's query
    to make it more precise and retrieval-friendly.
 
    Rules:
    1. Expand abbreviations ("Q3" -> "third quarter",
       "rev" -> "revenue")
    2. Replace vague references with specific terms using
       conversation history
    3. Add relevant domain context (year, company name)
       when clearly implied
    4. Do NOT add constraints the user did not express
    5. Return ONLY the rewritten query, no explanation
 
    Conversation history:
    {history_context if history_context else "None"}
 
 
    Original query: {user_query}
    Rewritten query:
    '''
    response = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": rewrite_prompt}
        ],
        max_tokens=200,
        temperature=0,                                          #B
    )
    return response.choices[0].message.content.strip()

The implementation builds a history context from the last three exchanges, prompts an LLM with explicit rewrite rules, and requests only the optimized query. section 7.4.2

In an enterprise RAG pipeline, rewriting runs after intent routing and before vector search. A semantic-cache hit can return immediately; time-sensitive requests bypass the cache for live search. On cache misses, the system records the selected route, rewritten query, sub-queries, and answer, creating an audit trail for debugging, compliance reporting, and improvement. section 7.5 section 7.6

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Provides LLM prompt rules, conversation-history handling, concrete transformations, and unified-pipeline placement; it does not alter index or embedding internals.
Examples
Uses Q3 revenue, Uber and Lyft comparison, and AWS Lambda scaling-and-timeout requests to demonstrate clarification.
What it assumes
Assumes familiarity with RAG retrieval and embeddings while grounding rewriting as a pre-retrieval query transformation.
What you can do after
Build it from scratch as an LLM-based rewriter that preserves intent and feeds optimized queries into a routed retrieval pipeline.
How it connects
Connects query rewriting to RAG, agentic routing, semantic caching, vector search, embeddings, and sub-query decomposition.

Also explained in

Compare how 7 books explain query rewriting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.