liveBook · Manning

Query rewriting

As explained in Retrieval Augmented Generation, The Foundational Ideas

Query reformulation is the transformation of an existing search query into an alternative formulation intended to retrieve more suitable information.

Query rewriting transforms a user's wording into a retrieval-oriented formulation. It can convert a natural-language question into keyword-focused search terms, or reformulate a failed query to target gaps identified during retrieval and answer generation. section 11.2.3 section 15.2.3

Use rewriting in two complementary ways:

  • Rewrite before retrieval to normalize user wording and improve matching; HyDE pipelines may combine this step with BM25, vector search, and reranking. section 7.6.5
  • Rewrite during an agentic loop when retrieval confidence is low, the answer is incomplete, or ambiguity requires sub-queries and additional evidence. section 15.2 section 15.2.3
  • Keep evaluation outside the rewrite decision when possible: autonomy can improve next-step judgment without ensuring that retrieved evidence is reliable. section 15.2.4

From a current-events question to a searchable query

A question such as “Who is the current director of Company Y?” can become “Company Y director 2025.” If internal compliance data is outdated, the rewritten web query can retrieve a current OFAC announcement, which is then combined with the historical internal profile. section 11.2.3

Listing 15.2 Agentic retrieval loop with reflection and replanning.
from openai import OpenAI
 
client = OpenAI()
MODEL = "gpt-4o-mini"
MAX_ITERATIONS = 3  #A
CONFIDENCE_THRESHOLD = 0.7  #B
 
def agentic_rag_loop(
    query: str,
    retrieve_fn: callable,
    k: int = 5
) -> dict:
    """Run an agentic RAG loop with reflection."""
    context = retrieve_fn(query, k=k)  #C
    for iteration in range(MAX_ITERATIONS):
        response = client.chat.completions.create(
            model=MODEL,
            messages=[
                {"role": "system", "content": (
                    "Answer based on the context. After your "
                    "answer, rate your confidence 0-1 and list "
                    "any missing information."
                )},
                {"role": "user", "content": (
                    f"Query: {query}\nContext: {context}"
                )},
            ],
        )
        answer = response.choices[0].message.content
        evaluation = _evaluate_response(query, answer)  #D
        if evaluation["confidence"] >= CONFIDENCE_THRESHOLD:
            return {"answer": answer, "iterations": iteration + 1}
        query = _reformulate_query(query, evaluation["gaps"])
        new_context = retrieve_fn(query, k=k)
        context = context + "\n" + new_context  #E
    return {"answer": answer, "iterations": MAX_ITERATIONS}
 

A compact Python implementation shows query rewriting as one step in a bounded retrieve–generate–evaluate loop, with confidence thresholds and a maximum iteration count preventing runaway searches. section 15.2.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Covers keyword conversion and iterative reformulation with a short implementation pattern, but defers detailed rewrite-model training.
Examples
Uses a Company Y director lookup, an outdated sanctions database, and a bounded Python agentic retrieval loop.
What it assumes
Assumes retrieval and answer generation, while introducing adaptive control through confidence, gap analysis, and sub-query planning.
What you can do after
Apply it via an OpenAI-based agentic RAG loop that retrieves, evaluates, reformulates gaps, and stops after a bounded number of iterations.
How it connects
Connects query rewriting to RAG, agentic RAG, web search, query expansion, sub-query decomposition, BM25, HyDE, and reranking.

Also explained in

Compare how 7 books explain query rewriting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.