Query rewriting transforms a user's wording into a retrieval-oriented formulation. It can convert a natural-language question into keyword-focused search terms, or reformulate a failed query to target gaps identified during retrieval and answer generation. section 11.2.3 section 15.2.3
Use rewriting in two complementary ways:
- Rewrite before retrieval to normalize user wording and improve matching; HyDE pipelines may combine this step with BM25, vector search, and reranking. section 7.6.5
- Rewrite during an agentic loop when retrieval confidence is low, the answer is incomplete, or ambiguity requires sub-queries and additional evidence. section 15.2 section 15.2.3
- Keep evaluation outside the rewrite decision when possible: autonomy can improve next-step judgment without ensuring that retrieved evidence is reliable. section 15.2.4
From a current-events question to a searchable query
A question such as “Who is the current director of Company Y?” can become “Company Y director 2025.” If internal compliance data is outdated, the rewritten web query can retrieve a current OFAC announcement, which is then combined with the historical internal profile. section 11.2.3
Listing 15.2 Agentic retrieval loop with reflection and replanning.
from openai import OpenAI
client = OpenAI()
MODEL = "gpt-4o-mini"
MAX_ITERATIONS = 3 #A
CONFIDENCE_THRESHOLD = 0.7 #B
def agentic_rag_loop(
query: str,
retrieve_fn: callable,
k: int = 5
) -> dict:
"""Run an agentic RAG loop with reflection."""
context = retrieve_fn(query, k=k) #C
for iteration in range(MAX_ITERATIONS):
response = client.chat.completions.create(
model=MODEL,
messages=[
{"role": "system", "content": (
"Answer based on the context. After your "
"answer, rate your confidence 0-1 and list "
"any missing information."
)},
{"role": "user", "content": (
f"Query: {query}\nContext: {context}"
)},
],
)
answer = response.choices[0].message.content
evaluation = _evaluate_response(query, answer) #D
if evaluation["confidence"] >= CONFIDENCE_THRESHOLD:
return {"answer": answer, "iterations": iteration + 1}
query = _reformulate_query(query, evaluation["gaps"])
new_context = retrieve_fn(query, k=k)
context = context + "\n" + new_context #E
return {"answer": answer, "iterations": MAX_ITERATIONS}
A compact Python implementation shows query rewriting as one step in a bounded retrieve–generate–evaluate loop, with confidence thresholds and a maximum iteration count preventing runaway searches. section 15.2.3






