liveBook · Manning

RQ-RAG

As explained in Retrieval Augmented Generation, The Foundational Ideas

RQ-RAG is a retrieval technique that trains a language model to decide whether to retrieve, rewrite, decompose, or disambiguate a query.

RQ-RAG makes query refinement a learned retrieval capability rather than a prompt-only runtime wrapper. A fine-tuned 7B Llama2 model chooses whether to skip retrieval, rewrite, decompose, or disambiguate a query. section 7.6.5

At inference time, RQ-RAG can select among four behaviors:

Chaining a multi-hop refinement

For “What kind of university is the school where Rey Ramsey was educated?”, RQ-RAG retrieves that he attended Rutgers University, then refines the next question toward Rutgers University's classification as a land-grant university. Candidate trajectories are ranked using perplexity, confidence, or an ensemble. section 7.6.5

Table 7.2 Comparison of HyDE variants and query-side techniques
Variation
Primary benefit
Implementation cost/complexity
Latency impact
Use case
Standard HyDE
Significant zero-shot retrieval improvement
Low
Medium
General-purpose RAG where baseline retrieval is poor.
Multi-Sample HyDE
Improved stability and robustness to noise
Medium (requires averaging)
High
Applications where query robustness is critical.
Hybrid HyDE (Fusion)
Combines semantic and lexical strengths
High (requires fusion logic)
Medium
High-precision search that cannot miss keyword matches.
HyDE + Re-ranker
Maximizes precision for top results
High (requires a second model)
High
Safety-critical or expert systems (e.g., legal, medical).
Reverse HyDE
Eliminates query-time LLM latency
Medium (shifts cost to indexing)
None at query time
Read-heavy corpora that change rarely (docs, catalogs, KBs).
RQ-RAG
Built-in rewrite / decompose / disambiguate; strong on multi-hop
High (requires supervised fine-tuning of the generator)
High (multiple sequential refinements per query)
Systems where query patterns are complex and a custom-trained generator is acceptable.

Compare RQ-RAG with HyDE variants and other query-side techniques across benefits, implementation cost, latency, and use cases. section 7.6.5

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains training behaviors, control tokens, chained trajectories, and selection, but does not detail the fine-tuning procedure.
Examples
Uses trivial chit-chat, a Rey Ramsey multi-hop question, control tokens, and trajectory selection.
What you can do after
Evaluate whether supervised query refinement fits traffic patterns requiring rewriting, decomposition, disambiguation, or retrieval skipping.
How it connects
Connects query refinement with retrieval, query decomposition, disambiguation, HyDE, and Self-RAG.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.