RQ-RAG makes query refinement a learned retrieval capability rather than a prompt-only runtime wrapper. A fine-tuned 7B Llama2 model chooses whether to skip retrieval, rewrite, decompose, or disambiguate a query. section 7.6.5
At inference time, RQ-RAG can select among four behaviors:
- Skip retrieval for trivial conversational inputs. section 7.6.5
- Rewrite unclear or poorly formed queries. section 7.6.5
- Decompose multi-hop questions into sequential sub-queries. section 7.6.5
- Disambiguate queries when user intent is unclear. section 7.6.5
Chaining a multi-hop refinement
For “What kind of university is the school where Rey Ramsey was educated?”, RQ-RAG retrieves that he attended Rutgers University, then refines the next question toward Rutgers University's classification as a land-grant university. Candidate trajectories are ranked using perplexity, confidence, or an ensemble. section 7.6.5
Table 7.2 Comparison of HyDE variants and query-side techniques
|
Variation
|
Primary benefit
|
Implementation cost/complexity
|
Latency impact
|
Use case
|
|
Standard HyDE
|
Significant zero-shot retrieval improvement
|
Low
|
Medium
|
General-purpose RAG where baseline retrieval is poor.
|
|
Multi-Sample HyDE
|
Improved stability and robustness to noise
|
Medium (requires averaging)
|
High
|
Applications where query robustness is critical.
|
|
Hybrid HyDE (Fusion)
|
Combines semantic and lexical strengths
|
High (requires fusion logic)
|
Medium
|
High-precision search that cannot miss keyword matches.
|
|
HyDE + Re-ranker
|
Maximizes precision for top results
|
High (requires a second model)
|
High
|
Safety-critical or expert systems (e.g., legal, medical).
|
|
Reverse HyDE
|
Eliminates query-time LLM latency
|
Medium (shifts cost to indexing)
|
None at query time
|
Read-heavy corpora that change rarely (docs, catalogs,
KBs).
|
|
RQ-RAG
|
Built-in rewrite / decompose / disambiguate; strong on
multi-hop
|
High (requires supervised fine-tuning of the generator)
|
High (multiple sequential refinements per query)
|
Systems where query patterns are complex and a
custom-trained generator is acceptable.
|
Compare RQ-RAG with HyDE variants and other query-side techniques across benefits, implementation cost, latency, and use cases. section 7.6.5
