liveBook · Manning

Soft hyde

As explained in Retrieval Augmented Generation, The Foundational Ideas

Soft HyDE is a HyDE variant that generates a concise outline, keyword expansion, or bulleted pseudo-passage instead of a polished hypothetical document.

Soft HyDE asks an LLM for an outline, bulleted answer, or keyword-expanded pseudo-passage instead of a polished hypothetical document. Because the encoder needs topical fingerprint rather than prose quality, the shorter representation preserves retrieval signal while reducing tokens and cost. section 7.6.5

Deploy Soft HyDE with ordinary query-side and downstream safeguards:

  • Rewrite the query before generating the hypothetical. section 7.6.5
  • Combine the generated representation with BM25 in a hybrid pipeline, preserving exact identifier matches. section 7.6.5
  • Cache generations against normalized queries so repeated questions avoid another LLM call. section 7.6.5
  • Use a re-ranker, citation requirements, and answer-level evaluation to catch plausible but incorrectly framed retrieval. section 7.6.5

Support search

For a support corpus with repeated long-tail questions, normalize the query, generate a short Soft HyDE outline on a cache miss, retrieve through hybrid BM25 and semantic search, then rerank the results. section 7.6.5

Table 7.2 Comparison of HyDE variants and query-side techniques
Variation
Primary benefit
Implementation cost/complexity
Latency impact
Use case
Standard HyDE
Significant zero-shot retrieval improvement
Low
Medium
General-purpose RAG where baseline retrieval is poor.
Multi-Sample HyDE
Improved stability and robustness to noise
Medium (requires averaging)
High
Applications where query robustness is critical.
Hybrid HyDE (Fusion)
Combines semantic and lexical strengths
High (requires fusion logic)
Medium
High-precision search that cannot miss keyword matches.
HyDE + Re-ranker
Maximizes precision for top results
High (requires a second model)
High
Safety-critical or expert systems (e.g., legal, medical).
Reverse HyDE
Eliminates query-time LLM latency
Medium (shifts cost to indexing)
None at query time
Read-heavy corpora that change rarely (docs, catalogs, KBs).
RQ-RAG
Built-in rewrite / decompose / disambiguate; strong on multi-hop
High (requires supervised fine-tuning of the generator)
High (multiple sequential refinements per query)
Systems where query patterns are complex and a custom-trained generator is acceptable.

Compare Soft HyDE's compact query representation with standard HyDE, multi-sample, hybrid, reranked, and reverse variants using benefit, complexity, latency, and use case. section 7.6.5

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains representation choice, deployment components, caching, and guardrails, but does not detail encoder internals.
Examples
Uses outlines, bulleted answers, keyword expansions, support-corpus questions, normalized-query caches, and hybrid BM25 retrieval.
What it assumes
Assumes an instruction-following LLM, a sentence encoder, and a retrieval pipeline, while requiring no retraining.
What you can do after
Apply it via a hybrid retrieval pipeline with caching, reranking, citation checks, and answer-level evaluation.
How it connects
Connects query rewriting, BM25, hybrid retrieval, reranking, caching, citation requirements, and answer-level evaluation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.