liveBook · Manning

Pseudo-supervision

As explained in Retrieval Augmented Generation, The Foundational Ideas

Pseudo-supervision is training or retrieval guidance supplied by synthetic signals generated from an existing model rather than direct human labels.

Pseudo-supervision guides training or retrieval with synthetic signals generated by an existing model instead of direct human labels. In retrieval, a generative model writes a relevant passage, creating a synthetic positive example whose relevance signal an unsupervised encoder can use to locate the right vector-space neighborhood. section 7.3

A retrieval pipeline applies the idea in three steps:

  • Prompt a large generative model to write a passage answering the query. section 7.3
  • Embed the generated passage with an unsupervised dense encoder. section 7.3
  • Use that embedding to identify the relevant neighborhood in vector space without relevance labels. section 7.3

Startup-growth query

For “What are the best practices for a startup's user growth?”, HyDE generates a hypothetical answer passage and embeds it before retrieval. On reported benchmarks, this approach substantially improves over BM25 and Contriever while approaching supervised dense retrieval. section 7.3

Listing 7.3 HyDE in LangChain.
from providers import get_chat_model, get_embeddings
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_classic.chains.hyde.base import (
    HypotheticalDocumentEmbedder)
llm = get_chat_model(temperature=0)
base_embeddings = get_embeddings()
prompt = PromptTemplate(
    input_variables=["question"],
    template=("Please write a passage to answer the question.\n"
              "Question: {question}\nPassage:")
)  #A
hyde_embedder = HypotheticalDocumentEmbedder(
    llm_chain=prompt | llm | StrOutputParser(),  #B
    base_embeddings=base_embeddings,
)
query = "What are the best practices for a startup's user growth?"
hypothetical_doc_embedding = hyde_embedder.embed_query(query)  #C
 

The listing shows a LangChain implementation that combines a chat model, prompt, output parser, and base embedding model into a hypothetical-document query embedder. section 7.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains the synthetic-positive mechanism, retrieval pipeline, implementation, and benchmark comparison without detailing model training internals.
Examples
Demonstrates startup user-growth search, LangChain code, and NDCG@10 comparisons across TREC DL19, TREC DL20, SciFact, and FiQA.
What it assumes
Assumes access to a generative model and an unsupervised embedding model, while avoiding human relevance labels.
What you can do after
Apply it via LangChain by generating a hypothetical passage and embedding it as a retrieval query.
How it connects
Connects generative models, dense retrieval, lexical retrieval, unsupervised encoders, supervised fine-tuning, and vector spaces.

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.