liveBook · Manning

Document encoder

As explained in Retrieval Augmented Generation, The Foundational Ideas

A document encoder is a model component that transforms documents or passages into representations used for indexing and retrieval.

A search system can miss the right passage when a question and its answer use different vocabulary. A patient might ask why their chest hurts on the stairs, while a relevant passage describes “substernal discomfort provoked by exertion.” The document encoder turns passages into vectors so retrieval can compare them with a search representation rather than rely only on shared words. section 7.1

In a dual-encoder retriever such as DPR, a document encoder maps passages into an index and a query encoder maps questions into the same vector space. Canonical RAG keeps the document encoder and its prebuilt index fixed while fine-tuning the query encoder alongside the generator. Moving only the query side avoids rebuilding the index during training; DPR’s earlier question-to-passage training makes that split more useful than freezing an arbitrary similarity model. section 4.1 section 4.4

The document side changes the training and retrieval tradeoffs:

  • With fewer than 1,000 training examples, query-side fine-tuning can be both efficient and a useful guard against overfitting, because the document encoder stays frozen. section 6.5
  • HyDE addresses query–document mismatch differently: a language model drafts a hypothetical answer, then a document encoder embeds that document-like text for retrieval. section 7.1

Compare retrieval on a vocabulary mismatch

On a five-passage medical corpus, a direct dense search retrieves one of two relevant passages in its top two, while BM25 retrieves neither. Searching with an embedded hypothetical answer retrieves both. The comparison tests whether changing the search input helps the document representations locate relevant passages; it does not show that a document encoder alone solves every mismatch. section 7.1

Listing 7.2 Three retrievers over the same corpus.
encoder = SentenceTransformer("all-MiniLM-L6-v2")
corpus_vecs = encoder.encode(corpus)
def dense_top2(text):  #A
    vec = encoder.encode(text)
    scores = util.dot_score(vec, corpus_vecs).numpy()[0]
    return np.argsort(scores)[::-1][:2]
 
bm25 = BM25Okapi([doc.split() for doc in corpus])
bm25_scores = bm25.get_scores(query.split())
bm25_top2 = np.argsort(bm25_scores)[::-1][:2]
runs = [("BM25", bm25_top2), ("Dense", dense_top2(query)),
        ("HyDE", dense_top2(hyde_doc))]  #B
for name, top in runs:
    recall = len(set(top) & set(relevant)) / len(relevant)
    print(f"{name:6s} R@2 {recall:.2f}   retrieved {list(top)}")
 

The listing encodes the corpus once and scores both the original question and the hypothetical document against those vectors, making the fixed document-side comparison concrete. section 7.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains the fixed-index training decision and demonstrates embedding-based retrieval, without detailing document-encoder internals.
Examples
Compares BM25, direct dense retrieval, and HyDE on a five-passage medical corpus using Recall@2.
What it assumes
Introduces query–document similarity as an inner product while assuming basic familiarity with embeddings and retrieval.
What you can do after
Evaluate/understand it by weighing fixed-index training costs against retrieval results on mismatched queries.
How it connects
Connects document encoding to query encoders, dense passage retrieval, RAG, HyDE, and query–document mismatch.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.