A search system can miss the right passage when a question and its answer use different vocabulary. A patient might ask why their chest hurts on the stairs, while a relevant passage describes “substernal discomfort provoked by exertion.” The document encoder turns passages into vectors so retrieval can compare them with a search representation rather than rely only on shared words. section 7.1
In a dual-encoder retriever such as DPR, a document encoder maps passages into an index and a query encoder maps questions into the same vector space. Canonical RAG keeps the document encoder and its prebuilt index fixed while fine-tuning the query encoder alongside the generator. Moving only the query side avoids rebuilding the index during training; DPR’s earlier question-to-passage training makes that split more useful than freezing an arbitrary similarity model. section 4.1 section 4.4
The document side changes the training and retrieval tradeoffs:
- With fewer than 1,000 training examples, query-side fine-tuning can be both efficient and a useful guard against overfitting, because the document encoder stays frozen. section 6.5
- HyDE addresses query–document mismatch differently: a language model drafts a hypothetical answer, then a document encoder embeds that document-like text for retrieval. section 7.1
Compare retrieval on a vocabulary mismatch
On a five-passage medical corpus, a direct dense search retrieves one of two relevant passages in its top two, while BM25 retrieves neither. Searching with an embedded hypothetical answer retrieves both. The comparison tests whether changing the search input helps the document representations locate relevant passages; it does not show that a document encoder alone solves every mismatch. section 7.1
Listing 7.2 Three retrievers over the same corpus.
encoder = SentenceTransformer("all-MiniLM-L6-v2")
corpus_vecs = encoder.encode(corpus)
def dense_top2(text): #A
vec = encoder.encode(text)
scores = util.dot_score(vec, corpus_vecs).numpy()[0]
return np.argsort(scores)[::-1][:2]
bm25 = BM25Okapi([doc.split() for doc in corpus])
bm25_scores = bm25.get_scores(query.split())
bm25_top2 = np.argsort(bm25_scores)[::-1][:2]
runs = [("BM25", bm25_top2), ("Dense", dense_top2(query)),
("HyDE", dense_top2(hyde_doc))] #B
for name, top in runs:
recall = len(set(top) & set(relevant)) / len(relevant)
print(f"{name:6s} R@2 {recall:.2f} retrieved {list(top)}")
The listing encodes the corpus once and scores both the original question and the hypothetical document against those vectors, making the fixed document-side comparison concrete. section 7.1
