A retrieval system can find a passage that repeats a question’s words while missing one that answers it. Query-side fine-tuning addresses this ranking failure without rebuilding the corpus index: freeze the document encoder, keep its indexed embeddings fixed, and train only the query encoder to place queries nearer useful documents. section 6.4.1 section 6.8
For answer generation, the reader can supply the training signal. Perplexity Distillation favors documents that make the correct answer easier for the reader to generate, rather than requiring document-relevance labels. This shifts retrieval toward usefulness to the answer, not just similarity to the question. section 6.5 section 6.8
Choose an update strategy according to data volume and indexing cost:
- Full index update re-embeds the corpus after retriever changes. It offers the highest potential accuracy but incurs substantial update overhead. section 6.4.1
- Re-ranking retrieves a broad candidate set from the existing index, then re-embeds and ranks those candidates with the updated model. It depends on the initial search finding the best documents. section 6.4.1
- Query-side fine-tuning avoids historical index refreshes. In a 64-example comparison, it outperformed full fine-tuning; with larger datasets, its peak accuracy may be lower. section 6.4.1
Adapting an enterprise knowledge search
An analyst asks why operating margins declined. A baseline search ranks a passage reporting the margin above one explaining its causes. After training on 64 examples, the retriever gives more weight to causal commentary. When another quarter’s reports arrive, the team embeds and adds the new documents, then fine-tunes the query side with a small new example set—without re-embedding the historical corpus. section 6.7
Table 6.5 Update strategies
|
Strategy
|
Mechanism
|
Computational Cost
|
Use Case
|
Key Trade-off
|
|
Full Index Update
|
Re-embed the entire corpus periodically.
|
Very High
|
Full-dataset fine-tuning; when accuracy is paramount.
|
Highest potential accuracy at the cost of
downtime/overhead.
|
|
Re-ranking
|
Retrieve
N≫k with stale index;
re-rank N live.
|
Low
|
Balancing freshness and efficiency; production systems.
|
Assumes the best documents are in the initial N
candidates.
|
|
Query-Side Fine-tuning
|
Freeze document encoder; only train query encoder.
|
Very Low
|
Few-shot learning (e.g., <1000 examples); rapid
adaptation.
|
Potentially lower peak accuracy with large datasets.
|
The comparison puts index-refresh cost, candidate re-ranking, and query-only training alongside their intended uses and limitations. section 6.4.1
