Adaptive retrieval decides whether and when to fetch external information during generation instead of searching for every prompt. This avoids wasted compute, context pollution, and unnecessary exposure to irrelevant passages. section 9.0
A generator emits a Retrieve control token before or during generation. It can answer from parametric memory, request passages, repeat retrieval when a new sub-question appears, and stop when the available evidence is sufficient. section 9.0 section 9.3.3
Thresholding turns the retrieval decision into an adjustable production policy:
- Compare P(Retrieve=Yes) with a threshold τ; skip the vector search when the probability is below τ. section 9.3.3
- Lower τ to retrieve more often, improving recall on borderline knowledge questions while increasing compute and context-pollution risk. section 9.3.3
- Raise τ to favor parametric answers and efficiency, accepting that some borderline questions may not trigger retrieval. section 9.3.3
One support assistant, three decisions
For “Thanks, that helped!”, P(Retrieve=Yes) is about 0.02, so the assistant answers without searching. For a v3-versus-v4 release-note question, probability rises to about 0.91; the assistant retrieves, then performs a second retrieval at about 0.78 for missing deprecation-policy details. For “Write a haiku about debugging,” probability is about 0.08, so retrieval is skipped. section 9.3.3
Adaptive retrieval can also pair retrieval with self-reflection: the model filters irrelevant documents, checks generated claims against relevant passages, and marks responses as supported or unverified through reflection tokens. section 9.2.1
