A retrieved passage can look relevant and still support the wrong answer. A compliance search, for example, might return a 2020 policy on expedited due diligence while missing a 2024 directive that overrides it. A retrieval evaluator puts a quality gate between retrieval and answer generation, so doubtful evidence can trigger another search rather than flow straight into an answer. section 11.1.3 section 11.2.4
In corrective retrieval-augmented generation (CRAG), the evaluator is separate from the answer-generating model. It scores how relevant a retrieved document is to a query and uses confidence thresholds to select the next action. It may be implemented with a fine-tuned T5 model, a compact model, a cross-encoder, or a calibrated LLM judge. section 11.2.1 section 11.2.4
The score routes retrieval into three paths:
- High confidence: retain the internal evidence and refine it for the generator. section 11.2.1
- Low confidence: discard misleading results and search externally for better evidence. section 11.2.1
- Intermediate confidence: keep internal evidence but search externally to corroborate or supplement it. section 11.2.1
An ambiguous compliance result
For a question about expedited onboarding of an EU politically exposed person, an internal 2020 policy scores 0.76, a 2024 directive scores 0.63, and an unrelated holiday policy scores 0.00. The result is classified as ambiguous, so the pipeline uses a hybrid source path instead of trusting the top internal hit. Thresholds such as 0.85 and 0.45 are starting points; tune them on held-out questions because changing them changes which path runs. section 11.2.4
The evaluator can grade only documents it receives. High confidence does not prove that a better, newer document was not ranked below the retrieval cutoff. Publication dates, supersession links, and retiring obsolete documents address that upstream problem; external search offers a route beyond the internal index when the evaluator flags doubt. Web-search fallback also adds latency, so its value depends on the cost of a wrong answer. section 11.2.1 section 9.5.1 section 11.2.5
Listing 11.3 The CRAG orchestrator.
class CRAGPipeline:
"""Orchestrates the retrieve-evaluate-correct loop."""
def __init__(self, retriever, evaluator, web_searcher):
self.retriever = retriever
self.evaluator = evaluator
self.web_searcher = web_searcher
def run(self, query: str) -> dict:
"""Executes the corrective RAG workflow."""
internal_docs = self.retriever.retrieve(query) #A
action, score = self.evaluator.evaluate(query, internal_docs) #B
context = ""
source = "internal"
if action == RetrievalAction.INCORRECT: #C
web_docs = self.web_searcher.search(query) #D
context = self.evaluator.refine_knowledge(query, web_docs)
source = "web"
elif action == RetrievalAction.AMBIGUOUS:
internal_ctx = self.evaluator.refine_knowledge( #E
query, internal_docs)
web_docs = self.web_searcher.search(query)
web_ctx = self.evaluator.refine_knowledge(query, web_docs)
context = f"{internal_ctx}\n\n{web_ctx}" #F
source = "hybrid"
else:
context = self.evaluator.refine_knowledge(query, internal_docs) #G
return {"context": context, "action": action.value,
"source": source, "score": score}
The Python orchestrator shows where evaluation sits after retrieval and how an incorrect verdict triggers external search. section 11.2.4
