A RAG system can retrieve the right documents and still produce an answer that drifts away from them. An LLM judge uses a language model to assess outputs or retrieved context, making failures visible that retrieval scores alone can miss. Its verdict is a measurement, not ground truth: a passing threshold can hide a worsening trend. section 14.0
The judge’s job depends on where it sits in the pipeline:
- In evaluation, prompted judges score RAG behavior; RAGAS can do so without reference answers, while G-Eval adds chain-of-thought reasoning to fine-grained scoring. section 14.3 section 14.3.3
- In Self-RAG training, a stronger critic labels examples with reflection tokens. A smaller student learns from those labeled examples and later emits its own control signals; the critic is not required for each student prediction. section 9.3 section 9.3.1
- In dual-path graph retrieval, a critic checks merged graph and vector context against the question. It can reject inadequate context and send the query back for another routing pass before answer generation. section 12.4.2
A retrieval improvement that masked an answer regression
A mortgage lender’s compliance assistant gained a cross-encoder reranker and retrieved better documents, yet answer faithfulness fell. The existing threshold did not flag the decline; comparing reports across weeks exposed it. Judge-based scoring is most useful here when its results are examined as diagnostic evidence rather than accepted because a score passes. section 14.0
Judge choice carries tradeoffs. A frontier critic is an option when API cost and data sensitivity permit; open-weight or domain-specific critics can suit privacy or specialized notions of support. Prompted evaluation needs no training, whereas the compared ARES approach trains a small evaluator using synthetic data and annotations. Neither sophistication nor fluency removes bias: judges can favor familiar or self-generated wording, formatting, or a position in a pair, and repeated runs can disagree. Check the judge’s behavior on the task at hand. section 9.3.1 section 14.0 section 14.3.3
Table 14.2 Comparing RAG evaluation approaches
|
Approach
|
Judge type
|
Training required
|
Confidence bounds
|
Cost per sample
|
Best for
|
|
RAGAS
|
Prompted LLM
|
None
|
No
|
High (multiple LLM calls)
|
Quick evaluation, prototyping
|
|
G-Eval
|
Prompted LLM + CoT
|
None
|
No
|
Medium (one LLM call + CoT)
|
Fine-grained scoring with CoT reasoning
|
|
ARES
|
Fine-tuned DeBERTa
|
Synthetic data + 150 annotations
|
Yes (PPI)
|
Low (small model inference)
|
Production monitoring, rigorous comparison
|
Compare prompted RAGAS and G-Eval with trained ARES by training needs, confidence bounds, sample cost, and intended use. section 14.3.3




















