ARES is an automated evaluation framework for retrieval-augmented generation systems. It replaces prompt-only LLM judges with specialized evaluators and adds statistical inference, addressing score instability from small prompt variations. section 14.3 section 14.3.2
ARES builds evaluation in three stages:
- Generates domain-specific synthetic query, passage, and answer triples, including relevant and irrelevant contexts, faithful and contradictory answers, and relevant or irrelevant answers. section 14.3.2
- Fine-tunes lightweight DeBERTa-v3-Large binary classifiers for context relevance, answer faithfulness, and answer relevance. section 14.3.2
- Combines judge predictions with approximately 150 human-annotated validation examples through prediction-powered inference to produce confidence intervals. section 14.3.2 section 14.7
Figure 14.4 The ARES three-stage evaluation pipeline.

The pipeline diagram connects synthetic training data, specialized scoring models, and statistically grounded quality claims. section 14.3.2
