Faithfulness measures whether a generated response is factually supported by the retrieved context. A faithful answer keeps its claims grounded in the supplied documents, reducing unsupported information and hallucination risk. section 8.2.2 section 8.4
Evaluate faithfulness in two stages:
- Decompose each answer sentence into atomic, self-contained statements without pronouns. section 8.2.2
- Judge every statement as 1 when the retrieved context directly supports it, or 0 when it does not. section 8.2.2
- Aggregate the statement judgments into a faithfulness score and compare it with answer correctness and context recall. section 8.2.5 section 8.4
Benchmark interpretation
A benchmark can report faithfulness of 0.9657 alongside answer correctness of 0.7774 and context recall of 0.7941. The high faithfulness indicates that answers rarely invent information, while the lower recall and correctness scores point toward incomplete retrieval or answer formulation problems. section 8.2.6
Figure 8.1 Evaluating different steps of a RAG pipeline
Use the pipeline view to locate faithfulness after context retrieval and answer generation, alongside tool selection and retrieval relevance. section 8.0
