Answer relevancy measures whether a generated response addresses the user's original question instead of drifting into generic or tangential information. It evaluates the generation component of a RAG pipeline and complements retrieval-focused measures such as context relevance. section 6.5.1 section 6.5
RAGAS estimates answer relevancy by generating several artificial questions from the answer, then measuring their similarity to the original question. This goes beyond simple keyword matching by testing whether the answer expresses the question's intended focus. section 6.5.2
Interpret the metric alongside neighboring RAG checks:
- Compare it with faithfulness, which checks whether answer claims follow from retrieved context rather than whether the answer addresses the question. section 6.5.1
- Pair it with context relevance and context recall to distinguish a poorly targeted answer from missing or insufficient retrieval context. section 6.5.1 section 6.5.2
- Use it as one component of the composite RAGAS score when comparing configurations or tracking system improvements. section 6.5.2
Figure 6.7 RAGAS Core Metric extensively covers LLM Grounding and Response (Source)
Use the RAGAS metric figure to see how answer relevancy fits alongside grounding and response evaluation. section 6.5.2
















