Fidelity
Fidelity is the degree to which an output, model, or representation accurately reflects or is supported by its relevant source, target, evidence, or behavior.
Effective Conversational AI
Faithfulness is a generation metric measuring whether an answer is factually supported by the retrieved context.
- How it is used
- Measures whether RAG-generated answers are factually supported by retrieved context.
- How deep it goes
- Defines faithfulness, distinguishes it from relevancy, and names evaluators without detailing their scoring procedures.
- Examples
- Uses a question about contract-breach consequences to illustrate checking generated answers against retrieved context.
- What you can do after
- Evaluate whether generated answers align with retrieved passages, using LLM assessment alongside human review.
Essential GraphRAG
Faithfulness is a metric that measures whether a generated answer is supported by the retrieved context without introducing unsupported information.
- How it is used
- Evaluates whether generated RAG answers remain factually grounded in retrieved context.
- How deep it goes
- Decomposes answers into atomic statements and applies binary context-support judgments, stopping short of measuring complete answer correctness.
- Examples
- Uses atomic statement prompts, retrieved contexts, benchmark datasets, and comparative metric tables.
- What you can do after
- Evaluate generated answers with RAGAS using faithfulness, answer correctness, and context recall metrics.
Generative AI in Action
Groundedness is the degree to which an AI output conforms to the information in its sources rather than speculation or fabrication.
- How it is used
- Uses groundedness to assess whether generated responses are supported by factual sources.
- How deep it goes
- Demonstrates an API request and returned ungrounded text, but does not explain score calibration.
- Examples
- Uses a medical summarization request with patient information as the grounding source.
- What you can do after
- Apply groundedness detection through the Azure Content Safety endpoint using a response and grounding sources.
The Art of AI Product Development
Groundedness is the degree to which generated content is supported by the information retrieved or provided as evidence.
- How it is used
- Evaluates whether RAG-generated content is supported by retrieved information.
- How deep it goes
- Defines groundedness and applies it as an evaluation criterion, without specifying a scoring scale.
- Examples
- Tests a report on AI trends in SaaS and contrasts grounding with relevance to a request for detailed analysis.
- What you can do after
- Evaluate RAG responses for source support using LLM-as-a-judge, while checking evaluation criteria closely.
Read how The Art of AI Product Development explains fidelity →
Related concepts
- Ragasused by · Ragas is an open-source framework and Python library for evaluating retrieval-augmented generation systems and applications.
- accuracycontrasts with · Accuracy is the degree to which an output, prediction, or measurement agrees with a correct or reference outcome under a specified criterion.
- hallucinationmeasures · A hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
- retrieved documentmeasures · A retrieved document is information selected by an information-retrieval process from a source and supplied as context or evidence for a query or task.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.