liveBook · Manning

Fidelity

Also called faithfulness

Fidelity is the degree to which an output, model, or representation accurately reflects or is supported by its relevant source, target, evidence, or behavior.

How 11 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

Faithfulness is a metric representing how accurately generated content reflects factual evidence or supported source information.

How it is used
Measures hallucination by checking whether generated claims are supported, including against retrieved context in retrieval-augmented generation.
How deep it goes
Explains claim decomposition, evidence verification, and score calculation, with caveats about ambiguous claims and support.
Examples
Calculates a 0.50 score for a financial answer with three supported claims and three unsupported claims.
What you can do after
Evaluate generated answers by decomposing claims, checking evidence, and interpreting faithfulness alongside retrieval quality.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains fidelity →

Retrieval Augmented Generation, The Foundational Ideas

Faithfulness measures whether a generated response is supported by the retrieved context rather than introducing unsupported claims.

How it is used
Diagnoses whether RAG generation introduces claims unsupported by retrieved passages, separately from retrieval quality and answer relevance.
How deep it goes
Works through atomic claim extraction, context verification, score calculation, per-claim debugging, and an LLM-judge implementation.
Examples
Uses a reapplication answer with one unsupported automatic-approval claim and compares grounded and fabricated compliance answers.
What you can do after
Diagnose/debug with it by inspecting unsupported claims and distinguishing generation drift from retrieval problems.

Read how Retrieval Augmented Generation, The Foundational Ideas explains fidelity →

Build an Advanced RAG Application (From Scratch)

Faithfulness measures whether the claims in a generated response are factually inferable from the retrieved context.

How it is used
Evaluates whether generated answers stay factually grounded in retrieved context within a RAG pipeline.
How deep it goes
Defines claim-level factual consistency and its purpose, but does not specify a scoring algorithm or implementation procedure.
Examples
Uses retrieved documents, generated responses, and their individual claims as evaluation artifacts.
What you can do after
Evaluate whether a RAG response is grounded and identify unsupported information that may indicate hallucination.

Read how Build an Advanced RAG Application (From Scratch) explains fidelity →

Building LLM Applications with DSPy

Groundedness measures how well the ideas in a summary are supported by the original document.

How it is used
Evaluates whether generated summaries are supported by their original documents.
How deep it goes
Develops a claim-based judge and its validation workflow, then combines groundedness with completeness; defers optimization of the combined program.
Examples
Tests a dialog summary against its source and a summary from another dialog, returning groundedness scores of 1.0 and 0.0.
What you can do after
Apply it via DSPy to evaluate summaries with a calibrated judge and combine groundedness with completeness.

Read how Building LLM Applications with DSPy explains fidelity →

Effective Conversational AI

Faithfulness is a generation metric measuring whether an answer is factually supported by the retrieved context.

How it is used
Measures whether RAG-generated answers are factually supported by retrieved context.
How deep it goes
Defines faithfulness, distinguishes it from relevancy, and names evaluators without detailing their scoring procedures.
Examples
Uses a question about contract-breach consequences to illustrate checking generated answers against retrieved context.
What you can do after
Evaluate whether generated answers align with retrieved passages, using LLM assessment alongside human review.

Read how Effective Conversational AI explains fidelity →

Essential GraphRAG

Faithfulness is a metric that measures whether a generated answer is supported by the retrieved context without introducing unsupported information.

How it is used
Evaluates whether generated RAG answers remain factually grounded in retrieved context.
How deep it goes
Decomposes answers into atomic statements and applies binary context-support judgments, stopping short of measuring complete answer correctness.
Examples
Uses atomic statement prompts, retrieved contexts, benchmark datasets, and comparative metric tables.
What you can do after
Evaluate generated answers with RAGAS using faithfulness, answer correctness, and context recall metrics.

Read how Essential GraphRAG explains fidelity →

Generative AI in Action

Groundedness is the degree to which an AI output conforms to the information in its sources rather than speculation or fabrication.

How it is used
Uses groundedness to assess whether generated responses are supported by factual sources.
How deep it goes
Demonstrates an API request and returned ungrounded text, but does not explain score calibration.
Examples
Uses a medical summarization request with patient information as the grounding source.
What you can do after
Apply groundedness detection through the Azure Content Safety endpoint using a response and grounding sources.

Read how Generative AI in Action explains fidelity →

Evaluating AI Systems

A measure of whether a response is faithful to its source information or supporting evidence.

How it is used
Uses faithfulness to assess whether RAG answers are grounded in retrieved evidence.
How deep it goes
Defines the metric and outlines its inputs, scoring approaches, output, and limitations without specifying a single implementation.
Examples
Uses a generated RAG answer and its retrieved context as evaluation inputs.
What you can do after
Evaluate whether an answer is supported by retrieved context while recognizing that faithfulness alone does not establish completeness or usefulness.

Read how Evaluating AI Systems explains fidelity →

The Art of AI Product Development

Groundedness is the degree to which generated content is supported by the information retrieved or provided as evidence.

How it is used
Evaluates whether RAG-generated content is supported by retrieved information.
How deep it goes
Defines groundedness and applies it as an evaluation criterion, without specifying a scoring scale.
Examples
Tests a report on AI trends in SaaS and contrasts grounding with relevance to a request for detailed analysis.
What you can do after
Evaluate RAG responses for source support using LLM-as-a-judge, while checking evaluation criteria closely.

Read how The Art of AI Product Development explains fidelity →

AI Model Evaluation

Groundedness is a metric measuring whether an output is supported by the information it is expected to rely on.

How it is used
Measures whether generated answers are supported by the retrieved context.
How deep it goes
Defines groundedness and gives a product-language interpretation, without detailing a calculation method.
Examples
Uses a RAG chatbot and explains improvement as more answers supported by retrieved context.
What you can do after
Evaluate whether retrieved context supports generated answers and interpret score changes in product-relevant terms.

Read how AI Model Evaluation explains fidelity →

LLM Customization and Fine-Tuning

A measure of whether a sentiment label matches the content of its sentence.

How it is used
Screens generated financial sentiment examples for agreement between each sentence and its assigned label.
How deep it goes
Defines a 1–5 label-match score and a pass threshold of 4 within a three-part gate, but does not assess teacher-model quality.
Examples
Scores whether a financial sentence’s sentiment label fits, alongside checks for realistic financial writing and plausible facts.
What you can do after
Apply it via an LLM-as-judge rubric to accept or reject candidate sentence-label examples.

Read how LLM Customization and Fine-Tuning explains fidelity →

Also covered in

What to know first

Related concepts

Questions about fidelity

What is fidelity?

Fidelity is the degree to which an output, model, or representation accurately reflects or is supported by its relevant source, target, evidence, or behavior.

Which Manning books explain fidelity?

LLM Evaluation and Alignment, The Foundational Ideas, Retrieval Augmented Generation, The Foundational Ideas, Build an Advanced RAG Application (From Scratch), Building LLM Applications with DSPy, Effective Conversational AI, Essential GraphRAG, Generative AI in Action, Evaluating AI Systems and 3 more.

What should I know before learning fidelity?

Manning books that teach fidelity usually introduce language model, precision, recall and hallucination first.

What is the difference between fidelity and accuracy?

Fidelity is the degree to which an output, model, or representation accurately reflects or is supported by its relevant source, target, evidence, or behavior. Accuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.

What is fidelity also called?

Fidelity is also called faithfulness.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.