Coherence
Coherence is the degree to which the parts of an output are logically connected, consistent, and intelligible as a whole.
Generative AI in Action
Coherence is the degree to which generated content is logically consistent and meaningfully connected.
- How it is used
- Evaluates whether generated outputs maintain logical flow and meaning across application-specific NLP tasks.
- How deep it goes
- Defines sentence-to-sentence logical flow and demonstrates rubric-based scoring, but does not derive a mathematical coherence measure.
- Examples
- Uses two article summaries, a G-Eval rubric, Azure OpenAI GPT-4, and comparative scores of 5 and 1.
- What you can do after
- Evaluate/understand it by applying a prompted 1-to-5 rubric and comparing results across summaries or model versions.
AI Model Evaluation
Logical coherence measures whether the claims and reasoning in an output fit together consistently and intelligibly.
- How it is used
- Uses coherence to evaluate open-ended generative outputs beyond engagement and deterministic correctness metrics.
- How deep it goes
- Defines coherence, identifies behavioral proxies, and pairs qualitative judgment with deterministic checks without specifying a formal scoring formula.
- Examples
- Uses repeated questions, user corrections, abandoned tasks, and movie recommendation explanations as concrete evaluation evidence.
- What you can do after
- Diagnoses coherence failures through re-queries, corrections, abandonment, and structured evaluation of generated explanations.
Also covered in
- Retrieval Augmented Generation, The Foundational IdeasIn early access (MEAP)
- AI for Everyday IT
- Vibe EngineeringIn early access (MEAP)
- Machines that ThinkIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- promptA prompt is text or other input supplied to an AI model to provide instructions, context, or other information that guides its response or generated output.
- AI agentAn AI agent is a computational system that uses artificial intelligence to interpret information and select or perform actions toward goals while interacting with an environment.
- GPT-4GPT-4 is a multimodal large language model developed by OpenAI that processes text and image inputs and generates text.
Related concepts
- LLM-as-a-judgemeasured by · LLM-as-a-judge is an evaluation technique in which a language model assesses or compares outputs according to specified criteria.
- rubricpart of · A rubric is a structured set of explicit criteria and guidance for judging the quality of an output, artifact, or behavior.
- contextmeasured by · Context is the relevant information, state, or circumstances surrounding an input, decision, or activity that influence its interpretation or behavior.
Questions about coherence
What is coherence?
Coherence is the degree to which the parts of an output are logically connected, consistent, and intelligible as a whole.
Which Manning books explain coherence?
Generative AI in Action and AI Model Evaluation.
What should I know before learning coherence?
Manning books that teach coherence usually introduce language model, prompt, AI agent and GPT-4 first.
What is the difference between coherence and clarity?
Coherence is the degree to which the parts of an output are logically connected, consistent, and intelligible as a whole. Clarity is the degree to which information or communication is easy to understand and interpret without ambiguity or unnecessary complexity.
What is coherence part of?
Coherence is part of rubric. A rubric is a structured set of explicit criteria and guidance for judging the quality of an output, artifact, or behavior.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

