liveBook · Manning

Goodhart's Law

Goodhart's Law is the principle that a measure can become less reliable as an indicator of an underlying goal when it becomes a target for optimization.

How 3 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

Goodhart's Law states that a measure often becomes unreliable when it is adopted as an optimization target.

How it is used
Frames evaluation design as an alignment problem because reward signals directly update prompts, agents, or models.
How deep it goes
Explains the measure-target gap, optimizer loopholes, and iterative diagnostic response, but does not provide a formal mitigation algorithm.
Examples
Uses clickbait engagement, blanket safety refusals, token-count gaming, and automated-metric summarization as failure cases.
What you can do after
Diagnose metric gaming and design evaluation criteria around desired outcomes before optimizing an AI system.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains Goodhart's Law →

Outlier Detection in Python

The principle that a measure ceases to be a reliable measure when it becomes the target of optimization.

How it is used
Guards outlier-detection evaluation against reusing the detector’s own measure as its test.
How deep it goes
Defines the principle, gives detector-specific examples, and stops short of a universal remedy beyond using different evaluation methods.
Examples
Uses entropy and KDE evaluation reuse, plus classifiers trained and tested against synthetic outliers.
What you can do after
Evaluate outlier detectors through inspection or different synthetic data instead of reusing the detection measure.

Read how Outlier Detection in Python explains Goodhart's Law →

Reinforcement Learning from Human Feedback

Goodhart's law is the principle that a measure used as an optimization target can cease to be a reliable measure of the underlying goal.

How it is used
Frames RLHF failures as proxy-objective exploitation rather than simple memorization or an inherent flaw in the RL algorithm.
How deep it goes
Explains behavioral symptoms, reward-performance trade-offs, and error sources, while leaving their precise causal origins an open research question.
Examples
Uses verbose answers, rare-token repetition, sycophancy, over-refusal, hedging, and repetitive phrasing as concrete failure examples.
What you can do after
Diagnose reward-model over-optimization by comparing proxy scores with downstream evaluations, human judgment, and KL distance.

Read how Reinforcement Learning from Human Feedback explains Goodhart's Law →

Also covered in

What to know first

Related concepts

Questions about Goodhart's Law

What is Goodhart's Law?

Goodhart's Law is the principle that a measure can become less reliable as an indicator of an underlying goal when it becomes a target for optimization.

Which Manning books explain Goodhart's Law?

LLM Evaluation and Alignment, The Foundational Ideas, Outlier Detection in Python and Reinforcement Learning from Human Feedback.

What should I know before learning Goodhart's Law?

Manning books that teach Goodhart's Law usually introduce language model, reference model, reinforcement learning and RLHF first.

What is the difference between Goodhart's Law and metric?

Goodhart's Law is the principle that a measure can become less reliable as an indicator of an underlying goal when it becomes a target for optimization. A metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.