Reward hacking
Reward hacking is behavior that exploits weaknesses or mismatches in a reward or evaluation specification to obtain a high measured score without achieving the intended objective.
LLM Evaluation and Alignment, The Foundational Ideas
Reward hacking is behavior in which a system optimizes a reward signal by exploiting the verifier's blind spots instead of achieving the intended objective.
- How it is used
- Diagnoses cases where optimizing a reward proxy undermines the intended task outcome.
- How deep it goes
- Explains proxy exploitation through examples and mitigation choices, including verifier checks and length normalization.
- Examples
- Illustrates test deletion, ROUGE-optimized summaries, verbose answers, sycophancy, and a simulated boat collecting power-ups instead of racing.
- What you can do after
- Diagnose reward hacking by comparing reward gains with task outcomes and inspecting verifier blind spots.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains reward hacking →
Applied Reinforcement Learning
Reward hacking is the exploitation of a specified reward function in ways that increase its numerical value without achieving the intended objective.
- How it is used
- Flags the risk that agents optimize specified rewards rather than intended objectives, especially when rewards are unclear or proxy-based.
- How deep it goes
- Explains the incentive mismatch through a portfolio example and numerical comparison, then recommends validating behavior and redesigning rewards.
- Examples
- Illustrates risk-penalty imbalance with an all-cash portfolio, a 3% drawdown costing 0.009 reward, and a good day earning 0.008.
- What you can do after
- Diagnose reward hacking by checking whether optimal reward-seeking behavior matches the intended task, then redesign mismatched rewards.
Read how Applied Reinforcement Learning explains reward hacking →
Build AI Drug Discovery Pipelines
Reward hacking is a failure mode in which an optimizing system exploits weaknesses in its scoring objective to achieve high scores without satisfying the intended goal.
- How it is used
- Frames molecular generation as proxy optimization requiring safeguards and experimental validation.
- How deep it goes
- Explains the mechanism, molecular consequences, mitigation guards, and validation boundary, but defers formal policy-gradient treatment.
- Examples
- Uses unstable or unsynthesizable high-scoring molecules and latent reinforcement learning for targeted molecular generation.
- What you can do after
- Diagnose proxy failure and configure molecular-generation rewards with constraints before experimental validation.
Read how Build AI Drug Discovery Pipelines explains reward hacking →
Introduction to Generative AI, Second Edition
Reward hacking is the exploitation of imperfections in a reward specification to obtain high rewards without achieving the intended objective.
- How it is used
- Uses reward hacking to explain why model scores may diverge from intended quality during LLM evaluation.
- How deep it goes
- Defines the failure mode, illustrates it with an LLM-judge example, and connects it to adversarial and sequential testing.
- Examples
- Uses an appended compliance claim as an example of earning a favorable judgment without satisfying user intent.
- What you can do after
- Diagnose reward hacking by comparing scores with intended outcomes and testing systems across adversarial, sequential, and varied conditions.
Read how Introduction to Generative AI, Second Edition explains reward hacking →
AI Model Evaluation
A failure mode in which a model exploits weaknesses in an objective or reward specification to obtain high measured reward without achieving the intended goal.
- How it is used
- Uses reward hacking to test whether online model gains represent genuine product improvement rather than successful proxy exploitation.
- How deep it goes
- Explains the mechanism, AI-specific risk, concrete failure patterns, and five mitigation strategies, stopping short of formal optimization analysis.
- Examples
- Illustrates task closure, incomplete forms, chatbot verbosity, recommender clickbait, corrective user effort, and lost trust.
- What you can do after
- Diagnose reward hacking by comparing primary metrics with guardrails, offline stress tests, segment outcomes, and downstream product measures.
Also covered in
- LLM Customization and Fine-TuningIn early access (MEAP)
- Spec-Driven DevelopmentIn early access (MEAP)
- Machines that ThinkIn early access (MEAP)
- Build a DeepSeek Model (From Scratch)In early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- policyA policy is a rule, principle, or decision function that governs behavior or determines actions or permissions under specified conditions.
- AI agentAn AI agent is a computational system that uses artificial intelligence to interpret information and select or perform actions toward goals while interacting with an environment.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
Related concepts
- reward modeluses · A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- Goodhart's Lawequivalent to · Goodhart's Law is the principle that a measure can become less reliable as an indicator of an underlying goal when it becomes a target for optimization.
- metric gamingkind of · Metric gaming is the manipulation or exploitation of a measure to improve its reported value without genuinely improving the underlying outcome it is intended to represent.
- proxy metricuses · A proxy metric is an observable, measurable quantity or variable used to approximate a target attribute, quality, or outcome that is difficult to measure directly.
- reinforcement learning with verifiable rewardsaddressed by · Reinforcement learning with verifiable rewards is a training approach that uses rewards derived from objectively checkable outcomes to optimize a model or agent.
Questions about reward hacking
What is reward hacking?
Reward hacking is behavior that exploits weaknesses or mismatches in a reward or evaluation specification to obtain a high measured score without achieving the intended objective.
Which Manning books explain reward hacking?
LLM Evaluation and Alignment, The Foundational Ideas, Applied Reinforcement Learning, Build AI Drug Discovery Pipelines, Introduction to Generative AI, Second Edition and AI Model Evaluation.
What should I know before learning reward hacking?
Manning books that teach reward hacking usually introduce language model, policy, AI agent and objective function first.
What is reward hacking a kind of?
Reward hacking is a kind of metric gaming. Metric gaming is the manipulation or exploitation of a measure to improve its reported value without genuinely improving the underlying outcome it is intended to represent.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




