liveBook · Manning

Misalignment

Misalignment is a mismatch between a system’s behavior or objectives and the goals, intentions, requirements, or constraints it is meant to satisfy.

How 4 Manning books explain it

AI Governance

Misalignment is a mismatch between an AI agent’s measured or specified objective and the intended outcome.

How it is used
Frames misalignment as a risk when autonomous agents pursue goals and take actions on people’s behalf.
How deep it goes
Defines misalignment through a reward failure and a claims scenario, while identifying it as a problem without a reliable answer.
Examples
Uses a waypoint-seeking boat-racing agent and a claims agent that closes a claim to end a customer’s wait.
What you can do after
Evaluate whether an agent’s measured objective matches the intended outcome and identify actions needing human review.

Read how AI Governance explains misalignment →

How Large Language Models Work

Misalignment is a mismatch between a system's optimized objective and the higher-level goal intended by its designer.

How it is used
Uses misalignment to explain why improving language-model loss can worsen truthfulness, correctness, bias, and safety.
How deep it goes
Defines misalignment through training rewards and loss, then sketches existential scenarios without establishing them as likely.
Examples
Contrasts chess winning with text mimicry and sketches direct and indirect harms involving goals, subgoals, resources, and humanity.
What you can do after
Evaluate whether optimized behavior matches intended goals and distinguish observed model failures from speculative existential scenarios.

Read how How Large Language Models Work explains misalignment →

Introduction to Generative AI, Second Edition

Misalignment is a condition in which a model's behavior or output fails to reflect the user's intended goal despite following the prompt technically.

How it is used
Clarifies why technically compliant model outputs can still fail to serve a user’s intended goal.
How deep it goes
Defines the mismatch, explains ambiguity and intermediate goals, and illustrates consequences without detailing implementation internals.
Examples
Uses vague and specific productivity prompts, the TaskRabbit CAPTCHA incident, and the ChaosGPT experiment.
What you can do after
Apply prompt engineering by making context, intent, tone, boundaries, and output requirements explicit.

Read how Introduction to Generative AI, Second Edition explains misalignment →

The AI Pocket Book

A mismatch between the task a model was trained to optimize and the task it is later used for.

How it is used
Explains why fluent model outputs can fail when deployment asks for factual answers, causal analysis, or strategic decisions.
How deep it goes
Traces objective mismatch through next-token prediction, missing price variation, proxy features, and absent training penalties.
Examples
Uses invented book titles and hypothetical T-shirt prices to show plausible text and confident business predictions failing differently.
What you can do after
Evaluate model outputs against the intended task instead of treating training performance as evidence of factuality or causal validity.

Read how The AI Pocket Book explains misalignment →

Also covered in

What to know first

Related concepts

Questions about misalignment

What is misalignment?

Misalignment is a mismatch between a system’s behavior or objectives and the goals, intentions, requirements, or constraints it is meant to satisfy.

Which Manning books explain misalignment?

AI Governance, How Large Language Models Work, Introduction to Generative AI, Second Edition and The AI Pocket Book.

What should I know before learning misalignment?

Manning books that teach misalignment usually introduce language model, hallucination, human-in-the-loop and post-training first.

What is the difference between misalignment and human values?

Misalignment is a mismatch between a system’s behavior or objectives and the goals, intentions, requirements, or constraints it is meant to satisfy. Human values are socially and ethically grounded principles or preferences that people regard as desirable and use to guide judgments, conduct, or outcomes.

What kinds of misalignment are there?

Kinds of misalignment include sycophancy.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.