liveBook · Manning

Alignment training

Alignment training is the process of adjusting an AI system’s behavior to better conform to intended human goals, preferences, values, or constraints.

How 9 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

Alignment is the process of shaping an AI model or system to follow instructions, respect constraints, and behave consistently with human values and preferences.

How it is used
Uses alignment to bring model and system behavior closer to human goals, instructions, and constraints.
How deep it goes
Explains competing alignment pillars and an iterative correction cycle, while introducing human-feedback and constitutional training approaches.
Examples
Illustrates alignment with a customer service chatbot that invents shipping dates when database information is missing.
What you can do after
Evaluate and understand alignment trade-offs, then identify targeted system changes for measured failures.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains alignment training →

Machines that Think

The process of shaping an AI system's behavior to avoid harmful or unacceptable outputs.

How it is used
Steers language-model responses toward safety and human expectations while addressing harmful behavior and bias.
How deep it goes
Explains supervised fine-tuning and preference-learning pipelines, then examines probabilistic limits and behavioral failures rather than implementation details.
Examples
Uses compliance-pattern jailbreaks and an alignment-faking experiment to illustrate how behavior can shift with context.
What you can do after
Evaluate and understand alignment methods and their limits when judging model behavior and designing safeguards.

Read how Machines that Think explains alignment training →

LLM Customization and Fine-Tuning

Steering a model's behavior toward human or organizational preferences and constraints.

How it is used
Targets specific behaviors or tones that a base model resists within an enterprise adaptation workflow.
How deep it goes
Explains safety loss during distillation and its mitigation, without detailing a preference-optimization procedure.
Examples
Uses a medical-domain distillation study where students produced unsafe content on 86% of red-team prompts refused by the teacher.
What you can do after
Evaluate and understand when alignment may address a resistant behavior, and test distilled students for safety regressions.

Read how LLM Customization and Fine-Tuning explains alignment training →

WordPress Power Toolkit: Harness AI to build next-level websites

The alignment problem is the challenge of ensuring that an AI system’s behavior remains consistent with human values and goals.

How it is used
Frames why increasingly capable AI should remain helpful and subject to human goals.
How deep it goes
Defines the alignment problem and discusses risks and safeguards, but provides no alignment-training procedure.
Examples
Uses a surprising Go move and Claude’s refusal to recommend stocks to illustrate unpredictability and safety-focused behavior.
What you can do after
Evaluate/understand why visible safety choices do not settle the wider problem of keeping advanced AI aligned with human goals.

Read how WordPress Power Toolkit: Harness AI to build next-level websites explains alignment training →

How Large Language Models Work

The alignment problem is the challenge of ensuring that an artificial system's goals and methods remain consistent with the goals and values intended by humans.

How it is used
Frames goal-setting and action constraints as risks when artificial systems pursue human objectives imperfectly.
How deep it goes
Defines under-specification, gives human analogies, and introduces probabilistic objections without developing technical solutions.
Examples
Uses loss-function design, CEO compensation, legal loopholes, and hypothetical human-threatening systems as examples.
What you can do after
Evaluate/understand how incomplete goals and unconstrained methods can produce misaligned behavior and overconfident risk claims.

Read how How Large Language Models Work explains alignment training →

Generative AI in Action

Alignment is the process of shaping an AI model's behavior to conform to intended goals, values, or constraints.

How it is used
Shapes model behavior toward intended goals and reduces harmful outputs within responsible-AI deployment.
How deep it goes
Identifies fine-tuning and RLHF, then emphasizes testing and layered mitigations without detailing training procedures.
Examples
Uses GPT-4, Claude, GPT-3, Llama 2, RLHF, MoCa scenarios, AuROC, and human moral judgments.
What you can do after
Evaluate alignment with human judgments and design layered safeguards around aligned models.

Read how Generative AI in Action explains alignment training →

Causal AI

AI alignment is the task of making an artificial intelligence system's behavior consistent with human goals and preferences.

How it is used
Aligns LLM causal interpretations with human judgments, goals, preferences, or instructed decision criteria.
How deep it goes
Defines alignment through causal responsibility and instructed criteria, emphasizing judgment rather than parameter-level training procedures.
Examples
Uses network logs, a bandwidth spike, packet loss, and an LLM judgment about Jake’s responsibility.
What you can do after
Apply it via prompting by specifying criteria, legal theories, or case law for an LLM’s judgment.

Read how Causal AI explains alignment training →

Introduction to Generative AI, Second Edition

Alignment is the property of an artificial-intelligence system whose behavior is consistent with intended human goals, values, and constraints.

How it is used
Frames language-model safety as the effort to make system behavior reflect creator goals and broader human values.
How deep it goes
Explains the objective and names several safeguards, but stops short of implementation details and a settled account of human values.
Examples
Uses detection, filtering, conditional pretraining, safety post-training, and machine unlearning as representative strategies.
What you can do after
Evaluate alignment efforts by distinguishing practical response safeguards from broader unresolved questions about values, accountability, and power.

Read how Introduction to Generative AI, Second Edition explains alignment training →

Reinforcement Learning from Human Feedback

Value alignment is the conformity of an intelligent system's objectives and behavior to the values held by the people or population it is intended to serve.

How it is used
Frames human preferences as an optimization signal for training systems whose behavior should reflect intended values.
How deep it goes
Explains reward-model and preference-based mechanisms, but defers detailed data collection and reward-modeling procedures.
Examples
Uses pairwise preferred answers, Bradley–Terry reward models, and learned rewards replacing fixed environment rewards.
What you can do after
Evaluate/understand how human preferences become proxy rewards and how empirical alignment differs from direct value calibration.

Read how Reinforcement Learning from Human Feedback explains alignment training →

Also covered in

What to know first

Related concepts

Questions about alignment training

What is alignment training?

Alignment training is the process of adjusting an AI system’s behavior to better conform to intended human goals, preferences, values, or constraints.

Which Manning books explain alignment training?

LLM Evaluation and Alignment, The Foundational Ideas, Machines that Think, LLM Customization and Fine-Tuning, WordPress Power Toolkit: Harness AI to build next-level websites, How Large Language Models Work, Generative AI in Action, Causal AI, Introduction to Generative AI, Second Edition and 1 more.

What should I know before learning alignment training?

Manning books that teach alignment training usually introduce language model, fine-tuning, RLHF and safety first.

What is the difference between alignment training and distillation?

Alignment training is the process of adjusting an AI system’s behavior to better conform to intended human goals, preferences, values, or constraints. Distillation is a training technique in which a student model learns to approximate the outputs or behavior of a teacher model.

What kinds of alignment training are there?

Kinds of alignment training include personality alignment.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.