How deeply does How Large Language Models Work cover alignment training?

Defines under-specification, gives human analogies, and introduces probabilistic objections without developing technical solutions.

Alignment training

As explained in How Large Language Models Work

The alignment problem is the challenge of ensuring that an artificial system's goals and methods remain consistent with the goals and values intended by humans.

Alignment is the challenge of making an artificial system’s goals and methods remain consistent with human intentions. It appears when a goal is underspecified or when the system’s permitted actions are not sufficiently constrained. section 9.2.2

A system can pursue the stated target while missing the intended outcome, much as a contract can leave loopholes. The concern is genuine, but catastrophic conclusions require care: outcomes are probabilistic, and extreme intelligence or harmful intent does not make results certain. section 9.2.2

Alignment concerns can arise through familiar human and technical decisions:

Choosing a loss function that rewards an imperfect proxy for the desired behavior. section 9.2.2
Balancing executive compensation against shareholders’ wishes. section 9.2.2
Writing agreements whose wording leaves room for loopholes or evasive strategies. section 9.2.2

How How Large Language Models Work covers it

liveBook manning.com

Read How Large Language Models Work on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

WordPress Power Toolkit: Harness AI to build next-level websitesThe alignment problem is the challenge of ensuring that an AI system’s behavior remains consistent with human values and goals.
liveBook manning.com

Generative AI in ActionAlignment is the process of shaping an AI model's behavior to conform to intended goals, values, or constraints.
liveBook manning.com

Causal AIAI alignment is the task of making an artificial intelligence system's behavior consistent with human goals and preferences.
liveBook manning.com

Introduction to Generative AI, Second EditionAlignment is the property of an artificial-intelligence system whose behavior is consistent with intended human goals, values, and constraints.
liveBook manning.com

Reinforcement Learning from Human FeedbackValue alignment is the conformity of an intelligent system's objectives and behavior to the values held by the people or population it is intended to serve.

Compare how 6 books explain alignment training →

Related concepts

distillationcontrasts with · Knowledge distillation is a training technique in which a student model learns to reproduce or approximate the outputs or behavior of a teacher model.
behaviormeasured by · Behavior is the externally observable actions, responses, outputs, or effects of a system or object under specified conditions or interactions.
evaluationcontrasts with · Evaluation is the systematic assessment of a system, result, or approach against relevant criteria, evidence, or objectives.
guardrailproduces · A guardrail is a constraint, safeguard, or control that limits actions or behavior to reduce risk and keep a system or process within acceptable bounds.

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.