OpenAI Model Spec
The OpenAI Model Spec is a published specification describing intended behavior and guiding the training, evaluation, and improvement of OpenAI language models.
LLM Evaluation and Alignment, The Foundational Ideas
The OpenAI Model Spec is a published specification describing intended model behavior and providing a framework for training, testing, and feedback.
- How it is used
- Defines intended model behavior and provides a published framework for training, testing, and feedback.
- How deep it goes
- Explains the authority hierarchy, alignment pillars, examples, and limitations rather than specifying every policy category.
- Examples
- Illustrates implicit guideline overrides with a pirate-style request and distinguishes tool output from instructions.
- What you can do after
- Evaluate model instructions by authority and identify where a written specification needs testing and revision.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains OpenAI Model Spec →
Reinforcement Learning from Human Feedback
A model specification is a written behavioral specification defining intended behaviors and principles for a language model.
- How it is used
- Defines intended model behavior before training and guides steering, evaluation, product deployment, and communication.
- How deep it goes
- Explains purpose, stakeholders, and alignment context but stops short of specifying the document’s complete policy contents or training algorithm.
- Examples
- Uses OpenAI’s 2024 Model Spec, Anthropic’s Constitution and soul document, and deliberative alignment for o1 as comparison points.
- What you can do after
- Evaluate model behavior against explicit intended behaviors and communicate alignment goals to developers, designers, and the public.
Read how Reinforcement Learning from Human Feedback explains OpenAI Model Spec →
What to know first
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- reward modelA reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- RLHFReinforcement learning from human feedback is a model-training approach that uses human evaluations or preferences as feedback to optimize model behavior.
Related concepts
- chain of commandhas part · A chain of command is an instruction-priority hierarchy that resolves conflicts among directives issued at different authority levels.
- behaviorconfigures · Behavior is the observable actions, responses, outputs, or effects of a system or object under specified conditions or interactions.
- personality alignmenthas part · Personality alignment shapes how a model communicates so that its tone, style, and character remain consistent with intended interaction goals.
- principled alignmenthas part · Principled alignment shapes model behavior around abstract human values and ethical principles.
- red-teamingproduced by · Red-teaming is an adversarial assessment practice that deliberately simulates attacks or uses challenging inputs to identify a system’s vulnerabilities, failures, or unsafe behavior.
Questions about OpenAI Model Spec
What is OpenAI Model Spec?
The OpenAI Model Spec is a published specification describing intended behavior and guiding the training, evaluation, and improvement of OpenAI language models.
Which Manning books explain OpenAI Model Spec?
LLM Evaluation and Alignment, The Foundational Ideas and Reinforcement Learning from Human Feedback.
What should I know before learning OpenAI Model Spec?
Manning books that teach OpenAI Model Spec usually introduce fine-tuning, reward model and RLHF first.
What is the difference between OpenAI Model Spec and Anthropic Constitution?
The OpenAI Model Spec is a published specification describing intended behavior and guiding the training, evaluation, and improvement of OpenAI language models. The Anthropic Constitution is an internal set of principles that guides training-time generation of critiques, revisions, and preference data for Anthropic models.
What are the parts of OpenAI Model Spec?
OpenAI Model Spec includes chain of command, personality alignment and principled alignment.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

