Chapter 1: Understanding reasoning models
The chapter introduction motivates reasoning methods for complex LLM tasks and agents, and explains why the book uses checkable math examples while considering the costs of added reasoning machinery.
After this chapter you will be able to
- Reasoning beyond pattern recognition. Explain that LLM success has largely relied on statistical pattern recognition, while advances in reasoning methods enable them to tackle more complex tasks such as logical puzzles and multistep math problems.
- Intermediate steps and reasoning models. Explain the book’s practical definition of LLM reasoning and what makes an LLM a reasoning model. Describe how its intermediate steps may be shown or hidden before a final answer.
- Probabilistic steps versus fixed rules. Distinguish an LLM’s generated reasoning steps from the fixed, rule-based steps of a symbolic logic engine, including what each implies about logical reliability.
- Pretraining objective and outcomes. Explain that pretraining uses massive amounts of unlabeled text to teach an LLM to predict the next token, producing a base model that can generate coherent, human-like text.
- Post-training technique roles. Distinguish instruction tuning from preference tuning: the former improves responses to user tasks, while the latter refines responses toward user preferences.
- Inference-time compute scaling. Explain how inference-time compute scaling can improve a model’s reasoning performance without training it or changing its weights.
- Reward-based reasoning training. Explain how reinforcement learning improves reasoning through reward feedback and how it differs from inference-time scaling.
- Reasoning pattern distillation. Describe how distillation transfers reasoning patterns from a more capable model to a smaller or more efficient one.
- Pattern matching and inference. Distinguish an LLM's statistically likely text continuation from logical reasoning that uses premises, rules, and intermediate steps. Explain why a plausible answer such as “Berlin” does not by itself demonstrate deduction.
- Penguin premises and background knowledge. Explain why the penguin prompt yields “yes” when only its premises count, but creates an inconsistency when background knowledge about penguins is allowed. Describe how a reasoning system should respond to that inconsistency.
- Implicit contradiction recognition. Explain how a conventional LLM can give a seemingly logical answer to a familiar contradictory premise without explicitly checking for contradictions.
- Reasoning capability spectrum. Distinguish earlier LLMs' simulated reasoning from dedicated reasoning models without treating either as an explicit rule-based logic pipeline.
- Choosing when to reason. Distinguish tasks that benefit from reasoning models from simpler tasks where they may be unnecessary, and explain why using them for everything can be a poor choice.
- Sources of reasoning cost. Explain the two reasons reasoning workflows can cost more: longer token-by-token outputs and multiple model runs for one task.
- Value of building from scratch. Justify implementing reasoning models and methods from scratch as a way to understand how they work and better assess their tradeoffs.
- Evaluation after each improvement. Explain why reasoning improvements are followed by evaluation, whether the change comes from an inference-time method or additional training.
- Inference versus training changes. Distinguish improving reasoning at inference time from improving the model through additional training, using whether the model weights change.
Section by section
Introduction
The chapter introduction motivates reasoning methods for complex LLM tasks and agents, and explains why the book uses checkable math examples while considering the costs of added reasoning machinery.
- Reasoning beyond pattern recognition: Explain that LLM success has largely relied on statistical pattern recognition, while advances in reasoning methods enable them to tackle more complex tasks such as logical puzzles and multistep math problems.
Why it matters: This establishes why reasoning methods are worth studying beyond ordinary LLM text generation. - Reasoning for practical agents: Describe why the introduction calls reasoning essential for practical AI agents: an agent may need to break a task into steps, use tools, and recover from mistakes.
Why it matters: Agent tasks give a practical reason to care about reasoning beyond standalone answers. - Checkable tasks and reasoning costs: Explain why the book uses math examples and what practical tradeoff the introduction identifies when adding reasoning machinery to a standard LLM.
Why it matters: Checkable examples make evaluation practical, and reasoning improvements still need to justify their costs.
Key terms: reasoning, statistical pattern recognition, multistep math problems, AI agents, use tools, recover from mistakes, math examples, check automatically, standard LLM, reasoning machinery
1.1 Defining reasoning in the context of LLMs
Defines LLM reasoning as generating intermediate steps and distinguishes those steps from deterministic logic and human thought.
- Intermediate steps and reasoning models: Explain the book’s practical definition of LLM reasoning and what makes an LLM a reasoning model. Describe how its intermediate steps may be shown or hidden before a final answer.
Why it matters: This definition clarifies what the book means when it calls a model or response reasoning-enhanced. - Probabilistic steps versus fixed rules: Distinguish an LLM’s generated reasoning steps from the fixed, rule-based steps of a symbolic logic engine, including what each implies about logical reliability.
Why it matters: A convincing chain of steps should not be mistaken for a guaranteed-correct proof. - Human resemblance and its limits: Explain why calling an LLM’s output thinking or reasoning does not establish that it reasons as a human does.
Why it matters: The language of thinking is useful shorthand, not a claim that the model has a human mind.
Key terms: reasoning, reasoning model, intermediate steps, chain of thought (CoT), <think>...</think>, autoregressively, statistical patterns, symbolic logic engine, deterministic reasoning, thinking
1.2 Understanding the standard LLM training pipeline
Explains how conventional LLMs move from next-token pretraining to post-training for user-facing responses, and distinguishes an instruction-tuned model from a chatbot interface.
- Pretraining objective and outcomes: Explain that pretraining uses massive amounts of unlabeled text to teach an LLM to predict the next token, producing a base model that can generate coherent, human-like text.
Why it matters: This explains what the model has learned before it is tuned to respond to users. - Token and tokenizer roles: Describe a token as a unit of text that may be a word, word part, or punctuation, and explain that a tokenizer converts tokens into numerical IDs for the model.
Why it matters: It clarifies what the model predicts when the section says “next token.” - Post-training technique roles: Distinguish instruction tuning from preference tuning: the former improves responses to user tasks, while the latter refines responses toward user preferences.
Why it matters: The two techniques explain how a base model becomes better suited to answering users. - Chatbot interface distinction: Explain why an instruction-tuned model alone is not a chatbot: an interactive, multi-turn chat interface adds guidance and orchestration around it.
Why it matters: It prevents conflating a tuned model with the surrounding chat application.
Key terms: pretraining, unlabeled text, next-token prediction, base models, token, tokenizer, numerical IDs, post-training, supervised fine-tuning, SFT
1.3 Improving LLM reasoning with training and inference techniques
This section explains three broad ways to improve an LLM’s reasoning—more computation at inference time, reinforcement learning, and distillation—and distinguishes reasoning-focused RL from RLHF.
- Inference-time compute scaling: Explain how inference-time compute scaling can improve a model’s reasoning performance without training it or changing its weights.
Why it matters: It separates improvements made while a model responds from improvements that require training. - Reward-based reasoning training: Explain how reinforcement learning improves reasoning through reward feedback and how it differs from inference-time scaling.
Why it matters: It identifies the approach that changes the model itself rather than only changing how it is used. - Reasoning pattern distillation: Describe how distillation transfers reasoning patterns from a more capable model to a smaller or more efficient one.
Why it matters: It explains how a smaller model can acquire reasoning behavior from a more capable one. - Reasoning RL versus RLHF: Distinguish reasoning-focused RL from preference-tuning RLHF by their sources of reward and what those rewards favor.
Why it matters: It prevents confusing rewards for verifiable correctness with rewards based on human preferences.
Key terms: inference-time compute scaling, inference-compute scaling, test-time scaling, model weights, reinforcement learning, RL, rewards, trial and error, distillation, knowledge distillation
1.4 Pattern matching vs. logical reasoning
This section contrasts LLM pattern matching with rule-based logical reasoning, using a penguin example to show why premises and background knowledge must be distinguished.
- Pattern matching and inference: Distinguish an LLM's statistically likely text continuation from logical reasoning that uses premises, rules, and intermediate steps. Explain why a plausible answer such as “Berlin” does not by itself demonstrate deduction.
Why it matters: A convincing answer alone does not show how the model arrived at it. - Penguin premises and background knowledge: Explain why the penguin prompt yields “yes” when only its premises count, but creates an inconsistency when background knowledge about penguins is allowed. Describe how a reasoning system should respond to that inconsistency.
Why it matters: Whether an answer is justified depends on which facts the task permits you to use. - Rule-based systems tradeoff: Explain why explicit rule-based systems remain useful in critical domains yet can be hard to implement, in contrast with data-trained neural networks.
Why it matters: The contrast explains why transparent rules are valuable without implying that they are easy to build.
Key terms: pretraining, next token, statistical associations, pattern matching, logical reasoning, premises, closed-world, open-world, background knowledge, inconsistency
1.5 Simulating reasoning without explicit rules
This section explains how LLMs can appear to reason through learned patterns rather than explicit logical rules, where that approach struggles, and how dedicated reasoning models refine it.
- Implicit contradiction recognition: Explain how a conventional LLM can give a seemingly logical answer to a familiar contradictory premise without explicitly checking for contradictions.
Why it matters: A correct-looking answer alone does not show that the model used explicit logical rules. - Limits of learned patterns: Describe the kinds of reasoning problems on which pattern-based responses tend to struggle.
Why it matters: Familiar examples can make pattern matching look reliable even when harder or unfamiliar problems expose its limits. - Reasoning capability spectrum: Distinguish earlier LLMs' simulated reasoning from dedicated reasoning models without treating either as an explicit rule-based logic pipeline.
Why it matters: It frames dedicated reasoning models as improvements to LLM behavior, not a switch to a conventional logic engine.
Key terms: probability-weighted patterns, statistical association, explicit contradiction checking, implicit pattern matching, novel, multistep logical relationships, reasoning models, intermediate steps, inference-compute scaling, post-training
1.6 Why build reasoning models from scratch?
The section explains when reasoning models are worth using, why they can cost more, and how building them from scratch helps clarify those tradeoffs.
- Choosing when to reason: Distinguish tasks that benefit from reasoning models from simpler tasks where they may be unnecessary, and explain why using them for everything can be a poor choice.
Why it matters: Choosing a model to fit the task avoids paying for reasoning that may not help. - Sources of reasoning cost: Explain the two reasons reasoning workflows can cost more: longer token-by-token outputs and multiple model runs for one task.
Why it matters: These two sources of cost explain why a more elaborate answer or workflow demands more compute. - Value of building from scratch: Justify implementing reasoning models and methods from scratch as a way to understand how they work and better assess their tradeoffs.
Why it matters: Building the methods yourself makes decisions about when their extra reasoning is worthwhile more informed.
Key terms: reasoning models, overthinking, tokens, forward pass, candidate solutions, verifier, from scratch, tradeoffs
1.7 A road map to building reasoning models from scratch
The section explains why the book starts with a pretrained LLM, evaluates changes to it, and distinguishes inference-time improvements from additional training.
- Pretrained LLM as baseline: Explain why the starting point is an existing pretrained LLM rather than a model trained from scratch, and how it serves as a baseline for improvements.
Why it matters: A baseline gives the reader a starting point for judging improvements. - Evaluation after each improvement: Explain why reasoning improvements are followed by evaluation, whether the change comes from an inference-time method or additional training.
Why it matters: Without measuring the result, an attempted improvement cannot be distinguished from an effective one. - Inference versus training changes: Distinguish improving reasoning at inference time from improving the model through additional training, using whether the model weights change.
Why it matters: The distinction separates changes to how an existing model is used from changes to the model itself.
Key terms: base LLM, pretrained, baseline, evaluation, inference-time methods, additional training, inference time, model weights, reasoning model
1.8 Summary
This summary recaps conventional LLM training, what LLM reasoning means and how it differs from explicit logic, and three broad ways to improve it.
Where this is used later
- Building a math verifier (chapter 3) · Introduces a math verifier for checking answers.
- Generating better responses with chain-of-thought prompting (chapter 4) · Chain-of-thought prompting builds on the idea of producing intermediate steps.
- Loading pretrained models (chapter 2) · Explains the origin of the pretrained model loaded for generation.
- Preparing input texts for LLMs (chapter 2) · Uses the distinction between text, tokens, and numerical IDs when preparing model inputs.
- The original RL pipeline with human feedback (RLHF) (chapter 6) · Provides context for RLHF as a preference-tuning technique.
- Introducing LLMs for text generation (chapter 2) · The distinction between pretraining and later training stages.
- Introduction to inference-time scaling (chapter 4) · Introduces inference-time scaling as an approach to improving reasoning.
- (chapter intro) (chapter 2) · Introduces techniques for improving an LLM's reasoning capabilities.
Concepts explained in this chapter
- multi-step reasoningThe process of deriving conclusions from premises through explicit inference rather than surface pattern recognition.
- reasoning modelA language model optimized to deliberate over complex problems, often producing intermediate steps or using tools to improve answer quality.
- pretrainingThe initial training stage in which a language model learns general statistical structure from large amounts of data before task-specific adaptation.
- pattern recognitionThe process of completing text or answers by relying on learned correlations and recurring examples instead of formal reasoning.
- post-trainingMethods applied after initial pretraining to improve model behavior or capabilities.
- preference tuningA post-training method that adjusts a model to produce responses aligned with preferred outputs or styles.
- inference-time compute scalingImproving a trained model's output quality by spending more computation at inference time without changing its weights.
- reinforcement learningA training approach where a model learns from rewards or feedback to improve its behavior.
- reinforcement learning with human feedbackA training approach that uses human preference labels to optimize a language model toward responses humans prefer.
- language modelA neural language model trained on large text corpora to predict and generate text.
- distillationA model adaptation approach that trains a smaller student model to imitate a larger teacher model’s outputs or behavior.
- symbolic AIReasoning based on explicit symbolic rules rather than statistical associations learned from data.
- reasoning methodologyA method applied to a language model to improve explicit step-by-step reasoning behavior.
- tokenA discrete text unit used as the basic element of language-model generation.
- tokenizerA component that splits text into token IDs for model input, with this section focusing on the Qwen3 reasoning variant that supports reasoning tags.
- evaluationThe process of assessing model performance on held-out data or benchmarks.
- trainingA method that improves model behavior by further updating the model through training.
- <think> tagA special markup tag used to enclose hidden reasoning steps in model output.
- tokenizationThe process of rendering text into a model-ready format and splitting it into tokens/IDs for training.
- generationThe process of producing new textual output from a language model given prior context or instructions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.