Multi-step reasoning
Multi-step reasoning derives a conclusion, solution, or decision through multiple connected intermediate inferential or problem-solving steps.
Inside AI
Logical reasoning is the process of deriving conclusions from information by applying rules of consistency and inference.
- How it is used
- Frames diagnosis and visual interpretation as tasks requiring conclusions that fit the situation.
- How deep it goes
- Works through a diagnostic decision sequence but does not specify a formal inference method or implementation.
- Examples
- Uses emergency triage, diagnostic testing, and a banana-shaped plastic object to illustrate decisions and inferences.
- What you can do after
- Evaluate/understand it by checking whether an automated conclusion accounts for context and whether repeated errors need human review.
Sutskever's List
Reasoning is the ability to derive conclusions or solve problems by applying structured relationships among facts or representations.
- How it is used
- Frames the challenge of connecting facts, comparisons, or decisions across steps where scaling pretraining alone did not yield dependable performance.
- How deep it goes
- Explains relational memory interactions and prompting strategies through worked tasks, but does not establish a general solution to reliable planning.
- Examples
- Uses CLEVR comparisons, bAbI supporting facts, vector-distance ranking, and GSM8K arithmetic word problems.
- What you can do after
- Evaluate/understand it by distinguishing benchmark success from dependable reasoning and by inspecting intermediate steps or explicit planning constraints.
Knowledge Graphs and LLMs in Action
Reasoning is the process of analyzing information, applying rules, and drawing conclusions from evidence or premises.
- How it is used
- Deploys reasoning to derive insights, classify new cases, answer knowledge-graph questions, and support intelligent-system decisions.
- How deep it goes
- Explains deductive and inductive forms, uncertainty, and a query-generation pipeline, while stopping short of formal reasoning algorithms.
- Examples
- Uses spam filtering, a river-crossing puzzle, and natural-language-to-Cypher knowledge-graph querying as concrete reasoning examples.
- What you can do after
- Build reliable query-generation workflows by supplying schema context, checking relationships, structuring reasoning steps, and using execution feedback.
Read how Knowledge Graphs and LLMs in Action explains multi-step reasoning →
Build a Reasoning Model (From Scratch)
The process of deriving conclusions from premises through explicit inference rather than surface pattern recognition.
- How it is used
- Frames intermediate-step generation as the core capability for improving LLM performance on complex tasks and practical agent workflows.
- How deep it goes
- Defines intermediate reasoning, contrasts it with pattern matching, and outlines prompting, sampling, and refinement without detailing training internals.
- Examples
- Uses multistep math, coding, logical puzzles, capital recall, and bird-and-penguin premises to distinguish inference from statistical association.
- What you can do after
- Evaluate, prototype, and design reasoning improvements using chain-of-thought, self-consistency, and other inference-time methods.
Read how Build a Reasoning Model (From Scratch) explains multi-step reasoning →
Machine Learning Platform Engineering
A multistep reasoning chain is an orchestrated sequence of intermediate operations that transforms a user query into a final model response.
- How it is used
- Frames a user query as an orchestrated LLM workflow whose intermediate stages need production debugging.
- How deep it goes
- Explains the component sequence and stage-level tracing, but does not specify an implementation of the orchestration.
- Examples
- Uses a documentation-query RAG application and Langfuse spans across retrieval, augmentation, and generation.
- What you can do after
- Diagnose/debug with Langfuse spans to locate failures and inspect timing and token usage across RAG stages.
Read how Machine Learning Platform Engineering explains multi-step reasoning →
AI Agents in Action
Reasoning is the process of deriving conclusions, plans, or solutions by relating information through intermediate steps.
- How it is used
- Uses reasoning to break down problems, select actions, evaluate solutions, and support planning in agent workflows.
- How deep it goes
- Defines reasoning, distinguishes it from planning, and demonstrates prompting and feedback, but does not detail model internals.
- Examples
- Demonstrates zero-shot prompts, time-travel puzzles, prompt-flow evaluations, logical puzzles, riddles, and hypothetical experiments.
- What you can do after
- Apply reasoning prompts in VS Code prompt flow and evaluate predictions with follow-up prompts and feedback.
Read how AI Agents in Action explains multi-step reasoning →
Natural Language Processing in Action, Second Edition
Reasoning is the process of deriving conclusions or solving problems by applying relationships among facts, premises, or representations.
- How it is used
- Distinguishes genuine problem solving from language models’ pattern-based production of plausible answers.
- How deep it goes
- Explains reasoning conceptually and tests it with one generation experiment, stopping short of formal reasoning algorithms or internals.
- Examples
- Uses standardized-test performance, post-training programming challenges, and GPT-2’s cow-leg counting outputs.
- What you can do after
- Evaluate apparent reasoning by inspecting generated outputs and separating plausible language from grounded, reliable conclusions.
Read how Natural Language Processing in Action, Second Edition explains multi-step reasoning →
Deep Learning with R, Third Edition
Reasoning is the process of deriving conclusions or selecting actions from representations, premises, or rules.
- How it is used
- Uses abstraction and logic to handle experiences that were not anticipated during development or training.
- How deep it goes
- Contrasts discrete reasoning with intuition and pattern recognition, but does not provide an algorithm for executing individual reasoning steps.
- Examples
- Uses mathematical proof, unfamiliar exam problems, future planning, and comparisons with lookup tables and deep networks.
- What you can do after
- Apply abstraction and reasoning to adapt to unfamiliar situations, anticipate possible futures, and plan over longer horizons.
Read how Deep Learning with R, Third Edition explains multi-step reasoning →
Interview Speak
The sequence of mental reasoning steps used to analyze information and reach, evaluate, or justify a conclusion or action.
- How it is used
- Uses interview answers to reveal how candidates analyze problems, make decisions, manage priorities, and respond under pressure.
- How deep it goes
- Explains a practical reasoning sequence—pause, identify difficulty, compare options, justify the choice, and report the outcome—without formal theory.
- Examples
- Uses guesstimates, brainteasers, past difficult decisions, and questions about handling specific problems as practice cases.
- What you can do after
- Practice explaining assumptions, alternatives, trade-offs, conclusions, and results in response to analytical interview questions.
Building Reliable AI Systems
Reasoning that reaches a conclusion through multiple connected inferential steps.
- How it is used
- Supports multi-step reasoning and agent collaboration in multi-agent workflows.
- How deep it goes
- Identifies cycle management and graph components, but does not detail reasoning implementation.
- What you can do after
- Build multi-agent workflows with LangGraph cycles for collaborative, multi-step reasoning.
Read how Building Reliable AI Systems explains multi-step reasoning →
Related concepts
- Tree-of-thoughtcontrasts with · Tree-of-thought is a reasoning technique that explores and evaluates multiple branching paths of intermediate reasoning to develop a solution.
- chain-of-thought promptingused by · Chain-of-thought prompting is a technique that elicits or guides a model to produce or follow intermediate reasoning steps before a final answer.
- ReActused by · ReAct is an agent technique that iteratively interleaves reasoning, actions, and observations so results can guide subsequent actions.
- reasoning modelused by · A reasoning model is an AI model designed or optimized to perform multistep inference or analysis before producing an answer or result.
- intuitioncontrasts with · Intuition is the formation of a judgment or insight without deliberate, explicit analytical reasoning.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.