liveBook · Manning

Reasoning

As explained in Designing AI Agents

An agent capability for evaluating information, forming hypotheses, and selecting conclusions or actions.

Reasoning is the agent capability that evaluates information and selects a strategy or plan. It receives filtered context from perception and can retrieve earlier evidence from memory. Action then turns its proposal into an interaction with an external system; a successful tool return alone does not prove the intended effect was accepted. section 2.2.1

The decision depends on what the model can distinguish in its input. Represent execution facts, current state, and control signals explicitly rather than relying on long prose to preserve every distinction. More context or a longer thinking budget cannot recover facts blurred during input assembly. section 5.0

Match reasoning depth to the task. A simple lookup may not justify the tokens, latency, and cost of a deeper analysis. Route work using difficulty, consequence, evidence needs, and budget; preserve independent alternatives when comparing paths, and test hypotheses against external state without giving the test authority to perform recovery. section 2.2.1 section 5.0

Code review with a weak-link check

A code-review agent decomposes a renaming refactor into steps. It verifies the internal call sites but cannot confirm whether external consumers updated their calls. The lowest self-rated step flags that dependency for human review; the score directs attention, but does not establish that other steps are correct. section 5.2.4

Argus Review · diff #4287
Chain confidence: 0.62 (lowest step flagged)
 
Step 1 (0.95): Identified change as renaming `processData` → `process_data`
               across the module boundary — Python snake_case convention.
Step 2 (0.91): Verified all 8 internal call sites updated in service.py,
               utils.py, and three test files. Counts match git diff.
Step 3 (0.62): ⚠ Cannot confirm whether external API consumers (SDK clients,
               webhook integrations) also updated their call signatures.
               Internal search found no OpenAPI spec or client-generation
               config in this repo. Dependency is inferred, not verified.
               ↳ Requesting human review on this step.

Inspect the review's step scores, evidence, and human-review request. section 5.2.4

Evaluate reasoning at the completed-task boundary, not by the polish or length of a trace. Check accepted outcomes and evidence coverage first; then examine routing regret, branch diversity, latency, cost, repeated tests, and escalation quality. Treat self-reported confidence as a score until replay against accepted outcomes calibrates it. section 5.1.5

Listing 5.1 ReasoningTrace dataclass for observable reasoning metrics (patterns/reasoning_trace.py)
from dataclasses import dataclass
 
@dataclass
class ReasoningTrace:
    """Observable record of reasoning decisions."""
    query_id: str
    classified_complexity: str  #A
    model_used: str
    output_tokens: int = 0
    reasoning_steps: int = 0
    backtracks: int = 0  #B
    hypotheses_generated: int = 0
    hypotheses_refuted: int = 0
    final_confidence: float = 0.0
    wall_time_ms: int = 0
 
    @property
    def steps_per_1k_output_tokens(self) -> float:  #C
        if self.output_tokens == 0:
            return 0
        return self.reasoning_steps / (self.output_tokens / 1000)
 
    @property
    def hypothesis_refutation_rate(self) -> float:  #D
        if self.hypotheses_generated == 0:
            return 0
        return self.hypotheses_refuted / self.hypotheses_generated
 
    def log(self):
        print(f"  [{self.query_id}] "
              f"complexity={self.classified_complexity} "
              f"model={self.model_used} "
              f"steps={self.reasoning_steps} "
              f"backtracks={self.backtracks} "
              f"confidence={self.final_confidence:.2f} "
              f"steps_per_1k={self.steps_per_1k_output_tokens:.2f} "
              f"time={self.wall_time_ms}ms")

Use the telemetry skeleton as a starting point, adding outcome and evidence fields required by the workload. section 5.1.5

Reasoning is not always the right implementation. If a judgment is stable enough to express in ordinary code, removing model reasoning can be preferable. If the correct business decision is undefined, a trace only records a path through that unresolved decision. section 5.1.5

How Designing AI Agents covers it

How deep it goes
Develops routing, evidence-based evaluation, hypothesis testing, and observable traces, while separating model reasoning from decision records and action evidence.
Examples
Uses a code-review agent's renaming refactor and a reasoning telemetry dataclass to illustrate verification and observation.
What you can do after
Enables readers to evaluate reasoning using accepted outcomes, evidence coverage, routing regret, calibration, and cost.
How it connects
Connects reasoning to perception, memory, action, reflection, and governance in an agent architecture.

Also explained in

Compare how 2 books explain reasoning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.