chapter five

5 Reasoning: How your agent decides what to do next

 

This chapter covers

  • Separating private model reasoning, decision records, and action-and-evidence traces
  • Building explicit claim-and-evidence chains without treating generated rationales as proof
  • Routing by difficulty, consequence, evidence needs, latency, and cost
  • Preserving genuinely independent alternatives with bounded parallel search
  • Testing hypotheses against external state while keeping recovery authority separate
  • Composing the four reasoning patterns around an inspectable decision contract
"Solving a problem simply means representing it so as to make the solution transparent."

— Herbert Simon, The Sciences of the Artificial (1969)

An architect I collaborate with wrote to me while his team was working through a major iteration of an enterprise SaaS agent [1]. The team had already put firm contracts on one side of the model-system boundary. At important handoffs, the model had to return an enum, a schema-conformant object, or another signal that ordinary software could parse and execute. With enough care, that direction worked.

5.1 What is reasoning? From represented state to an inspectable decision

5.1.1 Private reasoning, decision records, and execution traces

5.1.2 How the reasoning landscape changed

5.1.3 What remains an architectural decision

5.1.4 The reasoning patterns at a glance

5.1.5 Testing and observing reasoning

5.2 Pattern: Chain-of-Thought

5.2.1 Making the decision path explicit

5.2.2 In production: Separate the decision record from the action trace

5.2.3 Building it

5.2.4 Argus integration

5.2.5 When it breaks

5.3 Pattern: Complexity-Based Routing

5.3.1 Route on difficulty, override on consequence

5.3.2 In production: Route the decision, not only the model

5.3.3 Building it

5.3.4 Argus integration

5.3.5 When it breaks

5.4 Pattern: Parallel Exploration

5.4.1 A concrete branch decision: Independence before concurrency

5.4.2 Branching, scoring, and pruning

5.4.3 Building it

5.4.4 When it breaks

5.5 Pattern: Iterative Hypothesis Testing

5.5.1 The hypothesis-experiment loop

5.5.2 In Production: How coding agents actually debug

5.5.3 Building it

5.5.4 Argus integration