liveBook · Manning

Chapter 11: Patterns in the wild: Three end-to-end case studies

What you will learn in chapter 11 of Designing AI Agents

The introduction illustrates evidence-driven architecture changes and the need for exact rules at agent system boundaries.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The introduction illustrates evidence-driven architecture changes and the need for exact rules at agent system boundaries.

Key terms: self-review loop, structural checks, repair attempt, authority, shared state, external effects, claims of completion, evidence

11.1 How to read an architecture case

This section frames how to interpret architecture cases without mistaking context-bound evidence for a universal rule.

Key terms: evidence class, workload, operating conditions, architecture action

11.1.1 Five evidence classes

This section distinguishes five evidence classes by the claims they support and the limits they place on architectural conclusions.

Key terms: evidence classes, architecture action, deterministic fixture, field account, illustrative design exercise, repository-executed case, code, fixtures, expected outcomes, recorded benchmark

11.1.2 Five architecture actions

This section distinguishes five evidence-driven architecture actions and explains how to record changes to decisions.

Key terms: Keep, Repair, Recompose, gates, operating point, consequential seam, Simplify, Retire, architecture decision ledger, accepted choice

11.1.3 A zero-pattern control: When evidence says stop

An illustrative support workflow shows when to keep a deterministic baseline without adding agent patterns and what evidence would reopen that decision.

Key terms: deterministic route, general agent loop, reasoning or collaboration deficit, exception volume, policy surface, cross-session causal reasoning, tool choice

11.2 Case 1: From tool calls to a state-preserving execution chain

This section introduces a salary-template execution task and distinguishes the practitioner account from the case’s teaching reconstructions.

Key terms: salary-group template, identifier, snapshot, import, attributed practitioner account, teaching reconstructions, authoritative readback

11.2.1 Bound the completed task

This section bounds the salary-group task and explains why business identifiers need different ownership from narrative meaning.

Key terms: salary group, authenticated user, business system, tenant, business identifier, business identity, narrative meaning, state transitions

11.2.2 The smallest plausible baseline

The section describes a tool-using ReAct baseline and identifies its lack of runtime enforcement for known task dependencies.

Key terms: domain APIs, tools, Skill, ReAct loop, tool result, dependencies, matching, import, snapshot, recovery path

11.2.3 Failure one: Fixed steps drift

This section explains how Plan-and-Execute addressed skipped steps and dependency drift while leaving identifier provenance unresolved.

Key terms: Skill, ReAct, Plan-and-Execute, orchestrator, task DAG, scheduler, predefined workflow, tasks, dependencies, task graph

11.2.4 Failure two: A valid argument loses its origin

This section explains how a valid import call can lose the origin of a business identifier and why conversation history is insufficient as its source of truth.

Key terms: template_id, JSON Schema, business identifier, task graph, node dependencies, source of truth, business identity

11.2.5 The trace localizes the handoff

The run trace localizes the loss of a required identifier to the handoff between a matching tool and the next model call.

Key terms: trace, matching tool, identifier, model call

11.2.6 Three runtime structures for three kinds of truth

This section explains how three runtime structures separately preserve task progress, exact business values, and the narrative context of a run.

Key terms: Workspace, task graph, dependencies, SessionState, state coordinates, tool receipt, template_id, snapshot_id, contract, preconditions

11.2.7 Replay the task with the new ownership

Replays a salary-group request to show how runtime state, approval, execution, and verification preserve continuity across agent handoffs.

Key terms: ReAct, Tool Dispatch, SessionNarrative, SessionState, template_id, snapshot, Progress Tracking, approval receipt, snapshot_id, import_receipt

11.2.8 Evidence, decision, and reopen conditions

This section evaluates a two-stage architectural recomposition, compares narrower alternatives, and sets limits and reopening conditions for the decision.

Key terms: planned task graph, SessionState, parameter binding, Prompt Chaining, Plan-and-Execute, ReAct, Fan-Out/Gather, predefined route, bounded replay, state contracts

11.3 Case 2: Deciding what a modeling agent should judge

This introduction presents AI4MBSE's modeling context and the design changes explored in its research prototype.

Key terms: model repository, typed elements, relationships, identifier, checked plans, model self-review, structural checks

11.3.1 Start with the model behind the picture

The section defines a modeling task’s correct result, bounds its changes, and preserves unfinished work while awaiting location clarification.

Key terms: controller, existing sensor, diagram, diagram type, resolved project location, clarification, Progress Tracking, saved operation, current stage

11.3.2 Separate the work at its dependencies

This section separates model and diagram generation into dependency-ordered stages that pass checked artifacts and receive stage-specific context.

Key terms: Prompt Chaining, elements, relationships, view, checked artifacts, runtime, shared state, planner, independent agents, context assembly

11.3.3 Remove a review loop when the check can be computed

This section replaces an unproductive model self-review cycle with computable checks, bounded repair, and review reserved for qualitative judgments.

Key terms: post-write self-review cycle, usable writes, bounded repair, accepted element set, dangling relationship, naming quality, modeling granularity, type mapping, project location

11.3.4 Make the write and its result observable

This section explains how to make model writes observable and evaluate their correctness across the repository, view, and application boundaries.

Key terms: write path, undo handle, no-op, model repository, canvas, completion contract, acceptance conditions, boundary, semantic check, type translation

11.3.5 Reuse the division of responsibility

This section explains how to reuse and revisit the modeling case’s division of responsibility in other applications.

Key terms: semantic proposals, acceptance checks, Prompt Chaining, Progress Tracking, Guardrail Sandwich, completion boundary, stored state, decision ledger, accepted scope, custom runtime

11.4 Case 3: Operating evidence reopens the architecture

Introduces a deployed data agent and explains why changing operating conditions can reopen an accepted architecture.

Key terms: deployed architecture, original evaluation, operating evidence, data discovery, business definitions, SQL generation, query plans, user's permissions, inspectable result

11.4.1 One question crosses several boundaries

This section uses an illustrative analytics question to show why explaining a change requires contextual definitions, authorized data, and traceable evidence, not just plausible SQL.

Key terms: schema metadata, lineage, business definition, metric definition, state, authority, evidence, data snapshot

11.4.2 Two operating mismatches changed the design

This section explains how a data-agent team repaired overlapping tools and brittle instructions while preserving consequential boundaries, and how to compare those repairs with the earlier design.

Key terms: capability surface, tool calls, higher-level goal guidance, control contract, execution path, precondition, search strategy, permissions, data provenance, evidence of completion

11.4.3 Evals make drift visible

The section explains how SQL evaluations expose drift and guide architecture changes while preserving access and answer-inspection boundaries.

Key terms: golden SQL, golden set, canaries, first deviation, trace, result artifacts, architecture action, pass-through access, assumptions, execution steps

11.4.4 Decision and reopen condition

The case records tool and guidance changes while using evaluations to decide when architecture decisions need reconsideration.

Key terms: capability surface, overlapping tools, prescriptive guidance, eval-driven reopen mechanism, representative workload, failure cluster, decision ledger

11.5 Cross-case analysis

11.5.1 What the cases teach us

The section draws five evidence-driven principles for choosing, connecting, simplifying, and revisiting agent architectures.

Key terms: diagnosed deficit, Plan-and-Execute, Prompt Chaining, deterministic check, intermediate artifact, SessionState, accepted element set, review queue, parallel work, accepted elements

11.6 A failure taxonomy for composed agents

This section introduces diagnosing composed-agent failures by tracing contract deviations to the relevant architectural layer, illustrated by two case examples.

Key terms: failure trace, contract, diagnostic layers, seam, reviewer, relationship endpoints

11.6.1 Layer 1: Problem and contract

Diagnoses problem-and-contract failures and describes when to reassess, revise, or retire an architecture decision.

Key terms: external completion boundary, internal evaluator, accepted scope, authority, ASSESS, Task Completion Contract, acceptance gate, model control

11.6.2 Layer 2: Task-pattern fit

This section explains how to recognize a task-pattern mismatch and evaluate a replacement composition.

Key terms: task-pattern fit, dependencies, required judgment, pattern's preconditions, ROUTE, SELECT, baseline, candidate, bound workload

11.6.3 Layer 3: Pattern seam

This section explains how to identify and repair pattern seam failures, using approval and recovery safeguards from the salary replay.

Key terms: seam failure, Task, State, Authority, Evidence, receipt, template version, digest, approval, requesting principal

11.6.4 Layer 4: Parameter and model

This section explains how to diagnose and compare parameter or model changes within a valid architecture while respecting the accepted card.

Key terms: evidence path, loop limit, threshold, model, topology, simpler baseline, accepted card, authorized ranges, superseding card, accepted claim

11.6.5 Layer 5: Workload and scale

This section explains how workload and scale changes can invalidate earlier evaluation results and prompt a new architecture comparison.

Key terms: workload, concurrency, latency tail, operating point, stratify, selection card, hierarchy, compaction, bound comparison

11.7 How much evidence is enough

This section explains how to judge whether evidence is sufficient to support an agent architecture decision at the accepted level of risk.

Key terms: mechanism, handoff test, end-to-end evidence, operating evidence, authoritative readback, evidence class, evidence coverage, architecture claim, sample count, confidence and risk policy

11.8 The architecture decision ledger

This section explains how an architecture decision ledger preserves evidence and decision history as agent architectures change.

Key terms: Architecture Decision Record, architecture decision ledger, Evaluation Receipt, Pattern Selection Card, operating evidence, earliest affected phase

11.9 Taking the patterns into your own work

Apply agent patterns through small, evidence-driven tests, preserve evaluations, and share decisions with their operating conditions and limits.

Key terms: pattern map, failed assumption, trace, operating conditions, evaluation set, workload, conditions, limits, record

11.10 Summary

11.11 References

Lists sources referenced in the chapter’s architecture case studies and analysis.

Concepts explained in this chapter

← Chapter 10: Composing patterns: From map to terrain

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.