Chapter 3: Perception: What your agent sees determines what it does
The chapter introduction shows why an agent can fail despite having the right evidence: perception depends on how useful information is selected and presented, not merely on how much context is supplied.
After this chapter you will be able to
- Perception and usable evidence. Explain that perception selects, prioritizes, and assembles information before the model reasons, and that evidence can be present in the context window yet fail to influence the agent's answer.
- Useful context as bottleneck. Explain why useful context, rather than raw model speed or context-window size, can constrain an agent's performance, and why architects must decide what to include and exclude.
- Perception As Budget Gate. Explain why both admitting and excluding tokens are perception decisions that commit the context budget.
- Positional risk in agent context. Explain the U-shaped pattern of information use in long contexts and how accumulating tool results can leave important evidence in a less reliably used middle region. Distinguish that risk from the information being erased.
- High-signal context selection. Explain why perception is an architectural trade-off: the aim is to supply the smallest coherent set of high-signal tokens that improves the next reasoning step, not to use all available context capacity.
- Iatrogenic context and quality. Describe how reasoning quality rises with useful context, reaches a task-dependent peak, and can fall when additional material dilutes attention. Explain iatrogenic context as supposedly helpful information that worsens performance.
- Session context priority. Explain how Claude Code's layered instructions shape what enters a fresh session, and why priority does not itself enforce a rule.
- Signal-based output routing. Apply the section's qualitative signal-to-noise rule to decide when output should go directly into context, be isolated and summarized, or be pre-filtered.
- Repeated reads as warning signals. Explain what the re-read ratio tracks and why a sustained increase after a triage or compaction change warrants investigation.
- Cost per correct outcome. Describe how to calculate token spend per successful outcome and what must remain true for a reduction to count as an improvement.
- Critical information at placed positions. Explain how attention-zone coverage tests whether the model can use critical information where the pipeline places it.
- Controlled perception policy comparisons. Describe the measurement discipline for judging a change to a perception policy.
- Priority-based context routing. Describe how Context Triage classifies available information into priority tiers and fills the context window from highest priority down, leaving lower-priority information outside until needed.
- Context evidence admission. Explain how Context Triage applies prioritization to an agent's context window and why a larger window does not remove the need for an explicit selection policy.
- Four context priority tiers. Describe what happens to information in each Context Triage tier, from always-included P0 items to P3 items retrieved only when needed. Explain that the percentage allocations are examples to calibrate, not fixed rules.
- Safe deferral boundary. Explain when an item may be assigned to P3 and how its priority can change at runtime without removing protected evidence.
- Configured context priorities. Describe how Claude Code implements context triage by assigning different priorities to security rules and personal preferences.
- Protection of failure evidence. Explain why evidence from failed actions should stay in context while a decision is active, and who controls when that protection ends.
- Priority-based context ordering. Explain how the sort key orders context items: priority comes first, error traces move up within their tier, and more content wins among otherwise equal items. Distinguish that last, crude tiebreak from the recency or relevance scoring used in production systems.
- Deferrable item retrieval handles. Describe what happens to deferrable items instead of loading them into the context window.
- Misclassified context priorities. Explain how a supposedly deferrable source can turn out to be essential, forcing another retrieval and reasoning cycle, and why uncertain cases should be promoted.
- Overly restrictive context budgets. Describe how aggressive triage can leave an agent without needed dependencies, producing false positives, and how repeated retrieval can signal that the budget is too tight.
- Purpose of Semantic Compaction. Explain when Semantic Compaction is needed and how it differs from Context Triage: it compresses accumulated history during a long session, rather than deciding what enters the window at the start.
- Decision Critical Evidence. Explain why compression must preserve decision-critical evidence, including some details word for word, rather than replacing it with a vague summary. Describe the failure signal given when an agent asks for information already in the transcript.
- Failure Evidence Across Compaction. Explain why a compacted debugging history must retain the evidence that ruled out earlier approaches, even when the full transcript still exists.
- Lossy Summaries And Reloading. Describe what a compacted summary must support when the agent returns to a live case: preserving what matters for the next decision and using pointers to reload original material.
- Lightest sufficient compaction. Explain why the agent should use the lightest compaction level that frees enough space, and describe how tool result clearing replaces completed outputs with short summaries without losing access to the originals.
- Recent-turn preservation tradeoff. Describe how conversation summarization condenses older turns into a structured recap while leaving recent turns verbatim, and explain the tradeoff in choosing how many turns to preserve.
- Progressive summary fidelity loss. Explain how progressive summarization compresses turns and then groups of summaries for very long tasks, and why repeated rounds can distort the original information.
- Decision-critical evidence protection. Explain which evidence a compaction policy must protect, when it can be summarized, and why compacting it too early is harmful. Describe how protection can be defined for the workload rather than limited to code errors.
- Two-stage semantic compaction. Explain why the agent clears tool results before summarizing the conversation, and how the summary prompt affects what the agent retains.
- Compaction failure signals. Describe the recovery work that can signal failed compaction and explain how to evaluate and respond to it.
- Safe compaction boundaries. Explain why compaction should wait until a completed conversation segment rather than interrupt an active reasoning chain.
- Evidence-led context expansion. Explain how Progressive Discovery starts at the task boundary, follows evidence-bearing links, and loads broader structure only when the next decision requires it.
- Discovery cost placement. Distinguish per-task search, session setup, and a persistent index by where each pays the cost of understanding the environment, including the index's freshness obligation.
- Evidence-guided context search. Trace how a pull-request reviewer can build working context from the diff and changed files, using each finding to decide what to inspect next. Explain why the search can stop once the evidence supports a reasonable next move and resume or change direction as new evidence arrives.
- Broad-to-deep information search. Explain why an agent scans broadly before inspecting promising information more deeply when useful information is clustered.
- Discovery workload tradeoffs. Explain why the choice between on-demand foraging, persistent indexing, and disposable task context depends on the workload.
- Staged Agent Skill disclosure. Explain how Agent Skills expose information in forage, focus, and deepen tiers: the agent initially sees visible Skill names and permitted descriptions, loads SKILL.md when a task uses a Skill, and loads referenced files on demand.
- Unstructured Project Fallback. Explain why progressive discovery struggles in projects without useful file structure and when to switch from name-based searching to content search.
- Hidden File Locations. Explain why important files in unexpected locations can defeat keyword search and how project documentation can help locate them.
- Decision Driven Representation Choice. Explain why the representation of an input should depend on the evidence the next decision needs, using the Blueprint and build-log examples.
- Input form and relationships. Explain how an input’s representation can reveal or conceal relationships, determining which relationships remain available to an agent’s reasoning.
- Decision-preserving input representation. Explain how to choose between keeping an image as a vision input and converting it to compact text, based on the evidence the decision requires.
- Auditable representation decisions. Explain why the implementation makes each representation decision explicit and logged, and describe how its spatial-information test should change for production.
- Decision-relevant representation loss. Explain how converting an image to text or a table can preserve some information while losing relationships needed for a decision, and when to keep the source alongside a summary.
- Deployed image cost calibration. Describe why image-token estimates must be measured for the deployed configuration and checked against the downstream task.
- Perception pipeline composition. Trace the full perception pipeline from progressive discovery through multi-modal fusion, context triage, and semantic compaction, explaining that each active stage refines the previous stage’s output.
- Task-dependent pipeline intensity. Explain why stages may be omitted or used at different intensities, and describe the shift from early acquisition through progressive discovery to later management through semantic compaction.
- Evolving perception across cycles. Trace how Claude Code moves from finding candidate files to reading relevant files, acting on the bug, and verifying the test, while changing which information receives priority.
- Decision-relevant context density. Explain why the example favors context containing evidence needed for the current decision over simply providing a larger window.
- Four-step context discovery. Trace how Argus discovers review context: it extracts modified files from the diff, follows their Python imports, finds relevant tests, and loads project configuration.
- Priority-based context triage. Explain how Argus prioritizes discovered files and fits them into a token budget, including what happens when a file does not fit.
What it builds on
- Design the cognitive engine: The Perception-Reasoning-Action loop (chapter 2) · The Perception-Reasoning-Action loop places perception before reasoning.
- Set trust boundaries before granting autonomy (chapter 2) · Trust boundaries before granting autonomy.
- How information flows through cognitive modules (chapter 2) · Assumes that information flows into the agent's cognitive modules.
- The execution topology axis (chapter 2) · The route execution topology as a way to direct work along different paths.
- The PRA loop in 50 lines (chapter 2) · Argus's original diff-only PRA skeleton
Section by section
Introduction
The chapter introduction shows why an agent can fail despite having the right evidence: perception depends on how useful information is selected and presented, not merely on how much context is supplied.
- Perception and usable evidence: Explain that perception selects, prioritizes, and assembles information before the model reasons, and that evidence can be present in the context window yet fail to influence the agent's answer.
Why it matters: This helps distinguish a failure to present usable evidence from a failure to obtain it. - Useful context as bottleneck: Explain why useful context, rather than raw model speed or context-window size, can constrain an agent's performance, and why architects must decide what to include and exclude.
Why it matters: It explains why a larger window alone does not remove the need to make deliberate context choices.
Key terms: perception, context window, useful context, perception engineering, bottleneck
3.1 The context window as a treatment budget
This section explains why a context window is not uniform like RAM and why perception must decide what enters its limited budget.
- Unequal Context Position Treatment: Distinguish a context window from uniform RAM: the position of content in the window affects how it is treated.
Why it matters: Where information appears in context matters, not just whether it fits. - Perception As Budget Gate: Explain why both admitting and excluding tokens are perception decisions that commit the context budget.
Why it matters: An agent's available context depends on what perception lets in and leaves out.
Key terms: context window, RAM, perception, context budget, token
3.1.1 How attention varies by position
This section explains why information in the middle of a long agent context is less reliably used than information near its beginning or end.
- Positional risk in agent context: Explain the U-shaped pattern of information use in long contexts and how accumulating tool results can leave important evidence in a less reliably used middle region. Distinguish that risk from the information being erased.
Why it matters: An agent can have relevant evidence in its context yet use it inconsistently. - Observable context bloat signals: State the additional observable signals of context bloat identified alongside positional risk.
Why it matters: These signals give you more to watch than whether the agent overlooks evidence in the middle.
Key terms: U-shaped performance curve, dead zone, context, instruction adherence, remaining-capacity estimation, context bloat
3.1.2 The three-way squeeze
This section explains why agent perception must select coherent, decision-relevant context rather than simply fill the context window.
- High-signal context selection: Explain why perception is an architectural trade-off: the aim is to supply the smallest coherent set of high-signal tokens that improves the next reasoning step, not to use all available context capacity.
Why it matters: Choosing what the agent sees is an architectural decision, not a matter of filling available space. - Iatrogenic context and quality: Describe how reasoning quality rises with useful context, reaches a task-dependent peak, and can fall when additional material dilutes attention. Explain iatrogenic context as supposedly helpful information that worsens performance.
Why it matters: Extra information can hide the evidence an agent needs instead of making its decision better. - Cost of irrelevant tokens: Explain how irrelevant context consumes tokens on each uncached call while competing with decision-relevant evidence, and state the conditions for a meaningful token-leverage comparison.
Why it matters: Removing irrelevant context can save money, but savings alone do not show improvement if outcomes or safety worsen. - Perception and governance boundaries: Distinguish perception's input boundary from governance's action boundary: perception filters and ranks evidence entering context, while governance constrains permitted actions and records external effects.
Why it matters: Filtering an agent's evidence and authorizing its actions solve different problems, even if they use similar controls.
Key terms: perception, high-signal tokens, coherence, context window, context fill level, reasoning quality, iatrogenic context, attention, token leverage, uncached call
3.1.3 How production agents solve this
Claude Code manages perception by prioritizing session context, loading Skills on demand, isolating sub-agent work, and routing tool output according to its usefulness.
- Session context priority: Explain how Claude Code's layered instructions shape what enters a fresh session, and why priority does not itself enforce a rule.
Why it matters: It distinguishes controlling what the agent sees now from preserving information and enforcing rules. - On-demand Skill content: Describe how Agent Skills limit startup context while making fuller domain instructions available when needed.
Why it matters: An agent can have access to extensive instructions without paying their full context cost at startup. - Sub-agent context isolation: Explain what context a sub-agent receives and how isolation can keep exploratory output out of the main conversation.
Why it matters: Isolating narrow work can prevent bulky intermediate output from crowding the main agent's context. - Signal-based output routing: Apply the section's qualitative signal-to-noise rule to decide when output should go directly into context, be isolated and summarized, or be pre-filtered.
Why it matters: Choosing the route for tool output makes context management an explicit trade-off rather than allowing every result to crowd the agent's window.
Key terms: context window, CLAUDE.md, instruction hierarchy, auto memory, Perceive-Reason-Act (PRA) cycle, Agent Skills, progressive disclosure, SKILL.md, Knowledge ROI, sub-agents
3.1.4 Measuring perception quality
This section shows how to measure whether an agent’s perception pipeline preserves useful information while controlling cost.
- Repeated reads as warning signals: Explain what the re-read ratio tracks and why a sustained increase after a triage or compaction change warrants investigation.
Why it matters: Repeated reads can reveal when a leaner context makes the agent work to recover missing information. - Cost per correct outcome: Describe how to calculate token spend per successful outcome and what must remain true for a reduction to count as an improvement.
Why it matters: This keeps token savings tied to successful, safe work rather than making fewer tokens an end in itself. - Critical information at placed positions: Explain how attention-zone coverage tests whether the model can use critical information where the pipeline places it.
Why it matters: Including evidence in context is not enough if the model cannot reliably use it where it appears. - Controlled perception policy comparisons: Describe the measurement discipline for judging a change to a perception policy.
Why it matters: Controlled comparisons make it easier to tell whether a perception change actually helps the workload.
Key terms: re-read ratio, triage, compaction, token spend per successful outcome, input tokens, evaluation set, attention-zone coverage, task-specific position tests, workload baseline, perception policy
3.2 Pattern: Context triage
Introduces Context Triage as a way to give each agent judgment the most relevant evidence instead of filling its context with everything available.
- Evidence boundaries for judgments: Explain why the code-security workflow separated its three questions into focused perception stages, each receiving the sources needed for its own judgment.
Why it matters: Separating evidence by judgment can keep growing source material from overwhelming each decision. - Priority-based context routing: Describe how Context Triage classifies available information into priority tiers and fills the context window from highest priority down, leaving lower-priority information outside until needed.
Why it matters: The agent's decisions depend on what it gets to see, not just on how much information exists. - Triage policy and warning signs: State when Context Triage is useful, what its priority policy must decide, and how to recognize that useful evidence may have been ranked too low.
Why it matters: A priority policy needs observable warning signs so it can be adjusted without judging token savings alone.
Key terms: evidence boundary, perception stages, Context Triage, Perception × Route, priority tiers, context window, priority policy, useful context budget, runtime, task success
3.2.1 The emergency room
The emergency-room analogy explains why an agent must prioritize which evidence enters its limited context window rather than admit inputs indiscriminately.
- Rapid priority assessment: Explain how emergency-room triage uses a quick assessment under uncertainty to prioritize patients and allocate resources rather than treating them in arrival order.
Why it matters: The analogy shows why prioritization must happen before all the information is available. - Context evidence admission: Explain how Context Triage applies prioritization to an agent's context window and why a larger window does not remove the need for an explicit selection policy.
Why it matters: Selecting evidence deliberately helps the agent attend to what matters for the task.
Key terms: triage, severity tier, Context Triage, context window, perception policy, class of evidence
3.2.2 The four tiers
This section explains how Context Triage allocates limited context across four priority tiers and how to decide what can safely be deferred.
- Four context priority tiers: Describe what happens to information in each Context Triage tier, from always-included P0 items to P3 items retrieved only when needed. Explain that the percentage allocations are examples to calibrate, not fixed rules.
Why it matters: The tiers make limited context available to the information most useful for the current task. - Safe deferral boundary: Explain when an item may be assigned to P3 and how its priority can change at runtime without removing protected evidence.
Why it matters: Saving context tokens must not come at the cost of an unsafe decision or missing irrecoverable evidence. - Task-specific priority and trace: Explain why an item's priority can differ between tasks and why the classification reason should be recorded.
Why it matters: A recorded reason helps teams correct priorities when an agent misses evidence.
Key terms: Context Triage, P0 (Critical), P1 (Important), P2 (Supporting), P3 (Deferrable), lightweight handles, P3, recoverability, business consequences, policy-protected P0 or P1 evidence
3.2.3 Context triage in Claude Code
The section shows how configured priorities and protected evidence make context triage concrete in production agents.
- Configured context priorities: Describe how Claude Code implements context triage by assigning different priorities to security rules and personal preferences.
Why it matters: Configured priorities keep security rules from being treated like ordinary preferences. - Protection of failure evidence: Explain why evidence from failed actions should stay in context while a decision is active, and who controls when that protection ends.
Why it matters: Keeping consequential feedback visible helps an agent avoid repeating a failure or unsafe action.
Key terms: CLAUDE.md hierarchy, Enterprise level, User level, P0, P2, P1, protected class, release condition, failed actions, observations
3.2.4 Building it
This section explains how context triage orders items for the context window and keeps deferrable items available without loading them.
- Priority-based context ordering: Explain how the sort key orders context items: priority comes first, error traces move up within their tier, and more content wins among otherwise equal items. Distinguish that last, crude tiebreak from the recency or relevance scoring used in production systems.
Why it matters: The ordering determines which context receives attention when the window is limited. - Deferrable item retrieval handles: Describe what happens to deferrable items instead of loading them into the context window.
Why it matters: The agent can leave lower-urgency material out of its current context without losing a way to retrieve it.
Key terms: priority classification, sort key, error traces, recency, relevance scoring, deferrable items, handles list, just-in-time retrieval
3.2.5 When it breaks
Explains two ways context triage can fail and how to recognize or mitigate them.
- Misclassified context priorities: Explain how a supposedly deferrable source can turn out to be essential, forcing another retrieval and reasoning cycle, and why uncertain cases should be promoted.
Why it matters: A small saving in initial context can create a costly extra cycle when the omitted source matters. - Overly restrictive context budgets: Describe how aggressive triage can leave an agent without needed dependencies, producing false positives, and how repeated retrieval can signal that the budget is too tight.
Why it matters: A tighter budget is not a saving if missing context causes mistakes and repeated retrieval.
Key terms: priority classification, deferrable, P3, PRA cycle, promote, context budget, aggressive triage, dependencies, false positives, re-reads
3.3 Pattern: Semantic compaction
This section introduces Semantic Compaction as a way to keep long agent sessions usable without losing evidence needed for the next decision.
- Purpose of Semantic Compaction: Explain when Semantic Compaction is needed and how it differs from Context Triage: it compresses accumulated history during a long session, rather than deciding what enters the window at the start.
Why it matters: It lets an agent continue a long task without treating its entire history as equally necessary. - Decision Critical Evidence: Explain why compression must preserve decision-critical evidence, including some details word for word, rather than replacing it with a vague summary. Describe the failure signal given when an agent asks for information already in the transcript.
Why it matters: A short summary is only useful if it preserves the facts the agent needs to avoid repeating work or burdening a user.
Key terms: Semantic Compaction, Context Triage, context window, decision-critical evidence, verbatim, compaction boundary
3.3.1 The moment the window fills
This section explains why compacting an agent’s context must preserve decision-critical evidence and provide a way to reload original material.
- Failure Evidence Across Compaction: Explain why a compacted debugging history must retain the evidence that ruled out earlier approaches, even when the full transcript still exists.
Why it matters: Preserving the reason an approach failed helps the agent avoid repeating it. - Lossy Summaries And Reloading: Describe what a compacted summary must support when the agent returns to a live case: preserving what matters for the next decision and using pointers to reload original material.
Why it matters: A route back to the originals lets the agent act on current evidence rather than treating a lossy summary as complete.
Key terms: context compaction, active context, decision-critical evidence, traces, lossy, pointers, original material, current state, source evidence, constraints
3.3.2 Three levels of compression
The section explains three increasingly aggressive ways to compact an agent’s context while retaining information needed for its current work.
- Lightest sufficient compaction: Explain why the agent should use the lightest compaction level that frees enough space, and describe how tool result clearing replaces completed outputs with short summaries without losing access to the originals.
Why it matters: You can save context space without discarding more information than necessary. - Recent-turn preservation tradeoff: Describe how conversation summarization condenses older turns into a structured recap while leaving recent turns verbatim, and explain the tradeoff in choosing how many turns to preserve.
Why it matters: Keeping the right recent context lets the agent continue coherently after making room. - Progressive summary fidelity loss: Explain how progressive summarization compresses turns and then groups of summaries for very long tasks, and why repeated rounds can distort the original information.
Why it matters: More aggressive compression can create room for long tasks, but its recap becomes less faithful to what originally happened.
Key terms: compaction, tool result clearing, conversation summarization, structured recap, recent turns, progressive summarization, fidelity, compression, drift
3.3.3 Protect decision-critical evidence
This section explains how to keep decision-critical evidence intact during compaction so an agent can learn from the results of its actions.
- Decision-critical evidence protection: Explain which evidence a compaction policy must protect, when it can be summarized, and why compacting it too early is harmful. Describe how protection can be defined for the workload rather than limited to code errors.
Why it matters: Keeping consequential evidence intact lets the agent make its next decision using what actually happened. - Failed action observation pairing: Explain why compaction should retain a failed action together with its resulting observation.
Why it matters: A result is more useful when the agent can see which attempted action produced it.
Key terms: compaction policy, protected class, decision-critical evidence, full fidelity, protection predicate, retention policy, failed action, resulting observation
3.3.4 Building it
Explains how to compact an agent’s context while limiting information loss and preserving what matters.
- Two-stage semantic compaction: Explain why the agent clears tool results before summarizing the conversation, and how the summary prompt affects what the agent retains.
Why it matters: Compaction can free context without unnecessarily discarding information the agent needs.
Key terms: two-level strategy, tool results, summary prompt, information loss
3.3.5 When it breaks
This section shows how to detect and mitigate semantic compaction failures, especially when compaction interrupts unfinished reasoning, and clarifies what compaction manages.
- Compaction failure signals: Describe the recovery work that can signal failed compaction and explain how to evaluate and respond to it.
Why it matters: Measuring recovery work helps reveal when a shorter context is costing the agent more work than it saves. - Safe compaction boundaries: Explain why compaction should wait until a completed conversation segment rather than interrupt an active reasoning chain.
Why it matters: Choosing the boundary carefully preserves the reasoning the agent still needs to finish its current step. - Semantic compaction scope: Distinguish what Semantic Compaction manages from what Context Triage and cross-session memory manage.
Why it matters: The distinction keeps a current-session summary from being mistaken for a persistent memory strategy.
Key terms: Compaction failure, summary-to-source retrieval, uncompacted evaluation baseline, compaction boundary, active reasoning chain, completed conversation segments, decision point, Semantic Compaction, Context Triage, current session
3.4 Pattern: Progressive discovery
Introduces Progressive Discovery as a way to gather task-relevant evidence without loading an entire environment, and frames the choice of when to pay for discovery.
- Evidence-led context expansion: Explain how Progressive Discovery starts at the task boundary, follows evidence-bearing links, and loads broader structure only when the next decision requires it.
Why it matters: This keeps the agent focused on evidence it can use for the current decision. - Discovery cost placement: Distinguish per-task search, session setup, and a persistent index by where each pays the cost of understanding the environment, including the index's freshness obligation.
Why it matters: The choice affects request latency, reuse costs, and whether the agent relies on current evidence.
Key terms: Progressive Discovery, task boundary, evidence-bearing links, working set, per-task search, session setup, persistent index, freshness obligation, missed dependencies
3.4.1 From an entry point to working context
Shows how progressive discovery builds working context from a task-specific entry point, seeking enough evidence for the next move rather than loading everything.
- Evidence-guided context search: Trace how a pull-request reviewer can build working context from the diff and changed files, using each finding to decide what to inspect next. Explain why the search can stop once the evidence supports a reasonable next move and resume or change direction as new evidence arrives.
Why it matters: It helps an agent find useful context without treating the entire repository as equally relevant. - Discovery versus existing-context work: Distinguish Progressive Discovery from Context Triage and Semantic Compaction by saying whether each works on available material or decides what material to seek next.
Why it matters: The distinction clarifies whether the agent needs to manage context it has or find context it lacks.
Key terms: Progressive Discovery, working context, bounded rationality, Context Triage, Semantic Compaction
3.4.2 Forage, then focus, then deepen
Explains how progressive discovery narrows an agent’s search from broad, cheap scans to targeted, deeper inspection while keeping token costs low.
- Broad-to-deep information search: Explain why an agent scans broadly before inspecting promising information more deeply when useful information is clustered.
Why it matters: This explains why an agent should narrow its search before spending tokens on detailed inspection. - Token cost and search depth: Describe how the example’s three phases trade more tokens per operation for fewer operations, producing relevant information at a modest total token cost.
Why it matters: The cost tradeoff shows how targeted exploration can improve the signal an agent gets from its token budget.
Key terms: progressive discovery, information foraging theory, patches, token cost per operation, token leverage, glob, grep
3.4.3 The production spectrum
This section compares how production agents discover codebase context and explains when on-demand exploration, indexing, or task-specific context is useful.
- Production discovery approaches: Describe how Claude Code and Cline discover codebase context without relying on a pre-built index, including Cline’s separation of read-only planning from approved action.
Why it matters: These examples show that discovery can happen as part of the task rather than through a pre-built index. - Session and persistent indexes: Distinguish Cursor’s session-level repository index from Augment’s persistent, synchronized index across repositories.
Why it matters: The two examples show different ways to pay indexing costs ahead of later searches. - Discovery workload tradeoffs: Explain why the choice between on-demand foraging, persistent indexing, and disposable task context depends on the workload.
Why it matters: Choosing a discovery strategy means balancing repeated exploration costs against the needs of the current task.
Key terms: on-demand foraging, Plan mode, Act mode, session-level indexes, vector embeddings, persistent organizational indexes, Context Engine, persistent index, disposable task contexts, freshness
3.4.4 Building it
3.4.5 The progressive disclosure connection
The section shows how Claude Code’s Agent Skills apply progressive discovery by loading skill information in stages.
- Staged Agent Skill disclosure: Explain how Agent Skills expose information in forage, focus, and deepen tiers: the agent initially sees visible Skill names and permitted descriptions, loads SKILL.md when a task uses a Skill, and loads referenced files on demand.
Why it matters: This explains how a Skill can provide deeper context without loading all its material at session start.
Key terms: Agent Skills, progressive discovery, forage-focus-deepen, YAML header, markdown body, SKILL.md
3.4.6 When it breaks
Explains where progressive discovery breaks down and how to respond when file names or project structure do not lead the agent to useful evidence.
- Unstructured Project Fallback: Explain why progressive discovery struggles in projects without useful file structure and when to switch from name-based searching to content search.
Why it matters: A quick change in search strategy keeps the agent from spending its budget on irrelevant files. - Hidden File Locations: Explain why important files in unexpected locations can defeat keyword search and how project documentation can help locate them.
Why it matters: Project documentation can point the agent to evidence that its search terms would miss. - Text Evidence Boundary: State the evidence-format limit identified for the first three perception patterns.
Why it matters: Recognizing a text-only assumption helps the reader notice when a task falls outside these patterns’ scope.
Key terms: progressive discovery, forage phase, glob, grep-heavy exploration, CLAUDE.md, README.md, forage step, text, Multi-Modal Fusion
3.5 Pattern: Multi-modal fusion
Multi-Modal Fusion chooses how to represent each input so the agent retains the evidence needed for its next decision.
- Decision Driven Representation Choice: Explain why the representation of an input should depend on the evidence the next decision needs, using the Blueprint and build-log examples.
Why it matters: The wrong representation can hide the very relationships or details the agent needs to decide correctly. - Evidence Preservation Checks: Describe how to notice and assess whether a chosen representation has lost necessary evidence.
Why it matters: A compact or converted input is only useful if it preserves the evidence behind the agent’s answer.
Key terms: Multi-Modal Fusion, representation, Unreal Blueprint, build log, conversion, original, outcome accuracy, latency, token cost
3.5.1 When text is not enough
This subsection explains why the form of an agent’s input matters to what it can reason about.
- Input form and relationships: Explain how an input’s representation can reveal or conceal relationships, determining which relationships remain available to an agent’s reasoning.
Why it matters: Choosing an input form can change what the agent is able to notice.
Key terms: representation, input form, relationships, reasoning
3.5.2 The representation decision
This section explains how to choose a compact representation of an input without losing the evidence needed for an agent’s decision.
- Decision-preserving input representation: Explain how to choose between keeping an image as a vision input and converting it to compact text, based on the evidence the decision requires.
Why it matters: A smaller input is useful only if it still lets the agent make the intended decision.
Key terms: spatial relationships, vision input, compact text, token efficiency, decision evidence
3.5.3 Building it
This subsection makes multi-modal representation choices auditable and identifies how to replace a prototype spatial-information test in production.
- Auditable representation decisions: Explain why the implementation makes each representation decision explicit and logged, and describe how its spatial-information test should change for production.
Why it matters: An auditable choice helps you check how each input was represented, while the production test avoids depending on the example's keyword heuristic.
Key terms: representation decision, spatial-information test, keyword heuristic, image description
3.5.4 When it breaks
This section explains how multi-modal fusion can fail through lost decision-relevant relationships or inaccurate image-token budgets, and how to check for both.
- Decision-relevant representation loss: Explain how converting an image to text or a table can preserve some information while losing relationships needed for a decision, and when to keep the source alongside a summary.
Why it matters: A cheaper representation is not useful if it removes evidence the decision depends on. - Deployed image cost calibration: Describe why image-token estimates must be measured for the deployed configuration and checked against the downstream task.
Why it matters: Measured costs and downstream results keep image-routing decisions from being systematically misbudgeted or ineffective.
Key terms: multi-modal fusion, source, compact text summary, downstream error cost, image-token costs, routing policy, representation policy, downstream task
3.6 Putting it all together: The perception pipeline
This section combines four perception patterns into an adaptable pipeline that turns raw information into a selected, token-efficient context window at the start of a PRA cycle.
- Perception pipeline composition: Trace the full perception pipeline from progressive discovery through multi-modal fusion, context triage, and semantic compaction, explaining that each active stage refines the previous stage’s output.
Why it matters: The order shows how the agent moves from gathering information toward a usable context window. - Task-dependent pipeline intensity: Explain why stages may be omitted or used at different intensities, and describe the shift from early acquisition through progressive discovery to later management through semantic compaction.
Why it matters: Perception should change with the task rather than applying every stage equally throughout.
Key terms: perception pipeline, progressive discovery, multi-modal fusion, context triage, semantic compaction, PRA cycle, acquisition, management
3.6.1 Claude Code's perception cycle in action
A four-cycle login-test example shows how Claude Code changes what it keeps in context as it finds, fixes, and verifies a bug.
- Evolving perception across cycles: Trace how Claude Code moves from finding candidate files to reading relevant files, acting on the bug, and verifying the test, while changing which information receives priority.
Why it matters: The example makes perception a repeated decision about what matters now, not a one-time collection of context. - Decision-relevant context density: Explain why the example favors context containing evidence needed for the current decision over simply providing a larger window.
Why it matters: More context is not automatically better if it makes the relevant evidence harder to use.
Key terms: progressive discovery, context triage, semantic compaction, P0, P1, P2, signal density, useful context
3.7 Upgrading Argus: From blind reviewer to context-aware analyst
This section upgrades Argus from reviewing a diff alone to selecting and tracking task-relevant context within a token budget.
- Four-step context discovery: Trace how Argus discovers review context: it extracts modified files from the diff, follows their Python imports, finds relevant tests, and loads project configuration.
Why it matters: The reviewer needs evidence about dependencies, behavior, and conventions, not just the changed lines. - Priority-based context triage: Explain how Argus prioritizes discovered files and fits them into a token budget, including what happens when a file does not fit.
Why it matters: Selecting context prevents useful evidence from being buried in an indiscriminate dump. - Perception decision tracing: Describe what FileContext and PerceptionTrace record and how the pipeline's logging supports perception metrics.
Why it matters: A recorded selection trail makes Argus's perception choices observable rather than opaque. - Perception and memory boundary: Distinguish the context Argus assembles for a current review from information it could reuse across later reviews.
Why it matters: Better context for this review does not by itself give Argus a memory of previous reviews.
Key terms: Progressive Discovery, modified file paths, Python imports, test files, project configuration, Context Triage, P0, token budget, priority, high-attention zone
3.8 Summary
3.9 References
Lists the sources cited in Chapter 3 on agent perception.
Where this is used later
- The Argus checkpoint: From code reviewer to governed system (chapter 9) · Perception as the agent's view of information.
- What is governance? Autonomy without accountability is a liability (chapter 9) · The governance chapter develops the exit-side control boundary distinguished here.
- In production: Claude Code's scoped memory sources (chapter 4) · Provides the session-loading context for Claude Code's scoped memory sources.
- What the terrain adds (chapter 10) · Semantic Compaction as a way to compress context.
- What is memory? State, persistence, and knowledge across sessions (chapter 4) · Separates current-session visibility from evidence that may persist and return across sessions.
- Route in layers (chapter 7) · Provides an example of revealing Skill information in layers.
- What to observe: Metrics locate the problem; trajectories explain the path (chapter 9) · Observability uses recorded evidence and metrics to examine agent behavior.
Concepts explained in this chapter
- perceptionAn agent capability for identifying relevant information from inputs and loading context for subsequent processing.
- progressive discoveryProgressive discovery is a technique in which an agent incrementally obtains relevant information while navigating an unfamiliar information environment.
- context triageAn agent pattern that prioritizes relevant context and avoids loading irrelevant material to reduce unnecessary processing cycles.
- context isolationContext isolation is a technique that gives a task-specific agent a separate context containing only information relevant to its assigned task.
- instruction hierarchyA layered arrangement of instructions in which levels determine their priority and the context loaded for an agent session.
- agent skillA packaged agent capability that specifies how to perform a class of tasks, including relevant tools and workflow guidance.
- progressive disclosureProgressive disclosure is a technique that reveals detailed information or capabilities only when they become relevant to the current task.
- subagentAn agent assigned a bounded portion of a larger task or delegated plan.
- re-read ratioA metric measuring how often an agent retrieves the same file or information more than once during a task.
- token leverageThe ratio of agent output quality to the token cost required to produce it.
- perception pipelineA sequence of perception stages that acquires, combines, prioritizes, and compresses information into context for an agent.
- attention zone coverageA metric measuring whether critical information is positioned in regions of the context window that receive strong model attention.
- context windowThe bounded sequence of tokens that a language model can attend to during an interaction.
- ReActReAct is an agent technique that interleaves language-model reasoning with actions to interact with an environment.
- chain-of-thoughtA reasoning technique that represents a decision as a sequence of intermediate steps or claims.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
