Progressive discovery builds an agent’s working context incrementally when relevant sources are not known in advance. Start at the task boundary, follow evidence-bearing links, and load broader structure only when a decision needs it. Stop when the evidence supports a reasonable next move; resume searching as new evidence appears. section 3.4 section 3.4.1
A typical discovery loop has three phases:
- Forage: scan manifests, file patterns, and keywords with broad, cheap operations. section 3.4.2
- Focus: read the high-value files, tests, and governing configuration identified by the scan. section 3.4.2
- Deepen: follow relevant imports, documentation, schemas, and type definitions. section 3.4.2
Phase 1: FORAGE—broad, cheap operations ├── Read manifests: CLAUDE.md, README, package.json ├── Scan structure: glob for relevant file patterns └── Search content: grep for keywords from the task Phase 2: FOCUS—targeted, full-content reads ├── Read high-value files identified in Phase 1 ├── Read test files for modified modules └── Read configuration that governs behavior Phase 3: DEEPEN—follow references ├── Follow imports in focused files ├── Read referenced documentation └── Load schemas and type definitions
Use this listing to map each phase to concrete repository operations. section 3.4.2
Discovery chooses what to seek next; context triage and semantic compaction manage information already available. One illustrative search costs about 18K tokens: 2K for broad scans, 10K for focused reads, and 6K for following references. These are workload estimates, not guaranteed costs. section 3.4.1 section 3.4.2
Reviewing a code change
Argus starts with the diff, extracts modified paths, follows Python imports, finds related tests, and loads project configuration. It then sorts discoveries by priority and fills a token budget greedily. The trace records selected and dropped files; even a modified file can be skipped if it does not fit. section 3.7
Listing 3.8 Argus triage: fit discoveries into the token budget (ch03-perception/argus/perception.py)
# Triage: sort by priority, fill greedily
priority_order = {"modified": 0, "imported": 1, "test": 1, "config": 2}
contexts.sort(key=lambda c: priority_order.get(c.relevance, 3))
trace.files_discovered = len(contexts)
trace.tokens_considered = sum(c.tokens for c in contexts)
selected, used = [], 0
for ctx in contexts:
if used + ctx.tokens <= budget: #A
selected.append(ctx)
used += ctx.tokens
else:
trace.dropped_files.append((ctx.path, ctx.relevance))
trace.files_selected = len(selected)
trace.files_dropped = trace.files_discovered - trace.files_selected
trace.tokens_selected = used
logger.info(
"Perception: %d/%d files selected (%.0f%% selectivity), "
"%d tokens used of %d budget",
trace.files_selected, trace.files_discovered,
trace.selectivity * 100, used, budget,
)
if trace.dropped_files: #B
logger.warning("Dropped: %s", trace.dropped_files)
return selected, trace
Inspect the budget-selection loop and its record of dropped files. section 3.7
Choose where to pay for discovery. On-demand search starts with fresh data but repeats exploration per task; a session index amortizes it within a project session; a persistent index shifts work into maintained infrastructure and creates a freshness obligation. Measure time to relevant evidence, retrieval cost, and missed dependencies. section 3.4 section 3.4.3
Discovery is less reliable when names and structure provide little signal or critical files live in unexpected locations. If the first five glob results are unhelpful, switch to content searches; maintained manifests and README files can also point to non-obvious locations. Poorly maintained environments remain difficult to navigate. section 3.4.6
