An unattended agent can fail quietly: errors accumulate, outputs drift, and a broken build or security incident may be the first visible symptom. Observability collects and analyzes signals that make those failures visible early enough to investigate and intervene. It does not make an agent smarter or its output inherently trustworthy. section 5.3.4 section 5.0
Removing continuous human supervision also removes a feedback mechanism. Replace it with checks and signals, while retaining human review where it can catch systematic drift. section 5.3.4
A practical control set combines detection with a response:
- Validate outputs against expectations and flag anomalous agent behavior. section 5.3.4
- Keep audit logs for post-hoc debugging and stop pipelines with circuit breakers when quality degrades. section 5.3.4
- Review sampled outputs regularly to catch systematic drift that automated checks may miss. section 5.3.4
With multiple agents, a failure can cross several artifact handoffs. A testing agent’s problem may originate in an ambiguous design document or an underspecified prompt upstream. Track agents and their artifacts together; a single conversation log is no longer enough to reconstruct the cause. section 5.6.2 section 5.6.7
Pipeline: payment-retry-feature Status: Running (7m 23s) [✓] Architect (2m 14s, 12k tokens) → design.md [✓] Implementer-1 (3m 02s, 28k tokens) → resilience.py [◐] Implementer-2 (4m 11s, running) → gateway changes [ ] Tester-1 (waiting) [ ] Tester-2 (waiting) [ ] Documenter (waiting) Total tokens: 47k ($0.42) Estimated remaining: 35k tokens, 5 minutes
The pipeline view shows agent status, duration, token consumption, and artifacts. These signals expose stalls, waiting work, and potential cost or throughput bottlenecks. section 5.6.2
Use the signals to distinguish a transient failure, where a retry might help, from a systematic one, where retries may only consume more tokens. Compare stage durations too: fast parallel implementation offers little benefit if every agent waits behind a slower planning or review stage. section 5.6.2
For deployed code, logs record events, traces show the path of a particular request, and metrics summarize behavior over time. Parse latency percentiles, error rate, and input-size distribution can reveal changes after deployment. A p99 latency spike points to slow cases; a higher error rate points to cases that now fail. section 8.7.2
Listing 8.24 Logging example
def parse_markup(text: str) -> str:
start_time = time.time()
if len(text) > 10000:
logger.warning(f"Large input: {len(text)} chars")
try:
result = _do_parse(text)
except Exception as e:
logger.error(f"Parse failed: {e}", extra={'input_sample': text[:100]})
raise
duration = time.time() - start_time
if duration > 0.1:
logger.warning(f"Slow parse: {duration:.2f}s for {len(text)} chars")
return resultThe parser logs unusually large inputs, exceptions, and slow parses. Those events give investigators concrete starting points when generated code behaves unexpectedly. section 8.7.2
Treat production requests as continuous evaluation, not proof that unobserved cases are safe. Invest in instrumentation before scaling orchestration, and keep early workflows conservative and well understood. Observability is one part of the operational foundation alongside rollback and incident response. section 8.7.2 section 5.6.7 section 5.7
























