liveBook · Manning

Experience replay

As explained in Designing AI Agents

A pattern that uses past execution trajectories and relevant parts of their history to guide later work.

Experience replay uses a small, relevant part of past execution history to guide later planning or diagnosis. It separates raw evidence from the advice placed in an agent’s live context: a trace may suggest a lesson, but the lesson needs evidence before it can be reused. section 7.5

Feedback arrives on different clocks. A coding agent may compile its output, run tests, and inspect a deployment log within minutes. An operations recommendation may depend on people acting and a later business outcome. Both runs can be recorded immediately; neither recording alone establishes a reusable lesson. section 7.5

Keep three decisions separate:

  • Record execution traces as raw evidence when a run ends. section 7.5
  • Admit a candidate as a replayable heuristic only with scope, provenance, and supporting evidence, such as independent cases or causal evidence. section 7.5 section 7.6
  • Recall an admitted heuristic only when the task and trigger conditions match. Keep candidates out of live context and admitted lessons advisory. section 7.5
Figure 7.4 The Experience Replay hierarchy and recall path. Raw traces support candidate lessons; evidence admits scoped heuristics; task and trigger filters decide whether a replayable lesson enters live context.

The hierarchy separates traces, candidate lessons, and admitted heuristics; the recall path filters advice before it reaches live context. section 7.5

Recall can occur before planning for selected high-risk or repeated tasks, or during recovery after a relevant trigger. Measure retrieval, use, decision effect, later outcome, conflict, and context cost. Cross-run review matters: a locally useful safeguard can add cost or conflict when combined with others. section 7.9 section 7.5

Experience replay carries advice across tasks; a skill package carries an admitted executable capability. Both can draw candidates from traces, but each has a separate admission path. Tests, policy, gold labels, and production boundaries remain protected from the loop that changes behavior. section 7.6

How Designing AI Agents covers it

How deep it goes
Separates recording, admission, and recall, with evidence requirements and measurement; leaves recalled lessons advisory.
Examples
Contrasts coding-agent test feedback with operations recommendations whose business outcomes arrive later.
What you can do after
Evaluate whether a lesson has enough evidence for admission and matches a later task before recalling it.
How it connects
Connects reflection and hierarchical organization with incident postmortems, case-based reasoning, and replay buffers.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.