Experience replay uses a small, relevant part of past execution history to guide later planning or diagnosis. It separates raw evidence from the advice placed in an agent’s live context: a trace may suggest a lesson, but the lesson needs evidence before it can be reused. section 7.5
Feedback arrives on different clocks. A coding agent may compile its output, run tests, and inspect a deployment log within minutes. An operations recommendation may depend on people acting and a later business outcome. Both runs can be recorded immediately; neither recording alone establishes a reusable lesson. section 7.5
Keep three decisions separate:
- Record execution traces as raw evidence when a run ends. section 7.5
- Admit a candidate as a replayable heuristic only with scope, provenance, and supporting evidence, such as independent cases or causal evidence. section 7.5 section 7.6
- Recall an admitted heuristic only when the task and trigger conditions match. Keep candidates out of live context and admitted lessons advisory. section 7.5
Figure 7.4 The Experience Replay hierarchy and recall path. Raw traces support candidate lessons; evidence admits scoped heuristics; task and trigger filters decide whether a replayable lesson enters live context.
The hierarchy separates traces, candidate lessons, and admitted heuristics; the recall path filters advice before it reaches live context. section 7.5
Recall can occur before planning for selected high-risk or repeated tasks, or during recovery after a relevant trigger. Measure retrieval, use, decision effect, later outcome, conflict, and context cost. Cross-run review matters: a locally useful safeguard can add cost or conflict when combined with others. section 7.9 section 7.5
Experience replay carries advice across tasks; a skill package carries an admitted executable capability. Both can draw candidates from traces, but each has a separate admission path. Tests, policy, gold labels, and production boundaries remain protected from the loop that changes behavior. section 7.6
