liveBook · Manning

Reinforcement-learning search

As explained in Self-Improving Agents

A search method that uses reinforcement learning to improve candidate artifacts or policies based on outcomes.

Reinforcement-learning search treats an artifact as a policy and improves it based on outcomes. It fits policy-shaped artifacts such as agent memory and tool selection; prompts and code are usually better suited to other search families. section 1.4.2

Two examples show how the method applies:

  • MemRL learns values over episodic memory, favoring entries that have paid off on similar tasks. section 1.4.2
  • GRPO learns which tools work best for task types by comparing groups of trajectories. section 1.4.2
Figure 1.11 The search family arranged by cost against improvement reach. Hill-climbing is the cheapest and most limited, pairwise and population methods cost more and reach further, reinforcement-learning searches fit policy-shaped artifacts, and evolutionary archive search is the most expensive and reaches the furthest. Meta-search sits apart, since it changes the search itself rather than spending more effort on the same curve.

Shows where reinforcement-learning search fits among search families by cost, improvement reach, and artifact fit. section 1.4.2

How Self-Improving Agents covers it

How deep it goes
Defines the artifact as a policy and names two applications, but gives no implementation procedure.
Examples
Names MemRL for memory values and GRPO for learning tool choice from groups of trajectories.
What you can do after
Understand when to use reinforcement-learning search: for policy-shaped memory or tool-selection artifacts, not prompts or code.
How it connects
Connects reinforcement-learning search to episodic memory, tool selection, and alternative search families.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.