liveBook · Manning

SPO

As explained in Self-Improving Agents

SPO is a pairwise search method that generates and compares candidate variants.

A prompt can produce weak answers even when no correct-answer key exists to grade it against. SPO searches for a better prompt by comparing outputs instead: the current prompt and a challenger answer the same questions, and a pairwise LLM-as-Judge chooses which answers it prefers. That preference supplies the signal without handwritten labels. section 2.3.4

Each round has three steps:

  • Mutate: ask a model to rewrite the current best prompt, using feedback from the previous round. section 2.3.4
  • Score: run both prompts on a small set of questions and tally the pairwise judge’s wins and losses. section 2.3.4
  • Accept: keep the challenger only if it wins more questions than it loses; otherwise retain the current best and try another mutation. section 2.3.4
Figure 2.9 How SPO works. SPO is the search loop of Figure 2.8 made specific: mutate is the propose step, score is the measure step, and accept is the select step. Each round mutates the current best prompt into a candidate, scores both prompts with the pairwise judge, and accepts the candidate only when it wins more than it loses, so the prompt never regresses. The self-supervised callout is the core idea: the only signal is which output wins the comparison, not a score against a labeled answer key, which is what makes SPO the cheap starting rung in Figure 2.7.

The diagram maps mutate, score, and accept onto a propose–measure–select loop. Notice that the gate depends on a preference, not a score against a labeled answer. section 2.3.4

Improving an overly terse instruction

Start with a prompt that demands a single short sentence. A proposer uses judge feedback and a rubric for accuracy, completeness, clarity, and honesty to draft a replacement. After the two prompts answer sample questions, the judge compares their outputs. Repeated winning mutations can yield an instruction that asks for complete, well-organized answers and acknowledges uncertainty—without a person editing each revision or supplying answer keys. section 2.3.5

SPO is a single-incumbent hill climb: it compares two prompts in a round but retains only one best prompt for the next. Unlike score-based hill climbing, its acceptance gate reads a judge’s preference. A few sample questions can make a round cheap, but a candidate must win those comparisons to advance. If none does, the search stalls; a judge preference is not proof of ground-truth correctness. section 2.3.4 section 3.4.3

How Self-Improving Agents covers it

How deep it goes
Works through mutate, score, and accept in a minimal Python implementation, including feedback and the win-count acceptance rule.
Examples
Uses a terse starting prompt, a rubric-guided winning rewrite, and a revenue-task run that stalls.
What it assumes
Builds on a pairwise judge for output comparison while introducing the agent and offline loop components.
What you can do after
Build it from scratch as a small Python prompt-search loop that proposes variants and accepts judge-preferred winners.
How it connects
Connects pairwise LLM-as-Judge, prompt optimization, and hill climbing through a shared propose–measure–select loop.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.