A prompt can produce weak answers even when no correct-answer key exists to grade it against. SPO searches for a better prompt by comparing outputs instead: the current prompt and a challenger answer the same questions, and a pairwise LLM-as-Judge chooses which answers it prefers. That preference supplies the signal without handwritten labels. section 2.3.4
Each round has three steps:
- Mutate: ask a model to rewrite the current best prompt, using feedback from the previous round. section 2.3.4
- Score: run both prompts on a small set of questions and tally the pairwise judge’s wins and losses. section 2.3.4
- Accept: keep the challenger only if it wins more questions than it loses; otherwise retain the current best and try another mutation. section 2.3.4
Figure 2.9 How SPO works. SPO is the search loop of Figure 2.8 made specific: mutate is the propose step, score is the measure step, and accept is the select step. Each round mutates the current best prompt into a candidate, scores both prompts with the pairwise judge, and accepts the candidate only when it wins more than it loses, so the prompt never regresses. The self-supervised callout is the core idea: the only signal is which output wins the comparison, not a score against a labeled answer key, which is what makes SPO the cheap starting rung in Figure 2.7.

The diagram maps mutate, score, and accept onto a propose–measure–select loop. Notice that the gate depends on a preference, not a score against a labeled answer. section 2.3.4
Improving an overly terse instruction
Start with a prompt that demands a single short sentence. A proposer uses judge feedback and a rubric for accuracy, completeness, clarity, and honesty to draft a replacement. After the two prompts answer sample questions, the judge compares their outputs. Repeated winning mutations can yield an instruction that asks for complete, well-organized answers and acknowledges uncertainty—without a person editing each revision or supplying answer keys. section 2.3.5
SPO is a single-incumbent hill climb: it compares two prompts in a round but retains only one best prompt for the next. Unlike score-based hill climbing, its acceptance gate reads a judge’s preference. A few sample questions can make a round cheap, but a candidate must win those comparisons to advance. If none does, the search stalls; a judge preference is not proof of ground-truth correctness. section 2.3.4 section 3.4.3
