liveBook · Manning

Resampler

As explained in Agent Design Patterns

An agent design pattern that repeats stochastic model attempts and assesses their outputs to find an acceptable result.

A model may solve a task yet return an answer in a format that agent code cannot read. Because a stochastic worker can produce a different result on another attempt, an agent can retry rather than fail on the first malformed output. section 5.0 section 5.1

A Resampler wraps a worker in a bounded trial loop. Each trial generates a candidate, assesses it against a preset criterion, and either returns it or invokes the worker again. The worker receives no feedback about rejected candidates; the basic pattern relies on fresh sampling, not correction. section 5.2

Implementation decisions:

  • Choose an assessor for the constraint: a regular expression or parser for format, an assessment errand for quality, or a guardrail for safety. section 5.2.2
  • Set a maximum trial count. Repeated invocations raise cost; on exhaustion, return a suitable placeholder or raise an error. section 5.2
  • Do not use stochastic retries as a security guarantee: rejecting some leaking outputs does not remove the worker’s information leak. section 5.2.2

Recovering a final answer

Wrap a reasoning worker with an assessor that looks for `<answer>…</answer>`. Return a matching answer, or retry when the worker fails to conclude within its output budget. If no trial matches, raise a resampling failure. section 5.4.1

Enforcing summary style

For an email summarizer, assess brevity and neutrality using both the correspondence and the candidate summary. Regenerate a rejected summary instead of editing it, since a corrective rewrite could distort the facts. section 5.4.2

Resampling can also search for plausible solution trajectories when an answer is known. If repeated calls yield too little variation, an input-stochastic variant randomly augments each trial’s input; avoid that variant where factual accuracy requires preserving the input. Unlike a Refiner, a standard Resampler never passes assessment feedback to the worker. section 5.1 section 5.3.2 section 5.2.3

Listing 5.1 resampler.py: Generic resampler
from ..utils import E, IE
from typing import Callable
 
 
class Resampler(IE[[str], str]):
  def __init__(
    self,
    worker: IE[[str], str],
    assessor: E[[str], str],
    branching: Callable[[str], bool],
    n_max_trials: int,
  ):
    self.worker = worker
    self.assessor = assessor
    self.branching = branching
    self.n_max_trials = n_max_trials
 
  async def __call__(self, input: str) -> str:
    n_trials: int = 0
    while n_trials < self.n_max_trials:
      result = await self.worker(input)
      assessment: str = await self.assessor(result)
      criterion_satisfied: bool = self.branching(
        assessment
      )
      if criterion_satisfied:
        return result
      n_trials += 1
    raise ResamplingFailureError(
      f"Failed to produce a valid result in {n_trials} trials"
    )
 
 
class ResamplingFailureError(Exception):
  pass

A generic asynchronous implementation shows the worker call, assessor, branching decision, trial limit, and failure exception in one loop. section 5.2

How Agent Design Patterns covers it

How deep it goes
Specifies components, bounded trial control flow, failure handling, variants, and Python-like implementations; does not prescribe a universal assessment criterion.
Examples
Uses answer-tag checks for reasoning and brevity-and-neutrality checks for email summaries.
What it assumes
Assumes a stochastic worker and an assessable output criterion; defines the retry procedure from its components.
What you can do after
Build it from scratch as a bounded worker–assessor loop that returns a passing candidate or signals failure.
How it connects
Connects to guardrails, errands, regular expressions, parsers, and Refiners through assessment and retry behavior.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Agent Design Patterns. The section links point to the parts of the book it draws on.