A model may solve a task yet return an answer in a format that agent code cannot read. Because a stochastic worker can produce a different result on another attempt, an agent can retry rather than fail on the first malformed output. section 5.0 section 5.1
A Resampler wraps a worker in a bounded trial loop. Each trial generates a candidate, assesses it against a preset criterion, and either returns it or invokes the worker again. The worker receives no feedback about rejected candidates; the basic pattern relies on fresh sampling, not correction. section 5.2
Implementation decisions:
- Choose an assessor for the constraint: a regular expression or parser for format, an assessment errand for quality, or a guardrail for safety. section 5.2.2
- Set a maximum trial count. Repeated invocations raise cost; on exhaustion, return a suitable placeholder or raise an error. section 5.2
- Do not use stochastic retries as a security guarantee: rejecting some leaking outputs does not remove the worker’s information leak. section 5.2.2
Recovering a final answer
Wrap a reasoning worker with an assessor that looks for `<answer>…</answer>`. Return a matching answer, or retry when the worker fails to conclude within its output budget. If no trial matches, raise a resampling failure. section 5.4.1
Enforcing summary style
For an email summarizer, assess brevity and neutrality using both the correspondence and the candidate summary. Regenerate a rejected summary instead of editing it, since a corrective rewrite could distort the facts. section 5.4.2
Resampling can also search for plausible solution trajectories when an answer is known. If repeated calls yield too little variation, an input-stochastic variant randomly augments each trial’s input; avoid that variant where factual accuracy requires preserving the input. Unlike a Refiner, a standard Resampler never passes assessment feedback to the worker. section 5.1 section 5.3.2 section 5.2.3
Listing 5.1 resampler.py: Generic resampler
from ..utils import E, IE
from typing import Callable
class Resampler(IE[[str], str]):
def __init__(
self,
worker: IE[[str], str],
assessor: E[[str], str],
branching: Callable[[str], bool],
n_max_trials: int,
):
self.worker = worker
self.assessor = assessor
self.branching = branching
self.n_max_trials = n_max_trials
async def __call__(self, input: str) -> str:
n_trials: int = 0
while n_trials < self.n_max_trials:
result = await self.worker(input)
assessment: str = await self.assessor(result)
criterion_satisfied: bool = self.branching(
assessment
)
if criterion_satisfied:
return result
n_trials += 1
raise ResamplingFailureError(
f"Failed to produce a valid result in {n_trials} trials"
)
class ResamplingFailureError(Exception):
pass
A generic asynchronous implementation shows the worker call, assessor, branching decision, trial limit, and failure exception in one loop. section 5.2
