Chapter 5: Resampler: Reliability through repeated sampling
This introduction frames stochastic model output as a reliability risk at code–AI interfaces and introduces Resampler as a retry-and-check loop that can both constrain and exploit variation.
After this chapter you will be able to
- Stochastic interface failures. Explain why probabilistic language-model outputs can cause critical failures when agent orchestration code expects a specified response format. Illustrate this with MathSolver failing to produce a final answer marked by the expected XML or LaTeX delimiter, leaving a regex with no match.
- Resampler retry loop. Describe the Resampler pattern as a loop containing an intelligent element and a verifier for a crucial criterion, which returns to the intelligent element for another sample when that criterion is not met. State that it is appropriate when a failure is plausibly a random blip rather than a deeper unsolvable problem.
- Constructive stochastic exploration. Explain that Resampler can use randomness constructively as well as filter unwanted variation: by requiring a known correct final answer, MathSolver can repeatedly seek alternative solution paths that reach that answer. State the limits that novel solutions are not guaranteed and may merely be differently worded versions of the same approach.
- Stochastic output unreliability. Explain why stochastic AI systems can produce non-reproducible outputs from the same inputs and therefore cannot inherently guarantee consistent behavior, quality, or other desired attributes.
- Criteria-based repeated sampling. Describe the Resampler pattern as formulating criteria for outputs of crucial IEs and repeatedly sampling those IEs until the criteria are satisfied before downstream processing. Explain that it reduces randomness only along the properties monitored by its criteria, allowing desirable creativity while enforcing requirements such as a fixed output format.
- Resampler trial loop. Describe the Resampler pattern as a loop containing a worker IE, an assessor, and a branching procedure. Explain that each trial samples the worker for a candidate, returns the first candidate that satisfies the criterion, and otherwise loops back for another sample.
- Stochastic retry requirements. Explain that a Resampler depends on the worker IE producing potentially different outputs from the same input on repeated invocations. Describe how its iterativity and lack of material feedback distinguish it from a simple errand or guardrail.
- Trial limits and failures. Explain why a Resampler normally has an upper trial limit: cost grows linearly with trials and unlimited retries may never succeed. Describe that reaching the limit can yield a graceful placeholder or an escalating software error, with the generic code example raising an error for higher-level handling.
- Resampler configuration tradeoffs. Distinguish the tradeoffs of stochasticity-constraining and stochasticity-using Resampler configurations. Explain that the former enforces critical constraints and improves reliability but can raise cost through repeated IE invocations after rejection, while the latter expands agent capabilities and may improve interpretability or output-data quality.
- Common Resampler enforcement uses. Describe the three common uses of a Resampler—enforcing output formatting, quality, and safety—and identify the appropriate criterion satisfaction assessor for each.
- Security limitation of resampling. Explain why a Resampler is poorly suited to enforce security: placing a security guardrail in the loop admits that the worker can leak information and relies on stochastic chance for a secure output.
- Resampler and Refiner distinction. Distinguish Resampler from the Refiner-meta pattern: both repeatedly seek outputs meeting criteria, but Resampler relies only on worker IE stochasticity while Refiner sends criterion-assessor feedback to the worker IE.
- Small-scale resampling rationale. Explain why Refiners are ultimately more powerful, yet Resamplers are generally safer and can be more efficient and lower overall agent cost when ensuring reliability at a small scale.
- Resampling Guardrail pattern. Explain how a Resampling Guardrail combines a stochastic worker information element (IE) with a Guardrail to keep generating output until an output passes assessment or the trial budget is exhausted.
- Security guardrail exclusion. Justify why security guardrails should not generally be combined with resampling: one failure can show that the input is polluted, such as with confidential information, so it should be discarded rather than reused for another worker attempt.
- Need for input stochasticity. Explain why a Resampler may need random input augmentation when an inference engine produces too-similar outputs for repeated samples of the same input, especially in creative tasks shaped by post-training or prompt engineering.
- Per-trial input augmentation. Describe the input-stochastic Resampler mechanism: before every worker-IE trial, randomly augment the input so that each trial receives an additional source of stochasticity.
- Regex final-answer resampling. Describe the reasoning reliability resampler: it wraps the reasoner IE, checks each output for a valid final-answer marker using a regular expression such as <answer>(.*)</answer>, and resamples when that criterion is not met.
- Regeneration over summary rewriting. Explain why a summary that is too long or insufficiently neutral should be regenerated rather than shortened or rewritten after the fact. Relate post-hoc editing to the risk of introducing factual bias through poorly contextualized changes.
- Joint style acceptance criterion. Describe how the style enforcement resampler uses brevity and neutrality together as the criterion for accepting a generated email summary. State that a summary failing the criterion is regenerated, while only a summary judged conforming exits the component without corrections.
- Input-stochastic reaction discovery. Describe how an input-stochastic resampler finds reaction chains to a target compound by sampling a subset of available reactants before asking a language-model worker to produce a chain.
- Target-product stopping condition. Explain the worker prompt and stopping condition: the worker is asked for a reaction chain in a specified reaction representation format, and resampling stops when its output contains a reaction that lists the target compound as a product.
What it builds on
- Runtime and orchestration (chapter 2) · The distinction between agent runtime or orchestration code and the intelligent elements it invokes.
- How the Guardrail pattern works (chapter 4) · The idea of checking an intelligent element's output against a gating criterion.
- Agent Flow Diagrams (chapter 2) · Agent Flow Diagrams as a way to represent agent components and branching flows.
- How the Errand pattern works (chapter 3) · The idea of an errand as a work-performing unit that can be composed into a larger workflow.
- The pain point: Unsafe or inappropriate output (chapter 4) · The problem of unsafe or inappropriate output.
- Reflection errands (chapter 3) · Reflection errands as a related way to iteratively improve or analyze work.
- Sequential and parallel guardrails (chapter 4) · The distinction between sequential and parallel guardrails.
- Buffered and stream guardrails (chapter 4) · The distinction between buffered and stream guardrails.
Section by section
Introduction
This introduction frames stochastic model output as a reliability risk at code–AI interfaces and introduces Resampler as a retry-and-check loop that can both constrain and exploit variation.
- Stochastic interface failures: Explain why probabilistic language-model outputs can cause critical failures when agent orchestration code expects a specified response format. Illustrate this with MathSolver failing to produce a final answer marked by the expected XML or LaTeX delimiter, leaving a regex with no match.
Why it matters: Unlike a human, orchestration code cannot usually infer the intended answer from an unexpectedly formatted natural-language response. - Resampler retry loop: Describe the Resampler pattern as a loop containing an intelligent element and a verifier for a crucial criterion, which returns to the intelligent element for another sample when that criterion is not met. State that it is appropriate when a failure is plausibly a random blip rather than a deeper unsolvable problem.
Why it matters: The pattern gives an agent a simple way to turn occasional stochastic failures into additional attempts instead of immediate software failure. - Constructive stochastic exploration: Explain that Resampler can use randomness constructively as well as filter unwanted variation: by requiring a known correct final answer, MathSolver can repeatedly seek alternative solution paths that reach that answer. State the limits that novel solutions are not guaranteed and may merely be differently worded versions of the same approach.
Why it matters: Repeated sampling can support creative exploration while a verification constraint preserves a required outcome.
Key terms: probabilistic sampling, code–AI interfaces, orchestration code, MathSolver, regular expression, <answer>…</answer>, Resampler pattern, intelligent element, IE, crucial criterion
5.1 The problem: Random, unreliable, constraint-violating outputs
This section explains how stochastic AI outputs create reliability problems and how repeated sampling against criteria can enforce desired properties or search for solution trajectories.
- Stochastic output unreliability: Explain why stochastic AI systems can produce non-reproducible outputs from the same inputs and therefore cannot inherently guarantee consistent behavior, quality, or other desired attributes.
Why it matters: Recognizing randomness as the source of inconsistent results explains why an agent may need an explicit reliability mechanism. - Criteria-based repeated sampling: Describe the Resampler pattern as formulating criteria for outputs of crucial IEs and repeatedly sampling those IEs until the criteria are satisfied before downstream processing. Explain that it reduces randomness only along the properties monitored by its criteria, allowing desirable creativity while enforcing requirements such as a fixed output format.
Why it matters: A Resampler turns variable generation into output that is dependable on the specific constraints an agent needs. - Outcome-guided trajectory search: Explain how a known task outcome or answer can become a Resampler criterion around a worker IE, causing repeated stochastic sampling to search for a plausible trajectory from the input or problem to that known result. Describe its creative uses for reverse-engineering solutions, improving interpretability, or generating high-quality synthetic data from input-output examples.
Why it matters: This use treats model stochasticity as an exploratory asset rather than only a source of errors.
Key terms: stochasticity, probabilistic sampling, random sampling, deterministic, seed, pseudorandomness, Resampler pattern, criteria, resampling criteria, IEs
5.2 How the Resampler pattern works
This section explains how the Resampler pattern repeatedly samples a stochastic worker until an assessor accepts a candidate or a trial limit triggers failure.
- Resampler trial loop: Describe the Resampler pattern as a loop containing a worker IE, an assessor, and a branching procedure. Explain that each trial samples the worker for a candidate, returns the first candidate that satisfies the criterion, and otherwise loops back for another sample.
Why it matters: This loop is the defining mechanism that turns repeated stochastic outputs into a reliability technique. - Stochastic retry requirements: Explain that a Resampler depends on the worker IE producing potentially different outputs from the same input on repeated invocations. Describe how its iterativity and lack of material feedback distinguish it from a simple errand or guardrail.
Why it matters: Retrying only has value when another sample can differ, and withholding failure details preserves the pattern's inexpensive, non-learning character. - Trial limits and failures: Explain why a Resampler normally has an upper trial limit: cost grows linearly with trials and unlimited retries may never succeed. Describe that reaching the limit can yield a graceful placeholder or an escalating software error, with the generic code example raising an error for higher-level handling.
Why it matters: A retry mechanism needs bounded cost and a defined outcome when no acceptable candidate is found. - Sequential first-success execution: Describe that Resampler trials run sequentially and stop on the first successful trial. Explain that this makes the time and cost variable, while avoiding work on additional trials once a passing candidate has been found.
Why it matters: Sequential first-success execution explains both the Resampler's variable latency and why it does not spend on unnecessary extra samples. - Resampler placement and roles: Describe that a Resampler can be introduced at any agent level, commonly with a simple errand as its worker and treated as an errand that must succeed. Distinguish constraining stochasticity from using stochasticity to discover input-to-output trajectories, including their different likely output portions.
Why it matters: The same retry structure can either filter unwanted variation or exploit variation to find useful paths.
Key terms: Resampler, worker IE, assessor, criterion satisfaction, trial, candidate output, reliance on stochasticity, iterativity, absence of material feedback, criterion assessment
5.2.1 Tradeoffs
This section contrasts the reliability benefits and costs of constraint-enforcing Resamplers with the capability benefits of stochasticity-using Resamplers.
- Resampler configuration tradeoffs: Distinguish the tradeoffs of stochasticity-constraining and stochasticity-using Resampler configurations. Explain that the former enforces critical constraints and improves reliability but can raise cost through repeated IE invocations after rejection, while the latter expands agent capabilities and may improve interpretability or output-data quality.
Why it matters: Choosing a Resampler configuration means balancing constraint enforcement and invocation cost against added capabilities and output benefits.
Key terms: stochasticity-constraining configuration, stochasticity-using configuration, IEs, criterion assessor, reliability, interpretability
5.2.2 When and where to use Resampler
This section identifies appropriate Resampler uses, maps each use to an assessor, and warns against stochastic resampling for security or costly solution discovery.
- Common Resampler enforcement uses: Describe the three common uses of a Resampler—enforcing output formatting, quality, and safety—and identify the appropriate criterion satisfaction assessor for each.
Why it matters: Choosing an assessor that matches the constraint makes repeated sampling a practical reliability mechanism. - Security limitation of resampling: Explain why a Resampler is poorly suited to enforce security: placing a security guardrail in the loop admits that the worker can leak information and relies on stochastic chance for a secure output.
Why it matters: Security controls should not depend on repeatedly getting lucky after an information leak is possible. - Reverse solution trajectory search: Describe how Resamplers can support reverse invocation and solution reverse-engineering by searching for solution trajectories between a given problem and answer, and explain why the Refiner pattern may be preferable when stochastic search is costly.
Why it matters: Finding a trajectory can enable later generalization to similar tasks, but an uncontrolled stochastic search can consume excessive resources.
Key terms: Resampler, criterion satisfaction assessor, regular expression, parser, errand, IE, security guardrail, information leak, stochastic chance, reverse invocation
5.2.3 How Resampler works with other patterns
This section compares Resampler with Guardrail and the Refiner-meta pattern, explaining their relationship and why a Resampler can be the safer, cheaper choice for small-scale reliability.
- Resampling Guardrail combination: Describe how Resampler and Guardrail combine into a Resampling Guardrail, and state that this combined pattern is covered in the following section.
Why it matters: This connection shows how repeated sampling can be paired with output gating to improve reliability. - Resampler and Refiner distinction: Distinguish Resampler from the Refiner-meta pattern: both repeatedly seek outputs meeting criteria, but Resampler relies only on worker IE stochasticity while Refiner sends criterion-assessor feedback to the worker IE.
Why it matters: The distinction helps you choose whether retries alone are sufficient or whether an agent needs feedback-driven improvement. - Small-scale resampling rationale: Explain why Refiners are ultimately more powerful, yet Resamplers are generally safer and can be more efficient and lower overall agent cost when ensuring reliability at a small scale.
Why it matters: This tradeoff guides pattern selection when reliability is needed without paying for a more elaborate feedback loop.
Key terms: Resampler, Guardrail, Resampling Guardrail, Refiner-meta pattern, Refiner, criterion assessor, worker IE, stochasticity of the IE, reliability, small scale
5.3.1 Resampling guardrails
This section explains how combining repeated sampling with guardrails can retry stochastic outputs until one complies, while identifying cases such as security screening where retries are unsafe.
- Resampling Guardrail pattern: Explain how a Resampling Guardrail combines a stochastic worker information element (IE) with a Guardrail to keep generating output until an output passes assessment or the trial budget is exhausted.
Why it matters: It turns a guardrail from a pure blocker into a reliability mechanism that can seek a compliant alternative. - Resampling Guardrail configurations: Describe the sequential Resampling Guardrail flow and distinguish it from feasible parallel and streamed resampling guardrails, including why streaming is useful for smartphone personal assistants.
Why it matters: The configuration determines whether the design emphasizes a simple ordered check, concurrent attempts, or fast perceived response. - Security guardrail exclusion: Justify why security guardrails should not generally be combined with resampling: one failure can show that the input is polluted, such as with confidential information, so it should be discarded rather than reused for another worker attempt.
Why it matters: Retrying with contaminated input can propagate sensitive information instead of safely containing it.
Key terms: Resampling Guardrail, Guardrail, Resampler, worker agent, information elements, IEs, sequential guardrail, parallel guardrail, streamed resampling guardrail, AFD
5.3.2 Input-stochastic resamplers
This section explains how input-stochastic resamplers create output variety by randomly augmenting each trial’s input, while warning against their use for factually exact workloads.
- Need for input stochasticity: Explain why a Resampler may need random input augmentation when an inference engine produces too-similar outputs for repeated samples of the same input, especially in creative tasks shaped by post-training or prompt engineering.
Why it matters: Adding variation before each trial can restore useful diversity when repeated sampling alone does not produce it. - Per-trial input augmentation: Describe the input-stochastic Resampler mechanism: before every worker-IE trial, randomly augment the input so that each trial receives an additional source of stochasticity.
Why it matters: Changing the input on every trial can lead the worker to explore different valid outputs rather than repeating one response. - Augmentation methods and accuracy limit: Describe examples of random augmentation across modalities—such as visual transformations, language paraphrases or instruction variations, and subset, ordering, or grouping changes for set-like inputs—and state that inputs for workloads requiring the highest factual accuracy should not be augmented.
Why it matters: The choice of augmentation determines whether added variation is useful or instead damages information needed for an accurate result.
Key terms: input-stochastic resampler, IEs, worker IE, post-training, prompt engineering, random augmentation, resampler trial, input stochasticity, random augmentations, input payload
5.4.1 Regex-powered reasoning reliability resampler
This section shows how to wrap a reasoning step in a Resampler that uses a regular expression to accept only outputs containing a clear final answer.
- Reasoning failure consequences: Explain why complex reasoning can be unreliable and how a failed reasoning step can cause an entire agent to fail when later components rely on its output.
Why it matters: Recognizing these failure modes shows why a reasoning step may need reliability protection before its result is used elsewhere. - Regex final-answer resampling: Describe the reasoning reliability resampler: it wraps the reasoner IE, checks each output for a valid final-answer marker using a regular expression such as <answer>(.*)</answer>, and resamples when that criterion is not met.
Why it matters: This provides an immediate way to prevent unfinished reasoning outputs from being accepted by downstream agent components. - Reasoning length limit: State that the reasoner IE supplied to this resampler must enforce a maximum permissible reasoning length so that a model cannot continue reasoning indefinitely before producing a final answer.
Why it matters: A length limit makes the completion check practical by bounding reasoning that otherwise loops or fails to conclude.
Key terms: reasoning step, reasoner IE, output budget, reliability, Resampler, assessment element, regular expression, valid final answer, <answer>(.*)</answer>, maximum permissible reasoning length
5.4.2 Style enforcement resampler
This subsection applies a Resampler to produce brief, neutral email summaries by rejecting nonconforming generations and regenerating them rather than editing them afterward.
- Regeneration over summary rewriting: Explain why a summary that is too long or insufficiently neutral should be regenerated rather than shortened or rewritten after the fact. Relate post-hoc editing to the risk of introducing factual bias through poorly contextualized changes.
Why it matters: Regenerating preserves the chance to produce a faithful summary without accidentally changing its meaning during style corrections. - Joint style acceptance criterion: Describe how the style enforcement resampler uses brevity and neutrality together as the criterion for accepting a generated email summary. State that a summary failing the criterion is regenerated, while only a summary judged conforming exits the component without corrections.
Why it matters: A joint acceptance test ensures that the released summary satisfies both of the required style constraints without a downstream correction step. - Embedded components and retry handling: Describe the implementation choices in this example: the summarizer and style assessors are built into the target resampler, and each trial is enclosed in a try-except block. Explain that exception handling allows continued attempts to obtain a conforming summary while reporting encountered failures to the user through standard output.
Why it matters: Handling trial failures lets the component remain resilient while still making its problems visible to the user.
Key terms: style enforcement resampler, factual bias, neutral tone, brevity, Resampler, neutrality, style conformity assessment, regenerating the summary, summarizer, style assessors
5.4.3 Chemical reaction chain discovery
This subsection shows how an input-stochastic resampler can discover varied chemical reaction chains by randomly presampling reactants before each model trial.
- Input-stochastic reaction discovery: Describe how an input-stochastic resampler finds reaction chains to a target compound by sampling a subset of available reactants before asking a language-model worker to produce a chain.
Why it matters: Randomly varying the reactants supplied on each trial can produce a greater variety of candidate reaction chains. - Target-product stopping condition: Explain the worker prompt and stopping condition: the worker is asked for a reaction chain in a specified reaction representation format, and resampling stops when its output contains a reaction that lists the target compound as a product.
Why it matters: The target appearing as a product gives the resampler a concrete criterion for deciding when a sampled chain has reached the requested outcome.
Key terms: stochasticity-using resampler, input-stochastic resampler, presampling, reactants, worker IE, reaction chain, target compound, reaction representation format, target compound as one of its products, resampling
5.5 Summary
This summary explains how the Resampler pattern uses repeated independent trials to turn generative-model randomness into more reliable or creatively diverse agent outputs.
Concepts explained in this chapter
- ResamplerAn agent design pattern that repeats stochastic model attempts and assesses their outputs to find an acceptable result.
- randomnessThe property of involving chance or probabilistic variation rather than fully deterministic behavior.
- intelligent elementAn agent component that uses AI to assess information or perform a focused function.
- samplingSampling is the process of generating or selecting an output from a stochastic model or operation.
- model-to-code interfaceA code–AI interface is a boundary where program code exchanges model outputs that must conform to machine-readable expectations.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
