A generator-critic loop separates production from review. A generator creates an artifact, a critic checks a specific version against criteria and evidence, and the generator revises it or delivers it when the review is satisfied. The canonical path is generate → critique → revise; further reviews are bounded by a stop policy. section 7.2
The key design question is what the critic can see that the generator did not. A different model or role alone does not guarantee independence: a new review frame, domain constraints, tests, source documents, or human judgment can expose assumptions that the original generation missed. Each finding should stay within the scope of its evidence. section 7.1.1 section 7.2.1
Reviewing a SQL query
A query may assume a join key is unique. Paraphrasing the query can preserve that assumption; checking schema constraints, sampled records, execution results, or join cardinality can reveal it. A parser checks syntax, while expected-row tests check a known behavior; a domain reviewer may still need to decide whether the result answers the business question. section 7.2.1
Build the review path around an explicit contract:
- Specify the artifact version, review criteria, evidence sources, permitted verdicts, and maximum number of reviews. section 7.2 section 7.2.3
- Collect evidence outside the critic and link findings to durable evidence identifiers. Retain the exact version inspected with each critique. section 7.2.3
- Preserve candidate versions and select the best using domain evidence, such as regression results or a calibrated rubric. Another model’s preference alone is insufficient. section 7.2.3
- Protect tests, rubrics, and thresholds from changes by the producing process; legitimate changes need a separate review path. section 7.2.4
Listing 7.4 A bounded Generator-Critic chain
@dataclass
class ReviewResult:
artifact: str
status: str
history: list[ReviewStep] = field(default_factory=list)
def generator_critic(
task,
contract,
generate,
revise,
critic,
checks,
select_best,
):
artifact = generate(task)
versions = [artifact]
history = []
for _ in range(contract.max_reviews): #A
step = review_once(
task,
artifact,
contract,
critic,
checks,
)
history.append(step)
if step.critique.verdict == "accept":
return ReviewResult(artifact, "accepted", history)
if step.critique.verdict == "stop":
best = select_best(task, versions, checks)
return ReviewResult(best, "unresolved", history)
artifact = revise(task, artifact, step.critique)
versions.append(artifact) #B
best = select_best(task, versions, checks) #C
return ReviewResult(best, "budget_exhausted", history)
A bounded implementation records review history, handles accept and stop verdicts, and preserves versions for evidence-based selection. section 7.2.3
Reviews cost time and can rubber-stamp shared assumptions. Revisions can also oscillate, fixing one issue while reintroducing another. Detect repeated issue and artifact signatures, return the best known version, and hand unresolved trade-offs to a wider decision process. Calibrate model-based scores against known examples and record criterion-level findings. Measure confirmed-issue precision, quality change, oscillation, latency, and cost. section 7.2.4 section 7.2 section 2.3.2
Generator-critic improves the current artifact; it does not by itself authorize deployment or recover a failed execution. Adversarial review tests a contested claim through an independent search for defects, while an approval gate decides whether a proposed action may execute. section 8.4.1 section 7.6
