liveBook · Manning

Adversarial thinking

As explained in Designing AI Agents

An adversarial review is a review approach that examines work from a challenging or opposing perspective.

Adversarial review challenges a consequential claim through an independent attempt to falsify it. Use it when a subtle error is expensive, a credible opposing case exists, and ordinary execution checks cannot settle the whole question. section 8.4

A writer supports the claim, a reviewer searches for consequential counterexamples, and a judge decides what the evidence warrants. An agent may judge a bounded, low-risk decision; a human maintainer is often better suited to decisions affecting production risk or policy. section 8.4

Design the review as a bounded exchange:

  • Define the claim, the writer’s burden of proof, the reviewer’s search boundary, and the judge’s decision rule. section 8.4
  • Give the reviewer a route to independent evidence, such as a clean context, different tools or evidence, a different model, or different acceptance criteria. Opposing role prompts alone do not establish strong independence. section 8.4
  • Set a budget, stop condition, and escalation path. Revisit the changed artifact when new evidence prompts a revision; escalate consequential uncertainty that remains unresolved. section 8.10 section 8.4
Figure 8.3 Adversarial Review runs a bounded exchange around one claim. New evidence can change the artifact or verdict; an unresolved consequential claim escalates when the review budget ends.

The exchange links new evidence to artifact or verdict changes and shows when an unresolved claim escalates. section 8.4

A second agent does not automatically make a review adversarial: it must be able to find something outside the writer’s frame. Generator-Critic instead revises an artifact against feedback, while an Approval Gate decides whether a proposed action may proceed. Reviewers can also converge rather than challenge each other, weakening the review. section 8.4.1 section 2.3.2

Security review of a code change

Argus distributes a code diff to security, style, and complexity workers, retaining their findings. It can then challenge a selected security claim independently. A human maintainer judges the evidence and remaining uncertainty; the review does not itself grant permission to merge. section 8.9 section 8.4

Compare the review with a credible single-agent baseline. Measure consequential defects found beyond self-review, verdict changes, unresolved disputes, and cost per useful finding; also inspect tokens, latency, failures, conflicts, and integration effort. section 8.4 section 8.10

How Designing AI Agents covers it

How deep it goes
Specifies roles, evidence independence, decision rules, stopping conditions, escalation, and evaluation measures.
Examples
Uses a code-change security review with specialist findings and a human merge decision.
What you can do after
Evaluate contested claims using independent evidence, an explicit decision rule, and a bounded escalation path.
How it connects
Distinguishes Generator-Critic and Approval Gate from independent review, and identifies convergence as a collaboration failure.

Also explained in

Compare how 2 books explain adversarial thinking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.