liveBook · Manning

Red-teaming

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Red-team testing is adversarial testing in which evaluators deliberately seek ways to make a system fail or violate its requirements.

Red-teaming is adversarial evaluation: probe a deployed model deliberately to uncover failure modes and unwritten expectations before users do. Findings become additions to the behavioral specification, making the contract more testable, versioned, and auditable. section 8.5.5 section 7.6.4

A practical red-team loop is:

  • Probe the deployed model with adversarial prompts and varied attempts to expose behavior the current specification does not constrain. section 8.5.5 section 4.4.5
  • Record the failure modes and identify the expectation or requirement that was missing from the specification. section 8.5.5
  • Turn useful findings into new specification clauses, then retrain or realign the model against the revised contract. section 8.5.5

Safety evaluation

An adversarial prompt may fool both the evaluated model and an LLM judge. Treat red-teaming as one layer of defense, alongside rule-based filters and human review of flagged content. section 4.4.5

Figure 8.6 Where each alignment technique acts in the model lifecycle. Model spec midtraining acts between pretraining and post-training, on documents about the spec. RLHF, Constitutional AI / RLAIF, and deliberative alignment are done during post-training time, all split into the SFT and RL phases. Deliberative alignment enables the model to reason over principles while generating.

Use the lifecycle view to place red-teaming's findings upstream of specification updates and downstream alignment techniques. section 8.5.5

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains a probe-record-update loop but stops short of prescribing a detailed red-team test suite.
Examples
Uses adversarial prompts, deployed-model probes, uncovered failure modes, and new specification clauses as concrete artifacts.
What it assumes
Assumes a behavioral specification or quality criteria that can be inspected and revised.
What you can do after
Diagnose uncovered model failures and convert them into specification changes and subsequent alignment work.
How it connects
Connects red-teaming to Constitutional AI, RLHF, RLAIF, deliberative alignment, LLM judges, rule-based filters, and human review.

Also explained in

Compare how 8 books explain red-teaming →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.