liveBook · Manning

Red-teaming

Red-teaming is an adversarial assessment practice that deliberately simulates attacks or uses challenging inputs to identify a system’s vulnerabilities, failures, or unsafe behavior.

How 8 Manning books explain it

Introduction to Generative AI, Second Edition

A red-teaming protocol is a defined procedure for adversarially testing a system to discover vulnerabilities, unsafe behavior, or policy failures.

How it is used
Complements benchmarks by stress-testing AI systems for vulnerabilities and safety-mitigation failures.
How deep it goes
Outlines input creation, response checking, and vulnerability analysis, with an overview of manual and automated methods.
Examples
Illustrates testing with a sexist song prompt, hypothetical scenarios, unusual formatting, and possible agentic tool failures.
What you can do after
Evaluate and understand how adversarial inputs can expose model vulnerabilities and inform mitigation.

Read how Introduction to Generative AI, Second Edition explains red-teaming →

Generative AI in Action

A red team is an authorized group that deliberately probes a system for vulnerabilities, failure modes, or harmful behavior.

How it is used
Identifies application-specific harms and safety gaps as part of responsible AI risk assessment.
How deep it goes
Outlines planning, testing, recordkeeping, and follow-up, while distinguishing discovery from systematic measurement and mitigation.
Examples
Uses a customer service chatbot’s bias, privacy, and accuracy risks and names tools such as TextAttack and AI Fairness 360.
What you can do after
Evaluate and understand model and application risks, then use findings to guide measurement and mitigation.

Read how Generative AI in Action explains red-teaming →

Natural Language Processing in Action, Second Edition

A red team is an authorized person or group that tests a system by attempting adversarial actions against it.

How it is used
Uses authorized adversarial interaction to find LLM guardrail failures before harmful behavior reaches users.
How deep it goes
Explains roles, attack patterns, tester independence, and iterative defenses, but stops short of a complete testing protocol.
Examples
Uses adversarial prompt suffixes, unusual-request detectors, self-checks, and collected edge cases as testing artifacts.
What you can do after
Diagnose guardrail weaknesses and improve an LLM pipeline by converting adversarial failures into filters, checks, and regression cases.

Read how Natural Language Processing in Action, Second Edition explains red-teaming →

AI Agents in Action, Second Edition

Red team testing is adversarial testing that probes a system with inputs or patterns intended to expose failures, vulnerabilities, or unsafe behavior.

How it is used
Tests user-facing agents for safety breaks by probing terms, phrases, and patterns that may expose unsafe behavior.
How deep it goes
Defines adversarial safety testing and places it in an evaluation loop, but does not detail a full red-team procedure.
Examples
Uses targeted terms, phrases, and patterns as probes, then reruns tests to assess safety and behavioral stability.
What you can do after
Diagnose safety breaks in user-facing agents and feed the findings into iterative evaluation and improvement.

Read how AI Agents in Action, Second Edition explains red-teaming →

Machine Learning Platform Engineering

Adversarial testing is the deliberate evaluation of a system with challenging, manipulative, or harmful inputs to expose weaknesses.

How it is used
Strengthens evaluation and production hardening by exposing model errors and vulnerabilities in public-facing RAG and LLM systems.
How deep it goes
Connects robustness, edge cases, prompt injection, and vulnerability assessment, but supplies no detailed testing procedure.
Examples
Uses misclassified samples, edge-case simulations, harmful-content risks, and prompt-injection attacks as testing concerns.
What you can do after
Evaluate model robustness and LLM vulnerabilities within automated, tracked production-testing workflows.

Read how Machine Learning Platform Engineering explains red-teaming →

LLM Evaluation and Alignment, The Foundational Ideas

Red-team testing is adversarial testing in which evaluators deliberately seek ways to make a system fail or violate its requirements.

How it is used
Finds specification gaps and failure modes before deployment users encounter them.
How deep it goes
Explains a probe-record-update loop but stops short of prescribing a detailed red-team test suite.
Examples
Uses adversarial prompts, deployed-model probes, uncovered failure modes, and new specification clauses as concrete artifacts.
What you can do after
Diagnose uncovered model failures and convert them into specification changes and subsequent alignment work.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains red-teaming →

AI Governance

Adversarial evaluation in which authorized testers simulate attackers to discover weaknesses, failures, or harmful behaviors.

How it is used
Evaluates GenAI systems for adversarial failures that routine QA may miss and supplies evidence for launch readiness.
How deep it goes
Details team composition, adversarial inputs, finding triage, remediation, regression testing, and release timing.
Examples
Uses ClaimAssist cases involving name-based differences, fabricated PDF estimates, and provider-name spelling variations.
What you can do after
Practice adversarial evaluation, document findings, refine controls, and route issues for remediation or accepted risk.

Read how AI Governance explains red-teaming →

Building Reliable AI Systems

The practice of deliberately probing a system for weaknesses, unsafe behavior, or failure modes.

How it is used
Finds AI-system vulnerabilities and unexpected failures before they reach users.
How deep it goes
Outlines adversarial, extreme, and edge-case testing followed by analysis and repeated improvement.
Examples
Tests a ShopBot for false refund policies, invented products, impossible delivery promises, and internal-information disclosure.
What you can do after
Diagnose vulnerabilities before users encounter them and guide repeated system improvements.

Read how Building Reliable AI Systems explains red-teaming →

Also covered in

What to know first

Related concepts

Questions about red-teaming

What is red-teaming?

Red-teaming is an adversarial assessment practice that deliberately simulates attacks or uses challenging inputs to identify a system’s vulnerabilities, failures, or unsafe behavior.

Which Manning books explain red-teaming?

Introduction to Generative AI, Second Edition, Generative AI in Action, Natural Language Processing in Action, Second Edition, AI Agents in Action, Second Edition, Machine Learning Platform Engineering, LLM Evaluation and Alignment, The Foundational Ideas, AI Governance and Building Reliable AI Systems.

What should I know before learning red-teaming?

Manning books that teach red-teaming usually introduce language model, bias, generative modeling and guardrail first.

What is the difference between red-teaming and blue team?

Red-teaming is an adversarial assessment practice that deliberately simulates attacks or uses challenging inputs to identify a system’s vulnerabilities, failures, or unsafe behavior. A blue team is a cybersecurity group responsible for defending systems by monitoring, detecting, preventing, responding to, and mitigating attacks.

What is red-teaming part of?

Red-teaming is part of offensive security. Offensive security is the practice of proactively simulating or conducting attacks to discover and exploit weaknesses in systems and networks.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.