Red-teaming

As explained in Machine Learning Platform Engineering

Adversarial testing is the deliberate evaluation of a system with challenging, manipulative, or harmful inputs to expose weaknesses.

Red-teaming deliberately probes models and applications with challenging, manipulative, or harmful inputs to expose errors and vulnerabilities. It strengthens evaluation and becomes especially important for public-facing RAG and LLM systems. section 2.1.7 section 13.0

Operational red-teaming connects testing to the wider ML life cycle:

• Test model sensitivity, robustness, and edge cases alongside ordinary evaluation. section 2.1.4
• Analyze failures and systemic weaknesses, then feed findings back into data collection and improvement. section 2.1.4
• Combine adversarial testing with input sanitation, prompt safety, hallucination testing, safety guardrails, and vulnerability assessment. section 2.1.7 section 13.0
• Automate evaluation pipelines and centrally track results so repeated tests remain visible and comparable. section 2.1.4

Figure 12.1 The mental map where we’re focusing on building our RAG application (12)

Situate red-teaming within the broader RAG architecture, where LLM-specific testing extends existing infrastructure rather than replacing it. section 12.0

How Machine Learning Platform Engineering covers it

How deep it goes

Connects robustness, edge cases, prompt injection, and vulnerability assessment, but supplies no detailed testing procedure.

Examples

Uses misclassified samples, edge-case simulations, harmful-content risks, and prompt-injection attacks as testing concerns.

What it assumes

Builds on model evaluation and production observability while extending traditional MLOps with LLM-specific safety testing.

What you can do after

Evaluate model robustness and LLM vulnerabilities within automated, tracked production-testing workflows.

How it connects

Links model evaluation, monitoring, input sanitation, prompt safety, hallucination testing, safety guardrails, and governance.

liveBookmanning.com

Read Machine Learning Platform Engineering on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionA red-teaming protocol is a defined procedure for adversarially testing a system to discover vulnerabilities, unsafe behavior, or policy failures.
• 

  liveBookmanning.com
  Generative AI in ActionA red team is an authorized group that deliberately probes a system for vulnerabilities, failure modes, or harmful behavior.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA red team is an authorized person or group that tests a system by attempting adversarial actions against it.
• 

  liveBookmanning.com
  AI Agents in Action, Second EditionRed team testing is adversarial testing that probes a system with inputs or patterns intended to expose failures, vulnerabilities, or unsafe behavior.
• 

  liveBookmanning.com
  Building Reliable AI SystemsThe practice of deliberately probing a system for weaknesses, unsafe behavior, or failure modes.

Compare how 6 books explain red-teaming →

Related concepts

• adversarial inputuses · An input crafted, selected, or modified to exploit a system’s weaknesses or induce incorrect, unsafe, or otherwise unintended behavior.
• threat huntingcontrasts with · Threat hunting is the proactive investigation of systems and networks to discover malicious activity that automated detection may have missed.
• security controluses · A security control is a safeguard, policy, or restriction designed to reduce security risks or enforce security objectives.
• hypothesisused by · A hypothesis is a provisional proposition, explanation, or prediction that can be evaluated against evidence.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Platform Engineering. The section links point to the parts of the book it draws on.