liveBook · Manning

Agentic red teaming

As explained in AI Governance

Agentic red teaming is adversarial evaluation that tests whether an agent can be manipulated across multiple interactions and over time.

Agentic red teaming tests whether an attacker can manipulate an agent across interactions and over time. Unlike single-prompt vulnerability scans, it probes how accumulated context and goal pursuit can change behavior. section 4.8.8

Test realistic attack paths such as:

  • Memory poisoning that plants information to activate later. section 4.8.8
  • Tool chaining that combines legitimate tools in harmful sequences. section 4.8.8
  • Goal manipulation that gradually shifts what the agent is trying to accomplish. section 4.8.8

ClaimAssist

PyRIT multi-turn testing uncovered policy interpretation shifts and earlier-session context that bypassed controls—issues missed by single-turn testing. section 4.8.8

How AI Governance covers it

How deep it goes
Explains multi-turn attack orchestration and three escalation patterns, contrasting them with single-prompt vulnerability scans.
Examples
Uses PyRIT testing of ClaimAssist to demonstrate policy shifts and controls bypassed through earlier-session context.
What you can do after
Evaluate agent behavior across sessions by orchestrating multi-turn attack conversations with PyRIT.
How it connects
Connects prompt injection testing with memory poisoning, tool chaining, and goal manipulation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.