liveBook · Manning

Goal manipulation

As explained in AI Governance

Goal manipulation is the alteration of an agent’s pursued objective through adversarial influence.

Goal manipulation gradually shifts what an agent is trying to accomplish through adversarial influence across interactions. Agentic red teaming tests for this risk over time, not just with a single prompt. section 4.8.8

Test realistic attack paths that develop across sessions:

  • Probe whether earlier interactions can shift policy interpretation or bypass controls that work in isolation. section 4.8.8
  • Use PyRIT to orchestrate multi-turn attack conversations; single-prompt scanners alone do not test agents’ evolving context and goals. section 4.8.8

ClaimAssist

Carefully crafted sessions shifted the agent’s interpretation of policy documentation, while context from earlier interactions bypassed controls that worked in isolation. section 4.8.8

How AI Governance covers it

How deep it goes
Treats goal manipulation as a multi-step attack path and contrasts it with single-prompt vulnerability testing.
Examples
Uses ClaimAssist sessions that shifted policy interpretation and used earlier context to bypass isolated controls.
What you can do after
Diagnose agent vulnerabilities by orchestrating multi-turn attack conversations with PyRIT.
How it connects
Connects goal manipulation to agentic red teaming, prompt injection, memory poisoning, and tool chaining.

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.