liveBook · Manning

Prompt injection

As explained in AI Governance

An attack that uses instructions in content supplied to a model to redirect its behavior contrary to intended instructions.

An assistant reading an email may encounter hidden instructions that redirect it from its assigned task. This is prompt injection: attacker-controlled content causes a model to follow the attacker’s instructions instead of its intended instructions. The underlying problem is that a model cannot reliably distinguish instructions it should obey from material it is only supposed to read. section 4.0 section 8.2

Instructions can arrive through emails, documents, webpages, shared files, tool descriptions, or text embedded in images. A jailbreak is a specialized variant that targets model safety guardrails. Injection can affect an answer alone, but access to data and tools raises the stakes: a misled agent may expose records, select an inappropriate tool, or perform an unintended action. section 4.5 section 8.2

A zero-click email attack

In the disclosed EchoLeak attack, hidden instructions in a malicious email could induce Microsoft 365 Copilot to exfiltrate enterprise data without user interaction. The attack illustrates how ordinary content can become an instruction channel when an assistant has access to connected information. section 4.0

Figure 4.4 Image prompt injection causing a faulty decision in ChatGPT-5.

See an example of image-borne instructions influencing a document-validity decision. section 4.5

Treat prompt injection as a system security problem, not just a prompt-filtering problem. Traditional access controls and input validation remain useful, but blocking known phrases cannot reliably determine whether an instruction is legitimate. Restrict what an agent can read, limit its tools and permissions, require approval for hard-to-reverse actions, and record its activity. Continuously red-team and update guardrails; surrounding controls should limit the consequences when a bypass occurs. section 4.0 section 4.5 section 8.2

How AI Governance covers it

How deep it goes
Explains attack surfaces and failure modes, then outlines layered controls while acknowledging that reliable detection remains unresolved.
Examples
Uses the EchoLeak email attack and an image containing manipulative text that leads to a faulty document-validation decision.
What you can do after
Evaluate exposure by identifying what a misled agent can read or do, and apply permission limits, approvals, logging, and red-teaming.
How it connects
Connects prompt injection with jailbreaks, guardrails, access control, red-teaming, and autonomous agents.

Also explained in

Compare how 16 books explain prompt injection →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.