liveBook · Manning

Prompt injection

As explained in Context Engineering

Prompt injection is an attack that attempts to manipulate an AI system by introducing instructions into its input or context.

Prompt injection is an attempt to manipulate an AI system by placing instructions in its input or context. It succeeds when the model treats untrusted content as a directive rather than as data. section 9.1.2

The vulnerability is structural: system directives, user messages, retrieved documents, and tool payloads enter the model as tokens rather than through a reliable privilege boundary. A model also cannot enforce access control over its own context, so eligibility for sensitive data must be decided before the call. section 9.1.2 section 1.3.3

Two forms differ by where the attacker places the instruction:

  • Direct injection supplies adversarial instructions in a user prompt, attempting to override system directives or extract sensitive information. section 6.2.2
  • Indirect injection embeds a payload in external data, such as a retrieved document, web page, email, or tool output, to steer execution when that content enters context. section 6.2.2 section 9.1.2

An instruction inside a retrieved page

An agent retrieves a web page to answer a user's question. If the page contains an instruction to override the agent's directives, the attack relies on the agent interpreting page content as an instruction instead of evidence for the answer. section 9.1.2

Retrieval, tool use, and memory written in earlier sessions all bring outside material into context and widen the exposed surface. Delimiters and instructions to treat enclosed content as data can reduce misinterpretation, but they are themselves model-readable tokens, not enforced security boundaries. section 1.3.3 section 9.1.2

Use defense in depth across context assembly and execution:

  • Label untrusted spans and separate authority; pair prompt-level defenses with input sanitization and output validation. section 6.2.2 section 9.5
  • Restrict eligible records, memories, and tool outputs; use least-privilege tool execution to limit unauthorized actions and data leakage. section 9.5
Table 9.1 Safety and security controls mapped to context engineering stack components
Stack component Main risks Typical controls

User prompt and instructions

Prompt injection, policy bypass, and prompt drift

Input sanitization, approved templates, and versioned rollback

External knowledge and retrieval

Data poisoning, stale content, and unauthorized sources

Source allowlists, document permissions, and freshness monitoring

Memory and state

Memory poisoning, unsafe carryover, and cross-user leakage

Retention rules, memory eligibility policies, and session isolation

Tools

Unsafe actions, excessive authority, and parameter tampering

Tool allowlists, least-privilege permissions, and approval thresholds

Context management and orchestration

Unsafe routing, context pollution, and failure propagation

Routing policies, context isolation, and fallback paths

Evaluation, observability, and operations

Harmful output, unsupported claims, and unsafe rollout

Output validation, citation checks, and release gates with rollback triggers

Compare risks and controls across prompts, retrieval, memory, tools, and orchestration. section 9.1.2

How Context Engineering covers it

How deep it goes
Explains direct and indirect injection, the shared-token vulnerability, and layered controls; cautions that prompt instructions cannot enforce security boundaries.
Examples
Uses malicious user instructions and payloads embedded in web pages, emails, retrieved documents, or tool outputs.
What you can do after
Configure layered defenses using untrusted-span labeling, authority separation, least-privilege tool execution, and guardrails.
How it connects
Connects prompt injection to retrieval, tool use, memory, access control, and guardrails.

Also explained in

Compare how 16 books explain prompt injection →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.