Prompt injection
Prompt injection is an attack that introduces instructions through input or context to manipulate an AI system’s behavior or override its intended instructions.
Machine Learning Platform Engineering
A jailbreaking attempt is an effort to bypass a language model's safety constraints or guardrails through specially crafted inputs.
- How it is used
- Identifies attempts to redirect an LLM application through user input or external data, especially where RAG combines queries and retrieved documents.
- How deep it goes
- Explains RAG attack surfaces and demonstrates adversarial testing with Promptfoo; does not establish that input validation alone prevents every attack.
- Examples
- Uses pirate-role and competitor-comparison queries, plus a Promptfoo configuration and Python provider for testing a documentation assistant.
- What you can do after
- Diagnose/debug with Promptfoo by probing an LLM application for instruction overrides and scope violations before deployment.
Read how Machine Learning Platform Engineering explains prompt injection →
Introduction to Generative AI, Second Edition
A jailbreak prompt is an input crafted to make a generative model bypass restrictions or safety controls.
- How it is used
- Identifies attempts to redirect model behavior through crafted inputs or instructions embedded in data an AI system reads.
- How deep it goes
- Distinguishes direct from indirect attacks and surveys prompting strategies and agent risks without detailing an implementation.
- Examples
- Uses hidden webpage instructions, covert image content, and a table of red-teaming strategies to illustrate attack routes.
- What you can do after
- Evaluate/understand it by recognizing possible instruction sources and why agent tool access can make redirection consequential.
Read how Introduction to Generative AI, Second Edition explains prompt injection →
LLMs in Production
Prompt injection is an attack in which crafted input attempts to manipulate a language model into ignoring intended instructions or revealing unintended information.
- How it is used
- Frames instruction-bypass attempts as a security risk for LLM services, especially agents with access to tools and sensitive data.
- How deep it goes
- Defines the attack, illustrates consequences, and gives layered defensive practices; does not provide an implementation or a complete prevention method.
- Examples
- Uses a malicious incoming email, reported secret-extraction attempts, and a plausible but inaccurate cola recipe.
- What you can do after
- Diagnose/debug with it by recognizing instruction-bypass risks and applying validation, filtering, monitoring, and model updates.
Natural Language Processing in Action, Second Edition
A jailbreak is an input or interaction strategy that induces an AI system to bypass its intended behavioral restrictions.
- How it is used
- Frames adversarial prompts as a safety and reliability threat to guarded chatbots.
- How deep it goes
- Explains a suffix-based bypass and a prompt-side guardrail configuration, without establishing a complete defense.
- Examples
- Uses a prohibited request altered by a suffix and a Guardrails AI UnusualPrompt call.
- What you can do after
- Diagnose/debug with it by testing chatbot guardrails against attempted bypasses and examining failures.
Read how Natural Language Processing in Action, Second Edition explains prompt injection →
Generative AI in Action
Jailbreaking is an attack technique that manipulates an AI model into bypassing intended instructions, safeguards, or behavioral constraints.
- How it is used
- Identifies an LLM security threat in chatbots that read external content or use plugins.
- How deep it goes
- Defines direct and indirect attacks, explains instruction–data confusion, and outlines mitigation and evaluation without model-level internals.
- Examples
- Uses poisoned webpages, document summaries, and a bank-linked plugin to illustrate manipulated answers and unauthorized actions.
- What you can do after
- Evaluate/understand it by recognizing attack entry points, limiting access, and testing for harmful outputs and regressions.
Read how Generative AI in Action explains prompt injection →
AI Agents in Action, Second Edition
Jailbreaking is the use of crafted inputs or interactions to bypass an AI system's intended behavioral restrictions.
- How it is used
- Frames prompt injection as a primary agent-runtime threat requiring threat modeling and layered defenses.
- How deep it goes
- Explains untrusted content, instruction hierarchy, tool constraints, validation, and operational safeguards without covering model internals.
- Examples
- Uses a secure agent prompt with ranked instructions, untrusted-content rules, secret protection, and an explicit tool allowlist.
- What you can do after
- Operate/configure agent safeguards by classifying inputs as untrusted, restricting tools, validating results, and requiring approval for risky actions.
Read how AI Agents in Action, Second Edition explains prompt injection →
Prompt Engineering in Practice
Prompt injection is an attack technique that inserts adversarial instructions into model input to alter or override intended behavior.
- How it is used
- Frames prompt injection as a security risk wherever a prompt accepts external user or retrieved text.
- How deep it goes
- Explains the structural failure through a worked support-chatbot example and identifies limits of prompt-level defenses.
- Examples
- Contrasts a customer-data extraction request with a revised order-support prompt that restricts scope and specifies a fallback.
- What you can do after
- Diagnose/debug with it by locating untrusted directives and revising prompt boundaries, trust framing, permitted scope, and fallback behavior.
Read how Prompt Engineering in Practice explains prompt injection →
The Art of AI Product Development
An attack that manipulates a generative model by inserting instructions intended to override or redirect its intended behavior.
- How it is used
- Secures externally exposed LLM applications whose inputs can damage downstream systems and data.
- How deep it goes
- Distinguishes direct and indirect attacks and traces harmful recommendations to malicious instructions, but does not enumerate defenses.
- Examples
- Uses retail overstocking, halted operations, a mental-health chatbot, web pages, and injected user commands as examples.
- What you can do after
- Evaluate injection risk by distinguishing direct from indirect attacks in LLM-integrated systems.
Read how The Art of AI Product Development explains prompt injection →
Effective Conversational AI
Prompt injection is an attempt to manipulate a language model's instructions or behavior through specially crafted input.
- How it is used
- Treats resistance to prompt-injection attempts as a safety guardrail and a measure of conversational understanding.
- How deep it goes
- Shows a jailbreak example, mentions input screening and length limits, and includes resilience as an evaluation criterion.
- Examples
- Contrasts a refused bomb-making request with a grandmother-story framing and assesses answers for resistance to injection.
- What you can do after
- Evaluate resistance by reviewing representative logs and tracking answer quality against an annotated baseline.
Read how Effective Conversational AI explains prompt injection →
Related concepts
- jailbreakingcontrasts with · Jailbreaking is an adversarial input or interaction strategy intended to make an AI system bypass its instructions, safeguards, or behavioral restrictions.
- guardrailaddressed by · A guardrail is a constraint, safeguard, or control that limits actions or behavior to reduce risk and keep a system or process within acceptable bounds.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.