Prompt injection is an attack in which crafted input steers an LLM away from its intended instructions. It matters especially when a chatbot reads external content or can use plugins: an attacker may influence not just an answer, but an action. Jailbreaking is a related form of attack that tries to bypass safety features or elicit confidential or unsuitable responses. section 6.6 section 13.2.1
The entry point determines how the attack reaches the model:
- Direct injection: a user submits a prompt intended to override system instructions or circumvent safeguards. section 6.6
- Indirect injection: an attacker places instructions in a webpage, document, or other external content that the chatbot later reads. This is a particular concern for retrieval-augmented generation (RAG). section 13.2.1
A poisoned webpage
A user asks a chatbot to summarize a webpage. The page contains an attacker’s instruction rather than ordinary content. If the chatbot follows that instruction, the summary workflow may expose sensitive information. A connected bank plugin raises the stakes: rogue instructions on a visited site could lead to unauthorized purchases. section 6.6
The underlying problem is that an LLM may treat external data as instructions. Do not rely on the model alone to prevent injection. Give it only the access needed for its intended function, then test the deployed workflow for harmful outputs and unintended actions. section 13.2.1 section 13.3.2
Include jailbreak and injection attempts in risk-based evaluation. Record harmful results, repeat measurements after mitigations, and use red-teaming for higher-risk systems where feasible. These checks help reveal failures; they do not make the attack surface disappear. section 13.3.2 section 13.4
