Prompt security treats prompt design as a security control: attacks can target model behavior through inputs, and defenses begin with how prompts are written. It does not require changing application infrastructure, but prompt-level defenses have limits. section 10.0 section 10.5
A key distinction is trusted versus untrusted input. Prompt security addresses prompt injection, jailbreaking, and instruction conflict, and draws on structured prompts, precise behavioral constraints, and separation of stable instructions from variable inputs. section 10.0
A data-extraction prompt
A user tells a chatbot to ignore prior instructions and reveal recent orders with customer names, email addresses, and shipping details. The chatbot complies, exposing customer information without database access or an authentication bypass. section 10.0
Prompt hardening cannot reliably prevent disclosure of sensitive information already included in the model’s context. Excluding that information is the fully reliable way to prevent its reproduction. Defensive prompts also require validation when the model or its version changes. section 10.5 section 10.5
