Trusted input is prompt text the application controls, such as system instructions, few-shot examples, and task framing. Trust depends on who controls the text, not whether it is benign: runtime content from users, retrieved documents, or external APIs is untrusted because its contents cannot be guaranteed. A prompt does not inherently distinguish these sources, so their interaction must be managed explicitly. section 10.1 section 10.0
Classify prompt content by control:
- Trusted: system instructions, few-shot examples, and task framing established before user input. section 10.1
- Untrusted: runtime user messages, retrieved documents, and external API responses, whether malicious or benign. section 10.1
A mixed prompt
A chatbot places system instructions alongside a user request to reveal customer names and contact details. The system text is controlled; the request is runtime input. Without a boundary, the model may treat the combined text as one continuous block and let the later request influence its behavior. section 10.0 section 10.1
Figure 10.1 How a language model processes a prompt with no boundary between trusted and untrusted input
Illustrates how trusted instructions and untrusted content can appear as one continuous prompt when no boundary separates them. section 10.1
