Chat Markup Language (ChatML) is a message-formatting approach that distinguishes system prompts, user prompts, and external content for language-model processing. It helps applications reduce the influence of untrusted content during prompt-injection attacks. section 13.2.1
Use ChatML as one layer of a broader defensive design:
- Separate user instructions from external content so the model can identify where input originated. section 13.2.1
- Limit model access with least privilege, identity-based authentication, and function-level permissions. section 13.2.1
- Require user confirmation before sensitive operations and preserve human control over consequential decisions. section 13.2.1
- Combine markup with logging, trust limits, and visual warnings for potentially untrustworthy responses. section 13.2.1
