A language model interprets natural language for meaning. In a prompt that asks for an email summary, the instruction and email arrive as plain text; the model infers which is command and which is data from context, not a reliably enforced technical boundary. Filters can block obvious attacks, but they remain workarounds rather than hard boundaries. section 4.0
A model’s behavior is shaped by its weights: numerical parameters adjusted during training. Because those parameters encode behavior, inspecting a model file cannot reveal every vulnerability embedded in what the model has learned. section 4.4.1
Account for these risks when deploying or adapting language models:
- Treat prompt filters as bypassable safeguards, not as a guaranteed separation between instructions and data. section 4.0
- Check safety behavior after fine-tuning: adapting a model can weaken refusals, even when the added dataset is benign. section 4.4.1
Fine-tuning can change safety behavior
A reported study found that fine-tuning GPT-3.5 Turbo on 10 adversarial examples weakened its safety guardrails; ordinary datasets also degraded safety, though less dramatically. section 4.4.1
























































































