A refusal example is a training instance that pairs a harmful or disallowed request with a response declining it. Without such examples, a model trained on helpful answers may learn to answer even when it should refuse. section 5.1.4 section 7.8.1
Safety behavior can regress even when task performance is preserved. In black-box distillation, a student exposed only to benign teacher outputs does not see when or how the teacher refuses. Include harmful prompts paired with the teacher’s refusals, then re-test the student. Check teacher outputs for safety as well as quality: fluent, relevant responses can still be harmful. section 7.8.1
A smaller adapter did not restore safety
An IT-support fine-tuning dataset contained 450 helpful examples and no refusal examples. The base model refused 100% of safety prompts, while adapters at ranks 16 and 8 both refused 60%. Reducing rank did not address the missing safety data. section 5.1.4
Table 5.1 LoRA rank comparison; unexpected results from safety fix attempt
|
Metric
|
Base
|
r=16
(Original)
|
r=8 (Attempted
Fix)
|
|
Token-F1
|
0.149
|
0.154 (+0.005)
|
0.167 (+0.018)
|
|
Safety
|
100%
|
60% (-40%)
|
60% (-40%)
|
|
Hardware F1
|
0.152
|
0.188 (+0.037)
|
0.183 (+0.031)
|
|
Linux F1
|
0.129
|
0.144 (+0.014)
|
0.162 (+0.032)
|
Compare the base model and both adapter ranks on safety and task metrics. section 5.1.4
For a dataset of this scale, plan for 50 to 100 explicit refusal examples, roughly 10% to 20% of the data. Treat rank as a capacity and cost choice, not a safety control. Validate safety empirically against the base model before deployment; adding examples does not guarantee alignment. section 5.1.4
