liveBook · Manning

Regression

As explained in LLM Customization and Fine-Tuning

A change that makes a model less likely to decline requests outside its competence.

A fine-tuned healthcare model may become less likely to decline questions outside its competence, even as it improves at its target task. This is a behavioral regression: an adaptation degrades a useful behavior the model previously had. Safety regression includes losing the ability to refuse harmful requests or acknowledge uncertainty. section 6.12.6 section 8.9

Regression can occur without an intentional guardrail change. Fine-tuning may erode base-model alignment, while distillation may transfer capability without preserving all teacher safety behaviors. Preference optimization for helpfulness can also favor detailed answers where refusal is appropriate. section 2.9 section 8.9

A smaller adapter did not restore refusals

An IT support LoRA model trained on 450 helpful examples and no refusal examples fell from a 100% base-model refusal rate to 60%. Reducing rank from 16 to 8 left the refusal rate at 60%: changing rank alone did not fix the missing safety examples. Token-F1 differences on 50 examples were within noise, so they did not establish a reliable quality improvement. section 5.1.4

Table 5.1 LoRA rank comparison; unexpected results from safety fix attempt
Metric
Base
r=16 (Original)
r=8 (Attempted Fix)
Token-F1
0.149
0.154 (+0.005)
0.167 (+0.018)
Safety
100%
60% (-40%)
60% (-40%)
Hardware F1
0.152
0.188 (+0.037)
0.183 (+0.031)
Linux F1
0.129
0.144 (+0.014)
0.162 (+0.032)

Compare refusal rates and Token-F1 for the base model and both adapter ranks. section 5.1.4

Use regression testing as a deployment gate:

  • Compare the base and adapted models on the same sensitive prompts in a controlled environment. Check harmful-request refusal, uncertainty acknowledgment, bias neutrality, and general-knowledge retention. section 5.4.1 section 8.9
  • Treat a newly failed safety prompt as a blocker after preference optimization. A model can show no regression yet still have a weak baseline: one comparison passed only one of four checks both before and after DPO. section 8.2.9 section 8.9
  • If refusals degrade, add appropriate refusal examples and retrain, then test again. For critical applications, use an independent filter or model to screen outputs. section 5.4.1 section 8.9
  • Guard against broader behavioral loss with diverse data mixing: include 10–20% general-purpose instruction examples alongside domain data. Early stopping on validation loss helps prevent overfitting, which is a separate concern. section 6.14.4

Keep testing after deployment and model updates. A safety category falling below its threshold is a retraining trigger; input drift alone calls for investigation, not automatic retraining. section 9.7.1 section 6.12.6

How LLM Customization and Fine-Tuning covers it

How deep it goes
Explains failure mechanisms and provides comparative tests, training mitigations, and deployment gates.
Examples
Compares LoRA ranks 16 and 8 on IT support refusals and describes a healthcare model declining fewer out-of-competence questions.
What you can do after
Diagnose regressions by comparing safety responses before and after adaptation, then evaluate mitigations and deployment readiness.
How it connects
Connects regression to fine-tuning, catastrophic forgetting, distillation, preference optimization, and overfitting.

Also explained in

Compare how 11 books explain regression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.