Capability retention
Capability retention is the proportion of evaluated capabilities or performance preserved by a modified or derived model relative to a reference model.
LLM Customization and Fine-Tuning
The student model's score divided by the teacher model's score on the same evaluation metric and set.
- How it is used
- Uses retention as the primary relative success metric when evaluating a distilled student against its teacher and unadapted base.
- How deep it goes
- Defines the score ratio and works through category-level interpretation, scaling estimates, and quality targets, while noting overlap metrics do not assess safety.
- Examples
- Compares a 50-example IT support evaluation with DeepSeek-R1 students on MATH-500 and AIME 2024.
- What you can do after
- Evaluate/understand a student’s relative quality, set a task-sensitive target, and identify when more data or a different approach may be needed.
Read how LLM Customization and Fine-Tuning explains capability retention →
Rearchitecting LLMs
Capability retention is the proportion of a model's evaluated performance preserved after a structural modification such as pruning.
- How it is used
- Compares preserved benchmark capability after pruning or distillation against a baseline model.
- How deep it goes
- Operationalizes retention through controlled benchmarks, progressive pruning curves, and configuration experiments, but does not define a universal aggregation formula.
- Examples
- Uses WinoGrande, Lambada, ARC-Easy, HellaSwag, PIQA, progressive block removal, and a 14-block distilled student.
- What you can do after
- Evaluate/understand which compressed-model configurations preserve capabilities and where further modification causes unacceptable degradation.
Read how Rearchitecting LLMs explains capability retention →
Related concepts
- student modelmeasures · A student model is a model trained with guidance from a teacher model to approximate its outputs, behavior, or learned capabilities.
- teacher modeluses · A teacher model is a model whose outputs, behavior, or learned representations provide training signals for another model.
- retentionkind of · Retention is a measure of the proportion or amount of a defined population, relationship, activity, or information that persists over a specified period.
Questions about capability retention
What is capability retention?
Capability retention is the proportion of evaluated capabilities or performance preserved by a modified or derived model relative to a reference model.
Which Manning books explain capability retention?
LLM Customization and Fine-Tuning and Rearchitecting LLMs.
What is capability retention a kind of?
Capability retention is a kind of retention. Retention is a measure of the proportion or amount of a defined population, relationship, activity, or information that persists over a specified period.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

