liveBook · Manning

Fine-tuning

As explained in Machines that Think

Fine-tuning is the continued adjustment of a pretrained model on narrower data or objectives for a particular task or behavior.

Fine-tuning adapts a pretrained model to a narrower task, domain, or behavior through additional training. Rather than starting from scratch, it builds on structure learned from broad data and changes how the model behaves by modifying or augmenting its learned transformations. section 4.3.1

Figure 4.13 Pretraining and model adaptation. Models are first trained on broad data, then specialized through full or partial parameter updates.

The figure contrasts broad pretraining with full and partial model adaptation. section 4.3.1

Full fine-tuning adjusts all model parameters for a new objective or domain. Partial adaptation changes a limited subset while keeping the rest frozen; it may update existing weights or introduce new trainable components. These choices differ in scope and cost. Further training can also weaken capabilities acquired during pretraining, a risk known as catastrophic forgetting. section 4.3.1

Supervised fine-tuning uses curated examples of desired behavior to shape a language model’s tone, format, role, and response patterns. Examples can also teach it to refuse harmful requests or redirect sensitive interactions. Fine-tuning can reinforce fairer responses, but the result is a learned behavioral tendency, not a guaranteed rule. section 7.5 section 7.4.4

Changing model behavior through training differs from supplying external context or filtering outputs: those interventions can guide or constrain visible responses without necessarily changing what the model has learned. Prompts and corrections within a conversation likewise do not normally alter the underlying model or persist as accumulated experience. section 7.4.4 section 9.4.4

How Machines that Think covers it

How deep it goes
Contrasts full and partial parameter adaptation and explains their risks, without a training procedure or implementation.
Examples
Uses curated examples of refusals, safer redirection, tone, and response formats to illustrate supervised fine-tuning.
What you can do after
Evaluate fine-tuning by distinguishing learned behavioral tendencies from guarantees and weighing specialization against forgetting.
How it connects
Connects pretraining, alignment, bias mitigation, and catastrophic forgetting.

Also explained in

Compare how 52 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.