liveBook · Manning

Fine-tuning

As explained in Understanding AI

Fine-tuning is the continued training of a pretrained model on a smaller, task- or behavior-specific dataset to adapt its outputs.

Why does the same language model sometimes continue your sentence and sometimes answer your question? Fine-tuning is additional training that changes this default behavior. A pretrained foundation model learns broad language patterns by predicting likely next tokens; fine-tuning continues training on a smaller, carefully curated set of prompt-response pairs so it learns the pattern of responding as an assistant. section 2.2.2 section 2.3

The input becomes a request

Given “Summarize this concert review in one sentence,” a raw model may treat the words as text to extend. A fine-tuned conversational model has repeatedly seen prompts paired with helpful answers, so it is more likely to interpret the same input as an instruction and return a summary. section 2.2.2 section 2.3

Fine-tuning changes behavior through a simple sequence:

  • Start with a foundation model that can generate plausible continuations but may behave unpredictably in conversation. section 2.2.2
  • Train further on high-quality examples of user prompts and desired assistant responses. section 2.2.2
  • Learn the response pattern: answer questions, follow instructions, and organize explanations for people. section 2.2 section 2.2.2

Fine-tuning does not replace the underlying prediction mechanism. The model still predicts one token at a time; the additional examples redirect what kinds of continuations it has learned to produce. Alignment training is a related later step that uses human feedback to shape which responses people prefer and to create behavioral guardrails. section 2.2.4 section 2.3

The same approach can also make a model more specialized. After general instruction-following training, additional domain-focused training can adapt a shared foundation toward coding, law, medicine, or other kinds of data, vocabulary, and conventions. section 2.2.4 section 2.3

Figure 2.7 The same prompt, fed to two different models. The foundation model treats it as text to continue; it doesn't recognize an instruction, it simply extends the pattern. The fine-tuned conversational model reads it as a request and responds accordingly. The input is identical. What changes is how the model has learned to interpret it.

Compare identical input sent to a foundation model and a fine-tuned conversational model to see how learned interpretation changes the response. section 2.2.2

How Understanding AI covers it

How deep it goes
Explains curated prompt-response training and its behavioral effect, while deferring training mathematics and update mechanics.
Examples
Contrasts a concert-review summary prompt treated as text continuation with one answered as an assistant request.
What you can do after
Evaluate/understand why an assistant responds to a prompt differently from a raw foundation model.
How it connects
Connects fine-tuning with pretraining, foundation models, alignment training, human feedback, specialization, and next-token prediction.

Also explained in

Compare how 52 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.