liveBook · Manning

Fine-tuning

As explained in Building Specialized AI Systems

Fine-tuning is the process of adapting a pretrained model to a related task or specialized domain by continuing training with targeted data.

Fine-tuning adapts a pretrained foundation model to a related task or domain by continuing training on curated examples. Instead of building a model from scratch, it carries forward general language and reasoning capabilities, then adjusts weights toward target-domain knowledge, reasoning patterns, and output behavior. The result is a specialized language model optimized for precision and consistency where a general-purpose model is insufficient. section 1.4.1 section 1.3.1

Figure 1.5 Transfer learning illustrated: a model's knowledge, acquired during pretraining on a broad task, is carried forward and adapted for a more specific downstream task. Fine-tuning is the practical application of this principle in the context of LLMs — the mechanism by which a general-purpose foundation model becomes a SpeLM.

Transfer learning is the central idea: broad knowledge acquired during pretraining is retained and redirected toward a narrower downstream task. section 1.4.1

For supervised fine-tuning (SFT), training data consists of prompt-completion pairs: an input and the desired response. Repeated exposure to representative, high-quality pairs gradually teaches the model to produce domain-appropriate outputs. Other approaches include RLHF, which uses human preferences to guide behavior, and parameter-efficient methods such as LoRA or QLoRA, which adapt a limited set of trainable components rather than retraining every parameter. section 1.4.1 section 1.4 section 3.0

A practical fine-tuning cycle is iterative:

  • Select a foundation model with relevant baseline abilities while balancing capability, compute, and deployment constraints. section 1.4.2
  • Prepare curated input-output data by collecting relevant examples, standardizing formats, and reviewing quality. section 2.3.1
  • Configure and train an adaptation method, then evaluate outputs against target requirements. section 1.7
  • Return to model choice, data preparation, or training settings when evaluation exposes gaps, inconsistency, or overfitting. section 2.3.1

Credit-risk classification with LoRA

A general-purpose LLaMA-3 8B model is adapted to classify a loan applicant using income, debt, credit score, and late-payment data. Training uses a LoRA adapter, 4-bit quantization, formatted examples, and an SFT trainer; the resulting output labels the illustrated applicant as “High Risk.” section 2.1.2

Fine-tuning is most justified when high accuracy, consistent behavior, or deep domain reasoning is required. Prompt engineering can suit lower-stakes needs, while RAG is preferable for external knowledge without behavioral adaptation; combining RAG with fine-tuning fits applications needing both current information and specialized responses. Data quality is decisive, and specialization can require substantial preparation, iteration, evaluation, and compute. section 1.5.1 section 2.3

How Building Specialized AI Systems covers it

How deep it goes
Explains transfer learning, prompt-completion training, iterative workflow, and parameter-efficient methods; defers Transformer internals and method selection details.
Examples
Uses customer-support ticket classification, credit-risk assessment, and financial analyst-report summarization with structured input-output pairs.
What it assumes
Assumes a pretrained model and task data while introducing the end-to-end adaptation process without requiring every training detail initially.
What you can do after
Build it from scratch by selecting a base model, preparing examples, training an adapter, evaluating results, and deploying iteratively.
How it connects
Connects transfer learning, supervised fine-tuning, RLHF, LoRA, QLoRA, prompt engineering, RAG, and specialized language models.

Also explained in

Compare how 52 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building Specialized AI Systems. The section links point to the parts of the book it draws on.