liveBook · Manning

Chapter 7: Model specialization

What you will learn in chapter 7 of Rearchitecting LLMs

The chapter introduction explains how fine-tuning can specialize a model’s behavior, contrasts it with RAG, and motivates parameter-efficient fine-tuning.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The chapter introduction explains how fine-tuning can specialize a model’s behavior, contrasts it with RAG, and motivates parameter-efficient fine-tuning.

Key terms: RAG, fine-tuning, inference time, model's weights, structured formats, generalist LLM, full retraining, parameter-efficient fine-tuning (PEFT), less than 1% of its parameters

7.1 Efficient fine-tuning

This section explains why full fine-tuning is costly and can disrupt pretrained knowledge, then introduces parameter-efficient fine-tuning as an alternative.

Key terms: computational cost, VRAM, catastrophic forgetting, parameter efficient fine-tuning (PEFT), frozen architecture, base model, adapters, delta

7.1.1 The foundations of LoRA

This section explains how LoRA adapts a frozen weight matrix with a low-rank update, why that saves trainable parameters, and how the update starts without changing the model.

Key terms: LoRA, W, ΔW, A, B, rank, r, q_proj, trainable parameters, W_adapted

7.1.2 Extending LoRA with weight decomposition

This section explains why DoRA separates weight magnitude from direction, how its update is constructed, and why that separation can improve specialization over LoRA.

Key terms: LoRA, ΔW = A·B, magnitude, direction, full fine-tuning, DoRA, W0, magnitude vector m, direction matrix V, A @ B_trained

7.2 Quantization and the memory bottleneck

This section explains why quantizing a frozen base model can make adapter fine-tuning fit in GPU memory, and why NF4 is a useful four-bit storage format.

Key terms: PEFT, frozen base model, quantization, memory bottleneck, NF4, NormalFloat 4-bit, uniform quantization, normal distribution, quantization levels, storage format

7.3 Specializing with QLoRa and QDoRA

Introduces a clinical specialization experiment in which fine-tuning aims to move instructions from a long prompt into model weights without sacrificing general capabilities.

Key terms: fine-tuning, behavioral instructions, model's weights, structured JSON, strict schema, general capabilities

7.3.1 Base model and clinical dataset

Defines the clinical extraction task, its synthetic dataset, and the measurable goals for specializing the base model.

Key terms: structured clinical information extraction, SmolLM2-1.7B-Instruct, strict schema, validation failure, synthetic dataset, structured output format, Clean, Abbreviations, Implicit, Typos

7.3.2 Establishing the baseline: reliability and capabilities

Establishes a pre-training baseline for structured-response reliability and general capabilities, then identifies where prompt-only guidance fails.

Key terms: schema compliance, general performance baseline, standard benchmarks, knowledge preservation, check_schema_compliance, is_valid_json, matches_schema, failure_reason, STRICT_PROMPT, irrelevant inputs

7.3.3 QLoRA training on the clinical dataset

This section trains a 4-bit clinical extraction model with QLoRA to use a one-word prompt, then evaluates schema reliability and general-capability trade-offs.

Key terms: chat template, MINIMAL_PROMPT, system, user, assistant, strict JSON schema, 4-bit, prepare_model_for_kbit_training, LoRA adapter, use_dora=False

7.3.4 QDoRA training on the clinical dataset

This section switches a clinical fine-tuning run from QLoRA to QDoRA and compares their trainable parameters and results.

Key terms: QDoRA, QLoRA, use_dora=True, SFTConfig, SFTTrainer, trainable parameters, magnitude vectors m, direction V, target matrices, schema compliance

7.3.5 Evaluating the results against our goals

The section evaluates whether QLoRA and QDoRA met the specialization goals and weighs their quality and training-efficiency trade-offs.

Key terms: inference efficiency, format contract, schema compliance, minimal prompt, general knowledge, benchmarks, ARC-Easy, specialized behavior, QLoRA, QDoRA

7.3.6 Merging weights and dynamic adapter routing

This section explains how to deploy a trained adapter either by merging it into the base model or by loading adapters dynamically, and why dynamic routing should batch requests by task.

Key terms: PEFT, adapter, save_pretrained, merge_and_unload(), save_pretrained(), base model, PeftModel.from_pretrained(), LoRA, DoRA, adapter switch

7.4 From paper to practice

Connects the chapter’s specialization results to research on efficient fine-tuning, explaining why method choice depends on the task and why clinical fine-tuning can improve output reliability.

Key terms: LoRA, QLoRA, directional delta, 4-bit NormalFloat quantization, nf4, DoRA, magnitude, direction, full fine-tuning, semantic adaptation

7.5 Hands-on lab

7.6 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 6: Knowledge recovery through distillation Chapter 8: Attention optimization →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.