liveBook · Manning

Fine-tuning

As explained in Rearchitecting LLMs

Fine-tuning that adapts a model for a particular specialization or domain.

Fine-tuning adapts a model to a task or domain by training it further on a specific dataset. It changes model weights and behavior; retrieval-augmented generation (RAG), by contrast, supplies external information at inference time without changing the model. The two approaches can be combined, for example, to adapt a model to use retrieved information more effectively. section 1.1.2 section 7.0

Specialization can make a model produce structured outputs without relying on lengthy instructions in every prompt. Parameter-efficient fine-tuning limits training to a small fraction of parameters, reducing the cost of adaptation. In a tailoring pipeline, fine-tuning can follow pruning and knowledge distillation; an optional initial fine-tuning pass can also align the base model before knowledge recovery. section 7.0 section 1.2 section 1.2.4

Figure 1.1 The model tailoring pipeline consists of core phases (shown with solid arrows) and optional phases (shown with dashed arrows). In the first phase, we adapt the structure to the model's objectives through pruning. Next, we recover capabilities it may have lost through knowledge distillation. Finally, we can optionally specialize the model through fine-tuning. An optional initial phase calibrates the base model, via a brief fine-tuning pass on the target dataset.

The pipeline places optional specialization after structural optimization and knowledge recovery, with an optional calibration pass at the start. section 1.2

Build a function-calling specialist with LoRA

Start with the post-trained Qwen3-0.6B checkpoint, which already supports tool calling. Load it in float16, disable the cache, and enable gradient checkpointing. Configure LoRA with rank 16, alpha 32, and dropout 0.05 across attention and MLP projections. This setup trains about 1.5% of the model's parameters. section 13.3

Listing 13.6 LoRA configuration
lora_config = LoraConfig(
    r=16, 
    lora_alpha=32, 
    lora_dropout=0.05,
    bias="none", 
    task_type="CAUSAL_LM", 
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
)
train_model = get_peft_model(train_model, lora_config)
train_model.print_trainable_parameters()

The LoRA configuration names the targeted projection modules and reports the trainable parameter count. section 13.3

Train for three epochs at a learning rate of 2e-4, with batch size 4 and two gradient-accumulation steps. Completion-only loss excludes the prompt and tools block, concentrating training on generating the call. Save the adapter, merge it into the model, restore the cache, and switch to evaluation mode. section 13.3

Listing 13.8 Training and merging the adapter
trainer.train()
 
train_model.save_pretrained("./lora_adapter")
tokenizer.save_pretrained("./lora_adapter")
 
specialist_model = train_model.merge_and_unload()
specialist_model.config.use_cache = True
specialist_model.gradient_checkpointing_disable()
specialist_model.eval()

The training and merge listing shows the steps that turn the adapter into an evaluation-ready specialist. section 13.3

Measure both task performance and general capabilities before and after training. In a clinical JSON extraction baseline, a strict prompt achieved 85% schema compliance over 40 samples; the irrelevant-input category was especially difficult. Five general benchmarks, each limited to 500 samples, provide reference points for checking knowledge preservation. The small task sample makes its results directional rather than statistically robust. section 7.3.2

Closed-model fine-tuning uses provider tools and produces a hosted artifact whose weights cannot be exported. Training and inference costs may be higher, although pricing changes; a base-model upgrade or retirement can require repeating fine-tuning and evaluation. section 1.1.2

How Rearchitecting LLMs covers it

How deep it goes
Develops an adapter-training workflow from model loading and configuration through training, merging, and evaluation baselines.
Examples
Uses clinical JSON extraction and a three-function calling specialist to illustrate behavioral adaptation.
What you can do after
Apply it via LoRA to specialize a model for function calling and merge the trained adapter.
How it connects
Connects fine-tuning with retrieval-augmented generation, pruning, knowledge distillation, and parameter-efficient fine-tuning.

Also explained in

Compare how 52 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.