liveBook · Manning

Student model

As explained in LLM Customization and Fine-Tuning

A student model is a smaller model trained to mimic a teacher model during distillation.

A student model is a smaller or cheaper model trained to reproduce a stronger teacher model’s behavior. In black-box distillation, the teacher generates example responses, and the student learns from those responses through supervised fine-tuning. The trained student can serve requests without consulting the teacher. section 7.0 section 7.9

A locally served student can reduce inference costs, network latency, data egress, and dependence on an API provider. Teacher-data generation and training still have upfront costs, and prompts sent to an external teacher during data generation require care. The decision depends on whether the student retains enough task-specific quality to justify the tradeoff. section 7.3 section 7.9

A practical student pipeline has four steps:

  • Generate task examples with a teacher, then filter and verify its outputs; terse or repetitive responses can degrade the training set. section 7.0 section 7.9
  • Fine-tune the student on the verified data using chat-template formatting and standard supervised-training checks for learning rate, epochs, and overfitting. section 7.1.4
  • Evaluate the student against both the unadapted base and the teacher on the same held-out inputs. Check task behavior and answer format alongside lexical scores. section 7.2.2 section 7.9
  • Test safety independently before deployment. A student may lose refusal behavior even when its task scores and answer format resemble the teacher’s. section 7.9

LoRA is a useful student-training choice when data is limited: adapters can be versioned, rolled back, or swapped on a shared base model. Its limited capacity may miss behaviors that full fine-tuning can capture. If the task is too complex or the capacity gap is too large, full fine-tuning is an alternative. section 7.1.4

Listing 7.3 Training the student (train_student.py)
# chapter07/train_student.py (key sections -- full source in repo)
 
DEFAULT_MODEL = "Qwen/Qwen3-4B-Instruct-2507"
 
# ... argparse, dataset loading ...
 
# --- Student model with LoRA ---
model = AutoModelForCausalLM.from_pretrained(
    args.model, torch_dtype=torch.bfloat16,
    device_map="auto", trust_remote_code=True,
)
 
lora_config = LoraConfig(                                      #A
    r=args.lora_r,             # default: 16
    lora_alpha=args.lora_alpha, # default: 32
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)
 
sft_config = SFTConfig(                                        #B
    output_dir=str(out_dir),
    num_train_epochs=args.epochs,
    per_device_train_batch_size=args.batch_size,
    gradient_accumulation_steps=args.grad_accum,
    learning_rate=args.lr,       # default: 2e-4
    warmup_ratio=args.warmup_ratio,
    eval_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end=True,
    bf16=use_bf16,
    max_length=args.max_length,
    # ...
)
 
trainer = SFTTrainer(                                          #C
    model=model,
    args=sft_config,
    train_dataset=train_ds,
    eval_dataset=valid_ds,
    processing_class=tokenizer,
    peft_config=lora_config,
)
trainer.train()
trainer.save_model(str(out_dir))

The student-training listing shows a Qwen base model configured with a LoRA adapter for supervised fine-tuning. section 7.1.4

IT support student

On a 50-example held-out test, an IT support student scored 0.165 Token-F1 versus 0.168 for its teacher and 0.153 for the base model: about 98% of the teacher’s score. Its 130 MB adapter still requires the base model to run. Token-F1 measures word overlap, not factual correctness or faithful behavior, and the small test makes category results noisy. The student matched the teacher’s house format on every held-out prompt but lost harmful-request refusals that the base model had. section 7.2.2 section 7.9 section 7.1.5

Distillation transfers task-specific patterns such as response structure, detail, and expressions of uncertainty; it does not guarantee new capabilities beyond the teacher. Teacher mistakes can also be reproduced. Retention varies by task, and a small student can struggle especially on hard reasoning tasks. section 7.4.1 section 7.3.1 section 7.9

How LLM Customization and Fine-Tuning covers it

How deep it goes
Walks through teacher-data filtering, student fine-tuning, inference, three-way evaluation, and independent safety checks.
Examples
Uses an IT support assistant with a Qwen student, a LoRA adapter, and a 50-example held-out comparison.
What it assumes
Assumes familiarity with supervised fine-tuning, chat templates, and training hyperparameters while relating distillation to that workflow.
What you can do after
Apply it via SFTTrainer and LoRA to train and evaluate a student on teacher-generated responses.
How it connects
Connects knowledge distillation with supervised fine-tuning, LoRA, behavioral evaluation, and safety alignment.

Also explained in

Compare how 6 books explain student model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.