A student model is a smaller or cheaper model trained to reproduce a stronger teacher model’s behavior. In black-box distillation, the teacher generates example responses, and the student learns from those responses through supervised fine-tuning. The trained student can serve requests without consulting the teacher. section 7.0 section 7.9
A locally served student can reduce inference costs, network latency, data egress, and dependence on an API provider. Teacher-data generation and training still have upfront costs, and prompts sent to an external teacher during data generation require care. The decision depends on whether the student retains enough task-specific quality to justify the tradeoff. section 7.3 section 7.9
A practical student pipeline has four steps:
- Generate task examples with a teacher, then filter and verify its outputs; terse or repetitive responses can degrade the training set. section 7.0 section 7.9
- Fine-tune the student on the verified data using chat-template formatting and standard supervised-training checks for learning rate, epochs, and overfitting. section 7.1.4
- Evaluate the student against both the unadapted base and the teacher on the same held-out inputs. Check task behavior and answer format alongside lexical scores. section 7.2.2 section 7.9
- Test safety independently before deployment. A student may lose refusal behavior even when its task scores and answer format resemble the teacher’s. section 7.9
LoRA is a useful student-training choice when data is limited: adapters can be versioned, rolled back, or swapped on a shared base model. Its limited capacity may miss behaviors that full fine-tuning can capture. If the task is too complex or the capacity gap is too large, full fine-tuning is an alternative. section 7.1.4
Listing 7.3 Training the student (train_student.py)
# chapter07/train_student.py (key sections -- full source in repo)
DEFAULT_MODEL = "Qwen/Qwen3-4B-Instruct-2507"
# ... argparse, dataset loading ...
# --- Student model with LoRA ---
model = AutoModelForCausalLM.from_pretrained(
args.model, torch_dtype=torch.bfloat16,
device_map="auto", trust_remote_code=True,
)
lora_config = LoraConfig( #A
r=args.lora_r, # default: 16
lora_alpha=args.lora_alpha, # default: 32
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM",
)
sft_config = SFTConfig( #B
output_dir=str(out_dir),
num_train_epochs=args.epochs,
per_device_train_batch_size=args.batch_size,
gradient_accumulation_steps=args.grad_accum,
learning_rate=args.lr, # default: 2e-4
warmup_ratio=args.warmup_ratio,
eval_strategy="epoch",
save_strategy="epoch",
load_best_model_at_end=True,
bf16=use_bf16,
max_length=args.max_length,
# ...
)
trainer = SFTTrainer( #C
model=model,
args=sft_config,
train_dataset=train_ds,
eval_dataset=valid_ds,
processing_class=tokenizer,
peft_config=lora_config,
)
trainer.train()
trainer.save_model(str(out_dir))The student-training listing shows a Qwen base model configured with a LoRA adapter for supervised fine-tuning. section 7.1.4
IT support student
On a 50-example held-out test, an IT support student scored 0.165 Token-F1 versus 0.168 for its teacher and 0.153 for the base model: about 98% of the teacher’s score. Its 130 MB adapter still requires the base model to run. Token-F1 measures word overlap, not factual correctness or faithful behavior, and the small test makes category results noisy. The student matched the teacher’s house format on every held-out prompt but lost harmful-request refusals that the base model had. section 7.2.2 section 7.9 section 7.1.5
Distillation transfers task-specific patterns such as response structure, detail, and expressions of uncertainty; it does not guarantee new capabilities beyond the teacher. Teacher mistakes can also be reproduced. Retention varies by task, and a small student can struggle especially on hard reasoning tasks. section 7.4.1 section 7.3.1 section 7.9





