SFTTrainer is TRL’s supervised fine-tuning component for transformer language models. It reduces setup work by handling dataset formatting, tokenization, prompt masking and LoRA adapter wrapping in a message-format training workflow. section 5.1
The main configuration decision is whether to train an adapter or the full model. Pass a LoRA configuration as peft_config to attach and train adapter parameters; omit it for full-parameter supervised fine-tuning. Full fine-tuning gives more freedom to change behavior but costs more compute and carries greater overfitting risk. section 5.1.2 section 6.1.3 section 6.15
A practical training setup includes:
- A base model and tokenizer, plus training and validation datasets prepared in a consistent message format. section 2.8 section 5.1
- An SFTConfig specifying the training budget, batch size, gradient accumulation, learning rate and evaluation cadence. Pass the tokenizer as processing_class and call trainer.train(). section 2.8
- Validation monitoring and comparison against the base model. A widening training–validation loss gap can indicate overfitting; run safety regression tests before deployment. section 6.15
Listing 2.4 Step 3, train for 20 steps
from trl import SFTConfig, SFTTrainer
import torch
use_bf16 = torch.cuda.is_available() and torch.cuda.is_bf16_supported()
config = SFTConfig(
output_dir="chapter02/runs/ch2_quickstart",
max_steps=20,
per_device_train_batch_size=1,
gradient_accumulation_steps=8,
learning_rate=2e-4,
max_length=512,
logging_steps=5,
eval_strategy="steps", eval_steps=10,
save_strategy="no",
bf16=use_bf16, fp16=not use_bf16 and torch.cuda.is_available(),
seed=42, report_to=[],
)
trainer = SFTTrainer(
model=model, args=config,
train_dataset=train_ds, eval_dataset=valid_ds,
peft_config=lora_config, processing_class=tokenizer,
)
trainer.train()This configuration trains a LoRA adapter for 20 steps, evaluates every 10 steps and uses gradient accumulation to reach an effective batch size of eight. section 2.8
IT support quickstart
A Qwen3-4B-Instruct-2507 model trains on 40 examples for 20 steps with LoRA. On one 24 GB NVIDIA A30, the run took 78 seconds and used about 9 GB of GPU memory. Training loss fell from 2.94 to 1.69, while validation loss fell only from 1.98 to 1.87. This demonstrates the pipeline, not reliable generalization to new tickets. section 2.8
For a larger full-parameter run, gradient checkpointing can reduce memory use at the cost of extra computation. Monitor gradient norms as well as validation loss: sudden spikes can signal instability, while near-zero norms early in training may indicate a stalled run or a learning rate that is too low. section 5.1.2 section 6.1.3


