liveBook · Manning

SFTTrainer

As explained in Rearchitecting LLMs

SFTTrainer is a training component that performs supervised fine-tuning of language models using a configured dataset and training procedure.

SFTTrainer runs supervised fine-tuning on a model and training dataset. For clinical extraction, examples are formatted as system, user, and assistant messages, then exposed through a text column for next-token learning. section 7.3.3

Configure the training run through SFTConfig:

  • Set epochs, per-device batch size, learning rate, scheduler, warmup, precision, sequence length, and the dataset text field. section 7.3.3
  • Pass the model, training dataset, and configuration to SFTTrainer, then call trainer.train(). section 7.3.3
Listing 7.11 Configuring and training QLoRA
from trl import SFTTrainer, SFTConfig #1
 
training_args = SFTConfig( 
    output_dir="./qlora_output",
    num_train_epochs=3, #2
    per_device_train_batch_size=8,
    gradient_accumulation_steps=1,
    learning_rate=2e-4, #3
    lr_scheduler_type="cosine", #4
    warmup_steps=7, #5
    fp16=False,
    bf16=True, #6
    logging_steps=10,
    save_strategy="no",
    max_length=MAX_SEQ_LENGTH,         
    dataset_text_field="text", #7
    report_to="none",
)
 
trainer = SFTTrainer(
    model=model,
    train_dataset=train_dataset,
    args=training_args,
)
trainer.train()
 

See a complete QLoRA training configuration and SFTTrainer invocation. section 7.3.3

Clinical extraction

After fine-tuning, the model achieved 100% schema compliance on 40 test samples with a minimal prompt, compared with 85% for the baseline strict prompt. section 7.3.3

How Rearchitecting LLMs covers it

How deep it goes
Configures dataset field, epochs, batch size, learning rate schedule, warmup, precision, and sequence length; leaves trainer internals unspecified.
Examples
Uses formatted medical notes and structured labels for extraction, then evaluates schema compliance with an “Extract:” prompt.
What it assumes
Assumes a model prepared for k-bit gradients, a LoRA adapter, and training examples exposed in a text column.
What you can do after
Apply via TRL's SFTTrainer to fine-tune an adapted model on formatted examples, then evaluate schema compliance.
How it connects
Connects SFTTrainer to SFTConfig, LoRA, QLoRA, chat-template formatting, and next-token learning.

Also explained in

Compare how 3 books explain SFTTrainer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.