A pretrained model knows patterns from its original training data, but those patterns may not match your task or domain. Fine-tuning continues training from the pretrained parameters on more targeted data. For BERT classification, that means supplying task-specific inputs and labels, then training the model end to end. This is generally less expensive than pretraining BERT from scratch. section 9.3.2 section 9.3.4
Start with the behavior you need and data that represents it. A toxic-comment classifier, for example, uses comment text and six labels: toxic, severe, obscene, threat, insult, and hate. Inspect the examples before training, then split the labeled data into training and evaluation sets. Trying the classification task yourself can reveal whether the labels describe a reasonable NLP problem. section 9.3.4
Listing 9.35 Loading the pretrained model and fine-tuning
>>> from sklearn.metrics import roc_auc_score >>> from simpletransformers.classification import MultiLabelClassificationModel >>> model = MultiLabelClassificationModel( ... model_type, model_name, num_labels=len(labels), ... args=model_args) You should probably TRAIN this model on a downstream task to be able to use it for predictions and inference >>> model.train_model(train_df=train_df) #1
Load pretrained bert-base-cased as a multilabel classifier, set the number of labels, and call train_model on the prepared training data. section 9.3.4
Evaluate on held-out comments rather than trusting the training run. The example calls eval_model with roc_auc_score and then tries text preprocessing, evaluation during training, and early stopping. Its reported evaluation result improves from 0.9812 to 0.9894 for the displayed acc field; that result belongs to this dataset and setup, not to every moderation system. Review errors and the effects of preprocessing before relying on predictions about people’s speech. section 9.3.4
Fine-tuning can also change what a generative model tends to write. Instruction tuning uses labeled examples of tasks and expected outputs; a simpler alternative trains on a chosen text corpus to shift generation patterns. Neither is the same as supplying retrieved evidence for a particular answer. section 10.2.2 section 10.3.13
Listing 10.12 Fine-tuning GPT-2 with Hugging Face’s Trainer class
>>> from transformers import Trainer
>>> ft_model = GPT2LMHeadModel.from_pretrained("gpt2") #1
>>> trainer = Trainer(
... ft_model,
... training_args,
... data_collator=collator, #2
... train_dataset=train_dataset, #3
... eval_dataset=eval_dataset)
>>> trainer.train()
The GPT-2 example loads fresh pretrained weights and uses Hugging Face Trainer with tokenized text, separate training and evaluation datasets, training arguments, and a language-modeling data collator. section 10.2.2
Compare generations before and after training
Fine-tune GPT-2 on lines of natural-language NLP writing, then give the original and fine-tuned models the same prompt. Their different continuations make the change in generation patterns visible. A more on-topic sentence is still not proof that its claims are correct: generated text needs checking, and factual answers may also need semantic search or retrieval-augmented generation. section 10.2.2 section 10.5
Choose a model that fits the task and available compute. Large-model fine-tuning can exceed local hardware capacity, so compare that constraint with pay-per-use computing costs. Fine-tuning is a way to adapt behavior, not a guarantee of reliable or ethical output; relevant domain data and grounding remain important for user-facing systems. section 9.3.3 section 10.3.13
