simpletransformers provides high-level wrappers around Hugging Face transformer models, turning a multi-label text-classification workflow into DataFrame preparation, model configuration, fine-tuning, and evaluation. section 9.3.4
A practical workflow includes:
- Represent each example as [text, labels], using multi-hot vectors when one comment can receive several labels. section 9.3.4
- Split the toxic-comments dataset into training and validation sets, then configure model paths, seeds, sequence lengths, batches, and learning rates. section 9.3.4
- Fine-tune MultiLabelClassificationModel and evaluate it with ROC AUC, LRAP, loss, and accuracy; evaluation during training supports selecting a best model. section 9.3.4
Toxic-comment tagging
Load approximately 160,000 comments, assign toxic, severe, obscene, threat, insult, and hate labels, prepare train_df and eval_df, then fine-tune bert-base-cased with MultiLabelClassificationModel. section 9.3.4
Listing 9.35 Loading the pretrained model and fine-tuning
>>> from sklearn.metrics import roc_auc_score >>> from simpletransformers.classification import MultiLabelClassificationModel >>> model = MultiLabelClassificationModel( ... model_type, model_name, num_labels=len(labels), ... args=model_args) You should probably TRAIN this model on a downstream task to be able to use it for predictions and inference >>> model.train_model(train_df=train_df) #1
Browse the compact constructor and training call that load a pretrained BERT classifier and fine-tune it on prepared data. section 9.3.4
