TrainingArguments is the configuration object that tells a Hugging Face training run how to manage fine-tuning. It gathers practical choices such as output location, per-device batch size, training epochs, and checkpoint-saving strategy, while Transformers presets handle settings such as the optimizer, learning rate, and warm-up schedule. section 10.2.2
Configure a GPT-2 fine-tuning run
from transformers import TrainingArguments training_args = TrainingArguments( output_dir=DATA_DIR / 'ch10_checkpoints', per_device_train_batch_size=5, num_train_epochs=5, save_strategy='epoch' ) Start with a small batch size when GPU memory is uncertain; increase it only after checking for out-of-memory errors. Pass the resulting configuration to Trainer alongside training and evaluation datasets. section 10.2.2
Listing 10.11 Defining training arguments for GPT-2 fine-tuning
>>> from nlpia2.constants import DATA_DIR #1 >>> from transformers import TrainingArguments >>> from transformers import DataCollatorForLanguageModeling >>> training_args = TrainingArguments( ... output_dir=DATA_DIR / 'ch10_checkpoints', ... per_device_train_batch_size=5, ... num_train_epochs=5, ... save_strategy='epoch') >>> collator = DataCollatorForLanguageModeling( ... tokenizer=tokenizer, mlm=False) #2
Inspect the complete TrainingArguments and DataCollatorForLanguageModeling configuration used for GPT-2 fine-tuning. section 10.2.2
