Low-rank adaptation (LoRA) fine-tunes a large pretrained model without updating all its weights. It freezes the base model and trains two small added matrices at selected parts of the model. This makes adaptation practical when full fine-tuning would demand too much memory or compute; it also avoids updating every weight, which can risk catastrophic forgetting. section 5.3.3
LoRA trains roughly 0.1–1% of a model’s parameters. The comparison reports about 95–99% of full fine-tuning performance with low memory use, versus full fine-tuning’s higher memory demand and maximum adaptation. Treat those figures as a method comparison, not a guarantee for a particular task. section 5.3.3
peft_config = LoraConfig( #1
r=8, #2
lora_alpha=16, #3
target_modules=["q_proj", "v_proj"], #4
lora_dropout=0.1, #5
bias="none", #6
task_type="CAUSAL_LM" #7
)
model = get_peft_model(model, peft_config)
A PEFT configuration selects rank 8, alpha 16, dropout 0.1, and the attention projection modules q_proj and v_proj; get_peft_model attaches the adapters to the loaded model. section 5.3.4
Customer-support adaptation
Start with a conversational LLaMA-2-7B model and map a 1,000-example Bitext subset into instruction–response pairs about orders, shipping, and returns. Format each pair as a conversational training example, train with SFTTrainer, then try prompts about refunds, addresses, invoices, and order status. For a company deployment, the same mapping can use customer queries and agent responses from a CSV. section 5.3.4
QLoRA adds a memory-saving option: load the base model in 4-bit NF4 form during adapter training. Start from original full- or half-precision weights rather than an already pre-quantized model. For maintenance, set aside evaluation examples, track evaluation loss, and save the adapter-equipped model and tokenizer; reload the trained adapter with its base model. section 5.3.3 section 5.3.4
