5 Parameter-Efficient Fine-Tuning: LoRA and QLoRA
This chapter covers
- How LoRA adapts models with low-rank matrices
- Choosing LoRA rank, alpha, and other settings
- Using QLoRA for memory-constrained hardware
- Testing for safety regressions after tuning
- Evaluating ROI and selecting training platforms
Parameter-efficient fine-tuning (PEFT) enables organizations to update large language models for new tasks without the expense, technical requirements, or risks of retraining the entire model. With PEFT, only a small number of task-specific parameters are trained (typically 0.1% to 5% of the model’s parameters, which can still be millions), while the main model remains unchanged. Many PEFT approaches use adapter-like modules, small, trainable add-ons placed within specific layers, to guide the model’s responses for specific tasks or areas.
There are several PEFT approaches; however, LoRA (Low‑Rank Adaptation) [1] is the most widely used. It delivers most of the benefits of supervised fine‑tuning at a fraction of the cost. QLoRA extends this approach by combining LoRA with aggressive quantization, enabling fine‑tuning of very large models on memory‑constrained hardware. Other PEFT methods, such as prefix vectors, prompt embeddings, and activation scaling, follow the same core principle but differ in how and where adaptation is applied. In practice, LoRA has emerged as the dominant approach due to its simplicity, strong tooling support, scalability, and model quality.