The LoRA scaling factor, α, controls how strongly a low-rank adapter contributes to a model’s effective weights. Its effect depends on the ratio α/r, where r is the adapter rank; changing α without considering rank changes that ratio. section 5.3
W is the base weight, A×B is the low-rank adapter contribution, and α/r scales that contribution. section 5.3
A practical starting configuration is r=16 and α=32, giving α/r=2. The configuration example describes α as typically twice the rank. Rank is a separate decision: it controls adapter expressivity, with higher ranks increasing memory use and training time. section 5.3 section 5.1.2
Configure an IT support adapter
For an IT support fine-tuning run, use --lora_r 16 and --lora_alpha 32, the training script’s defaults. In PEFT, the corresponding LoraConfig fields are r=16 and lora_alpha=32; TRL’s SFTTrainer applies the adapter while training on the prepared data. section 5.1.2
If validation performance plateaus, try increasing rank from 16 to 32. The sample configurations pair r=32 with α=64, preserving the α/r ratio of 2. Their results change rank and α together, so they do not isolate the effect of α alone. section 5.3
