NEFTune (Noisy Embeddings Improve Instruction Finetuning) improves instruction fine-tuning by adding controlled uniform noise to input embeddings during training. The perturbation regularizes learning, reducing overfitting and encouraging more robust, generalizable representations without changing the parameter-update strategy used by methods such as LoRA. section 3.5.4 section 3.8
Apply NEFTune through these implementation rules:
- Add a random noise vector after tokens become embeddings and before the embeddings pass through the transformer. section 3.5.4
- Scale noise magnitude with α/√(L×d), using the noise alpha, sequence length, and embedding dimension. section 3.5.4
- Enable noise only during training; leave inference embeddings unperturbed for clean, deterministic outputs. section 3.5.4
Listing 3.33 Configuring the trainer to use NEFT
trainer = SFTTrainer(
# Other training parameters here
neftune_noise_alpha=5
)Configure NEFTune in SFTTrainer with the neftune_noise_alpha parameter. section 3.5.4
Instruction-following improvement
Standard fine-tuning of LLaMA-2-7B on Alpaca reached 29.79% on AlpacaEval; NEFTune reached 64.69%, an improvement of nearly 35 percentage points. section 3.5.4
