chapter three

3 Fine-tuning methods

 

This chapter covers

  • Traditional fine-tuning approaches
  • Parameter-efficient methods (LoRA, QLoRA, DoRA)
  • Using Noisy Embedding Fine-Tuning (NEFTune) to improve instruction-following

You built your first specialized models in chapter 2. You fine-tuned an LLM for credit risk assessment, created adapters for different tasks, and walked through analyst report summarization from start to finish. These hands-on examples showed you that fine-tuning works and gave you a feel for the complete process.

What we didn’t explore is how fine-tuning works under the hood or which method to choose for your situation. When you ran those training commands, what was happening inside the model? Why did we use LoRA instead of updating all the parameters? When should you freeze certain layers? What’s the difference between full fine-tuning and parameter-efficient approaches, and how do you decide between them?

3.1 The underlying engine of fine-tuning

3.1.1 Understanding underlying architecture

3.1.2 Understanding the training process: Gradient descent

3.2 Supervised fine-tuning

3.3 Full fine-tuning approaches

3.3.1 Full parameter fine-tuning

3.3.2 Layer-wise fine-tuning (LIFT)

3.4 Partial fine-tuning

3.4.1 Half fine-tuning (HFT)

3.4.2 Selective layer freezing

3.4.3 Freezing strategies

3.4.4 Implementation walkthrough

3.5 Parameter-efficient fine-tuning (PEFT)

3.5.1 LoRA (Low-Rank Adaptation)

3.5.2 QLoRA (Quantized LoRA)

3.5.3 DoRA (Weight-Decomposed LoRA)

3.5.4 NEFTune (Noisy Embeddings Improve Instruction Finetuning)

3.6 Unsupervised Fine-Tuning

3.7 References

3.8 Summary