Parameter-efficient fine-tuning

As explained in Domain-Specific Small Language Models

A family of fine-tuning methods that adapt a pretrained model using only a small subset of parameters or added modules to reduce compute and cost.

Parameter-efficient fine-tuning (PEFT) adapts a pretrained model by updating only a small fraction of its parameters or adding trainable modules. It reduces compute and cost while often preserving accuracy and robustness, making domain adaptation more practical than full fine-tuning. section 2.4 section 2.3

Common PEFT approaches include:

• LoRA freezes pretrained weights and trains low-rank updates for selected weight matrices. section 2.4
• Prefix tuning and prompt tuning optimize continuous task-specific vectors or prompt embeddings while keeping the model frozen. section 2.4
• AdaLoRA reallocates its parameter budget according to importance scores, while LoftQ combines quantization with low-rank initialization. section 2.4

Figure 2.3 The LoRA fine-tuning process

Use the process diagram to see how LoRA preserves pretrained weights while training a smaller adaptation component. section 2.4

LoRA domain adaptation

A practical workflow loads FLAN-T5-small in 8-bit mode, prepares it for k-bit training, targets the q and v modules with a LoRA configuration, trains on the SAMSum dialogue-summarization dataset, saves the adapter, and reloads it for generation. section 2.4

How Domain-Specific Small Language Models covers it

How deep it goes

Explains the PEFT family, compares major methods, and works through LoRA training, saving, reloading, and inference.

Examples

Uses FLAN-T5-small, the SAMSum dataset, 8-bit loading, selected q and v modules, adapter checkpoints, and generated summaries.

What it assumes

Assumes a pretrained model and task-specific data while demonstrating tokenization, preprocessing, quantization preparation, and trainer configuration.

What you can do after

Apply via Hugging Face Transformers and PEFT to train, save, reload, and evaluate a LoRA adapter.

How it connects

Contrasts with full fine-tuning and RAG, and connects LoRA with quantization, prompt tuning, prefix tuning, and vector databases.

liveBookmanning.com

Read Domain-Specific Small Language Models on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  LLMs in ProductionParameter-efficient fine-tuning is a fine-tuning approach that adapts a model by training only a small subset of parameters or added parameters.
• 

  liveBookmanning.com
  The Art of AI Product DevelopmentParameter-efficient fine-tuning is a fine-tuning method that adapts a model by updating a small subset or additional parameterization rather than all model parameters.
• 

  liveBookmanning.com
  Financial AI in PracticeParameter-efficient fine-tuning adapts a pretrained model by training a small subset of its parameters or adding lightweight trainable components.
• 

  liveBookmanning.com
  Building Reliable AI SystemsFine-tuning methods that adapt a small subset or representation of a model’s parameters to reduce training resource requirements.

Compare how 5 books explain parameter-efficient fine-tuning →

Related concepts

• low-rank adaptationincludes · Low-rank adaptation is a parameter-efficient fine-tuning method that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.
• fine-tuningkind of · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
• QLoRAincludes · QLoRA is a parameter-efficient fine-tuning method that quantizes a pretrained model’s weights while training low-rank adapters to reduce memory requirements.
• catastrophic forgettingaddresses · Catastrophic forgetting is the substantial loss of previously learned information or capabilities when a model learns new data, tasks, or objectives.

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.