liveBook · Manning

Parameter-efficient fine-tuning

As explained in Context Engineering

Parameter-efficient fine-tuning adapts a model by training a small number of parameters while keeping most base weights frozen.

Parameter-efficient fine-tuning (PEFT) adapts a model by training a small number of additional or decomposed parameters while keeping most base weights frozen. It can reduce training cost and simplify version management compared with full fine-tuning. section 9.3.3

PEFT is one of several model-customization approaches:

  • LoRA and adapter modules are common PEFT techniques that add or decompose trainable parameters. section 9.3.3
  • Prompt learning trains continuous prompt, prefix, or soft-prompt parameters while keeping the main model frozen. section 9.3.3
  • Alignment optimization adjusts behavior using preference or policy signals, rather than defining customization by which parameters are trained. section 9.3.3

Fine-tuning for an enterprise use case can draw on domain data such as support tickets or internal task demonstrations. It also requires dataset governance, training infrastructure, validation, and rollback plans. section 9.3.3

How Context Engineering covers it

How deep it goes
Defines PEFT and contrasts customization approaches, but does not provide implementation steps.
Examples
Names LoRA and adapter modules as common PEFT techniques.
What you can do after
Evaluate and understand PEFT's training-cost and version-management trade-offs against full fine-tuning.
How it connects
Connects PEFT with full fine-tuning, prompt learning, and alignment optimization.

Also explained in

Compare how 8 books explain parameter-efficient fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.