Parameter-efficient fine-tuning
Parameter-efficient fine-tuning adapts a pretrained model by training a limited set of existing or added parameters while leaving most of the model unchanged.
Building Specialized AI Systems
A method for adapting a pretrained model by training substantially fewer parameters than full fine-tuning.
- How it is used
- Enables specialization of pretrained models for domain tasks when full fine-tuning is too resource-intensive.
- How deep it goes
- Explains frozen backbones, adapters, low-rank updates, target modules, and quantization; defers deeper method selection and training internals.
- Examples
- Uses finance and customer-support adapter swapping, credit-risk classification, financial summarization, and Rotten Tomatoes sentiment classification.
- What you can do after
- Apply it via PEFT or Unsloth to configure, train, save, and load LoRA adapters on frozen models.
Read how Building Specialized AI Systems explains parameter-efficient fine-tuning →
LLMs in Production
Parameter-efficient fine-tuning is a fine-tuning approach that adapts a model by training only a small subset of parameters or added parameters.
- How it is used
- Supports task- and domain-specific model adaptation under memory and compute constraints.
- How deep it goes
- Explains LoRA weight updates and rank trade-offs with a matrix-size example, while deferring a full backpropagation tutorial.
- Examples
- Uses a rank-2 matrix calculation and training listings for LoRA token classification and prompt tuning.
- What you can do after
- Apply it via the PEFT library to configure LoRA or prompt-tuning adaptations.
Read how LLMs in Production explains parameter-efficient fine-tuning →
The Art of AI Product Development
Parameter-efficient fine-tuning is a fine-tuning method that adapts a model by updating a small subset or additional parameterization rather than all model parameters.
- How it is used
- Uses PEFT to make repeated fine-tuning and client-specific model maintenance more economical.
- How deep it goes
- Defines PEFT through selective parameter updates and practical cost benefits, without explaining specific methods or implementation.
- Examples
- Uses a client-specific document classification scenario with a 92% pilot accuracy and a feedback loop for incorrect source selections.
- What you can do after
- Evaluate and understand PEFT as a cost and scalability option when selecting and optimizing language models.
Read how The Art of AI Product Development explains parameter-efficient fine-tuning →
LLM Customization and Fine-Tuning
Adapting a pretrained model by updating only a small number of added or selected parameters rather than all its weights.
- How it is used
- Bridges prompting and full fine-tuning when a task needs behavioral adaptation without updating every model weight.
- How deep it goes
- Explains PEFT families and LoRA’s low-rank matrices, then addresses memory, safety checks, and selection tradeoffs.
- Examples
- Uses a Qwen3-4B adapter run and a two-hospital clinical-note study to illustrate cost and task performance.
- What you can do after
- Evaluate adaptation options using task performance, GPU memory, data work, and safety checks.
Read how LLM Customization and Fine-Tuning explains parameter-efficient fine-tuning →
Domain-Specific Small Language Models
A family of fine-tuning methods that adapt a pretrained model using only a small subset of parameters or added modules to reduce compute and cost.
- How it is used
- Adapts pretrained models to task or domain data while reducing the compute cost of updating all parameters.
- How deep it goes
- Defines PEFT, surveys six methods, and walks through LoRA training; does not detail every method's implementation.
- Examples
- Demonstrates LoRA with FLAN-T5-small and SAMSum dialogue summarization, including 8-bit loading and model reuse.
- What you can do after
- Apply Hugging Face PEFT to configure, train, save, and reload a LoRA model for generation.
Read how Domain-Specific Small Language Models explains parameter-efficient fine-tuning →
Financial AI in Practice
Parameter-efficient fine-tuning adapts a pretrained model by training a small subset of its parameters or adding lightweight trainable components.
- How it is used
- Adapts pretrained models to a domain-specific style or behavior while leaving factual retrieval to RAG.
- How deep it goes
- Defines PEFT, compares LoRA, adapters, and prefix tuning, and sketches a LoRA workflow without detailing every intricacy.
- Examples
- Uses 200 paired community posts and analyst-written summaries to teach the AlphaStream tone.
- What you can do after
- Apply PEFT to customize a model’s style at lower cost than full fine-tuning, with validation and version-transition planning.
Read how Financial AI in Practice explains parameter-efficient fine-tuning →
Context Engineering
Parameter-efficient fine-tuning adapts a model by training a small number of parameters while keeping most base weights frozen.
- How it is used
- Positions PEFT as an enterprise model-customization strategy for context-specific LLM deployments.
- How deep it goes
- Defines PEFT and contrasts customization approaches, but does not provide implementation steps.
- Examples
- Names LoRA and adapter modules as common PEFT techniques.
- What you can do after
- Evaluate and understand PEFT's training-cost and version-management trade-offs against full fine-tuning.
Read how Context Engineering explains parameter-efficient fine-tuning →
Building Reliable AI Systems
Fine-tuning methods that adapt a small subset or representation of a model’s parameters to reduce training resource requirements.
- How it is used
- Uses parameter-efficient updates when prompting and retrieval alone do not meet specialized-task needs.
- How deep it goes
- Explains LoRA’s frozen base and trainable adapters, then compares LoRA and QLoRA by parameter efficiency, performance, and memory.
- Examples
- Compares full fine-tuning, LoRA, and QLoRA across parameter share, performance, memory, and hardware choice.
- What you can do after
- Apply LoRA or QLoRA to fine-tune large models with reduced training-resource demands.
Read how Building Reliable AI Systems explains parameter-efficient fine-tuning →
Also covered in
- Rearchitecting LLMsIn early access (MEAP)
- Generative AI in Action
- Deep Learning with R, Third Edition
- Transformers in Action
What to know first
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- foundation modelA foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- prompt engineeringPrompt engineering is the deliberate design and refinement of inputs, instructions, and context to guide an AI model’s behavior or outputs.
Related concepts
- low-rank adaptationincludes · Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning technique that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.
- fine-tuningkind of · Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- QLoRAincludes · QLoRA is a parameter-efficient fine-tuning method that combines a quantized pretrained model with trainable low-rank adapters.
- catastrophic forgettingaddresses · Catastrophic forgetting is the substantial loss of previously learned information or capabilities when a model learns new data or tasks.
- DoRAincludes · DoRA is a parameter-efficient fine-tuning method that decomposes model weights into separately tunable magnitude and directional components, typically applying low-rank adaptation to the directional component.
- NEFTuneincludes · NEFTune is a fine-tuning method that injects scaled random noise into token embeddings during training to improve generalization.
Questions about parameter-efficient fine-tuning
What is parameter-efficient fine-tuning?
Parameter-efficient fine-tuning adapts a pretrained model by training a limited set of existing or added parameters while leaving most of the model unchanged.
Which Manning books explain parameter-efficient fine-tuning?
Building Specialized AI Systems, LLMs in Production, The Art of AI Product Development, LLM Customization and Fine-Tuning, Domain-Specific Small Language Models, Financial AI in Practice, Context Engineering and Building Reliable AI Systems.
What should I know before learning parameter-efficient fine-tuning?
Manning books that teach parameter-efficient fine-tuning usually introduce fine-tuning, foundation model, language model and prompt engineering first.
What kinds of parameter-efficient fine-tuning are there?
Kinds of parameter-efficient fine-tuning include low-rank adaptation, QLoRA, DoRA and NEFTune.
What is parameter-efficient fine-tuning a kind of?
Parameter-efficient fine-tuning is a kind of fine-tuning. Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
What is parameter-efficient fine-tuning also called?
Parameter-efficient fine-tuning is also called PEFT.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.







