liveBook · Manning

Chapter 5: Parameter-Efficient Fine-Tuning: LoRA and QLoRA

What you will learn in chapter 5 of LLM Customization and Fine-Tuning

Introduces PEFT as a lower-cost way to adapt a frozen model, explains LoRA’s adapter-based approach, and identifies when PEFT is useful.

After this chapter you will be able to

What it builds on

Section by section

Introduction

Introduces PEFT as a lower-cost way to adapt a frozen model, explains LoRA’s adapter-based approach, and identifies when PEFT is useful.

Key terms: parameter-efficient fine-tuning (PEFT), task-specific parameters, LoRA, pretrained model, adapter, inference, PEFT, prompting, QLoRA, quantization

5.1 Hands-on: Your first LoRA fine-tuning

This section introduces a single-GPU LoRA fine-tuning workflow and explains how SFTTrainer simplifies its training setup.

Key terms: evaluation step, safety checkpoint, data iteration, TRL, SFTTrainer, LoRA config, message format, prompt masking, completions-only training

5.1.1 Step 1: Prepare data

This section prepares a curated IT support dataset for LoRA training by building reproducible data splits and converting examples into chat messages.

Key terms: IT support dataset, house format, quality filters, build script, fixed seed, train, validation, and test splits, messages format, system, user, assistant, context field, SFTTrainer

5.1.2 Step 2: Train LoRA adapter

This section shows how to train a LoRA adapter, manage GPU memory, interpret training and validation loss, and use the saved adapter files.

Key terms: SFTTrainer, PEFT LoRA, target_modules, adapter parameters, rank, alpha, gradient checkpointing, activations, forward pass, backward pass

5.1.3 Step 3: Evaluate LoRA

This section shows how to compare a LoRA adapter with its base model on held-out IT-support examples and safety prompts, then interpret the resulting report cautiously.

Key terms: base model, adapter, held-out test split, deltas, report.json, report.md, exact match, Token-F1, reference answer, positive deltas

5.1.4 Step 4: Understanding the evaluation results

This section explains how to interpret a LoRA adapter’s task and safety evaluation results, including why flat overlap scores can hide useful changes and why a safety regression requires a data fix rather than a rank change.

Key terms: Exact Match (EM), Token-F1, Safety Refusal Rate, reference answer, IT support answer format, safety regression, explicit refusal examples, base model, LoRA rank, r=8

5.1.5 Step 5: Inference

This section shows how to generate with a trained LoRA adapter, compare its output with the base model, and choose whether to keep the adapter separate or merge it for deployment.

Key terms: LoRA weights, PeftModel.from_pretrained(), adapter_model.safetensors, chat template, --adapter, base model, training data, token limit, keep the adapter separate, merge adapter into base

5.1.6 Step 6: QLoRA

This section shows how QLoRA changes a LoRA fine-tune by loading the base model in 4-bit precision, then explains the evaluation, inference, and hardware trade-offs.

Key terms: QLoRA, 4-bit quantization, bitsandbytes, NormalFloat4 (nf4), prepare_model_for_kbit_training, LoRA adapter, --adapter, --quantized_4bit, QLoRA adapter, regular LoRA

5.1.7 Data quality iterations

This section explains how to curate and continually improve LoRA training data so an adapter learns reliable, appropriate responses.

Key terms: data quality, data curation, iterative improvement, balanced outcomes, response type distribution, safety refusals, complete context, clarification, hallucinations, inter-annotator agreement

5.1.8 Where an adapter beats prompting

This section uses a Contoso IT support comparison to show when a LoRA adapter adds value beyond a format prompt, while cautioning that the results are limited by a small test set.

Key terms: system prompt, house style, internal vocabulary, LoRA adapter, base model, format prompt, internal tool named, house format, Token-F1, golden set

5.2 How LoRA works

This section explains how LoRA adapts a frozen model with small trainable matrices, why that can be cheaper than full fine-tuning, and where its limits lie.

Key terms: LoRA, low-rank matrices, adapter modules, frozen weights, PEFT, prompting, full fine-tuning, consumer-grade GPUs, LoRA adapters, base model

5.2.1 LoRA core idea

This section explains how LoRA adapts a model by learning a removable low-rank update while leaving its original weights unchanged.

Key terms: LoRA adapter, base weight matrix W, low-rank update ΔW, effective weights

5.2.2 Why low rank?

This section explains why low-rank adapters can make fine-tuning cheaper, where their performance and inference trade-offs lie, and how instruct, chat, and reasoning models differ.

Key terms: LoRA, low-rank matrices, adapter weights, gradients, inference latency, modularity, adapters, base model, limited capacity, plateau

5.2.3 LoRA versus full fine-tuning: When to use each

This section recommends starting enterprise adaptation with LoRA and explains when its limits justify moving to full supervised fine-tuning.

Key terms: LoRA, enterprise adaptation, full supervised fine-tuning (SFT), maximum expressiveness, LoRA performance, architectural constraints, curated examples, full fine-tuning, LoRA’s capacity

5.2.4 When LoRA hits its limits

This section explains three limits of LoRA—fundamental behavior change, vocabulary mismatch, and architectural change—and when to choose another approach.

Key terms: low-rank constraint, core model patterns, full supervised fine-tuning, tokenizer, embedding layer, vocabulary expansion, RAG, architectural limitations, context length, RoPE

5.3 Choosing LoRA hyperparameters

Explains how to choose and lightly tune LoRA rank, scaling factor, learning rate, and epochs while balancing validation performance, resource cost, and safety.

Key terms: rank (r), expressivity, validation performance, rank, refusal rate, safety behavior, regression gate, scaling factor (α), α=2r, under-correcting

5.4 Safety and regression testing

This section explains how a LoRA adapter can weaken safety guardrails and why every adapter run needs a quick safety check before deployment.

Key terms: LoRA adapter, safety guardrails, safety regression, safety regression testing, jailbreak attempts, safety scenarios, build verification tests (BVTs)

5.4.1 Safety regression testing in practice

Shows how to detect safety regressions in a LoRA-adapted model and respond when they occur.

Key terms: safety regressions, base, LoRA-adapted models, safe completion, controlled environment, polite refusals, LoRA learning rate, base data, independent backup filter

5.5 QLoRA: Fine-tuning under memory constraints

This section explains how QLoRA reduces fine-tuning memory needs, what makes its 4-bit approach practical, and how to interpret its training, serving, and evaluation trade-offs.

Key terms: QLoRA, 4-bit quantization, base model, LoRA adapter, NF4, NormalFloat4, double quantization, scale and offset values, LoRA, VRAM

5.6 Multi-LoRA: One base, many adapters

This section explains how multiple task-specific LoRA adapters can share one base model, reducing storage and allowing independent updates.

Key terms: Multi-LoRA, base model, LoRA adapters, inference time, modular adapters, single endpoint, adapter composition, merging, domain-specific tuning

5.6.1 Choosing the right adapter at inference time

This section compares ways to route requests to LoRA adapters and shows why multiple adapters on one base model are useful in deployment.

Key terms: explicit routing, keyword routing, classifier routing, zero-shot model, base model, adapters, base weights, runtime

5.7 Assessing value and ROI

This section explains how PEFT’s lower training costs and faster iteration affect the value and return on investment of custom models.

Key terms: PEFT, LoRA, adapters, retraining cycle, adapter, API spend, ROI, data collection, annotation costs, GPU time

5.7.1 Lowe’s: Catalog intelligence at scale

The Lowe’s case shows how fine-tuning on catalog-specific data improved product tagging and error detection, while distinguishing those measured gains from an unreported financial payback.

Key terms: taxonomy tagging, category hierarchies, attribute-value patterns, product tagging accuracy, error detection, conversion, catalog QA effort, payback period

5.7.2 Finance: Dragon LLM domain models

The section uses finance-domain models to show how organizations can adapt a shared base with LoRA instead of building separate models from scratch.

Key terms: financial corpora, regulatory type, financial terminology, finance-optimized base, LoRA, institution-specific policies, two-stage approach, training from scratch, pretraining, full fine-tuning

5.7.3 Calculating your own ROI

This section gives a framework for estimating a LoRA project's return on investment and comparing its payback time with full fine-tuning.

Key terms: Monthly ROI, GPU compute, evaluation and iteration cycles, operational efficiency, break-even point, ROI Time, LoRA adapters, full fine-tuning

5.8 LoRA training frameworks

This section compares five PEFT training frameworks by their workflows, capabilities, and practical strengths.

Key terms: Hugging Face PEFT, Torchtune, Axolotl, YAML, distributed LoRA fine-tuning, LLaMA-Factory, Unsloth, experiment tracking, memory efficiency, RL workflows

5.8.1 Choosing a framework

This subsection matches LoRA training frameworks to workflow needs and explains the chapter’s choice of tools.

Key terms: Transformers+PEFT, TorchTune, Axolotl, LLaMA-Factory, Unsloth, open-source tools, managed services, Hugging Face ecosystem, Transformers, PEFT

5.9 Multi-LoRA inference

This section explains why serving many LoRA adapters can be costly and how multi-LoRA inference frameworks address that cost.

Key terms: multi-LoRA inference frameworks, base model, LoRA adapters, batching

5.9.1 Specialized multi-LoRA frameworks

This section compares LoRAX and vLLM Multi-LoRA as frameworks for serving multiple LoRA adapters.

Key terms: LoRAX, vLLM Multi-LoRA, multi-LoRA inference, OpenAI-compatible API, tenant isolation

5.10 Common pitfalls and how to avoid them

This section explains common LoRA project pitfalls and how to prevent or detect them through appropriate base-model choice, sound evaluation, recordkeeping, and ongoing checks.

Key terms: validation set, success criteria, hyperparameters, data versions, reproducibility, domain mismatch, domain-appropriate base model, retrieval (RAG), data contamination, overfitting to quirks

5.11 Responsible AI and evaluation

The section explains how to make privacy, fairness, safety, evaluation, and ongoing governance part of fine-tuning rather than treating them as checks after deployment.

Key terms: Responsible AI, governance, auditability, model cards, ongoing monitoring, personally identifiable information, anonymized, GDPR, HIPAA, data residency

5.12 Managed fine-tuning services

The section introduces managed fine-tuning services as an alternative to running training infrastructure yourself.

Key terms: managed fine-tuning services, control, scale, governance

5.12.1 Enterprise cloud platforms

This section compares enterprise-managed fine-tuning platforms and describes the simpler Azure OpenAI fine-tuning API workflow.

Key terms: Azure Foundry, Vertex AI, SageMaker, Mosaic AI, LoRA, QLoRA, Azure OpenAI Fine-Tuning API, LoRA-style fine-tuning, prompt-completion examples, JSONL

5.12.2 Specialized inference platforms

5.12.3 Developer-focused platforms

This subsection surveys developer-focused platforms for rapid prototyping and smaller-scale LoRA production, highlighting how their offerings differ.

Key terms: Replicate, Modal, Hugging Face AutoTrain, LoRA, serverless GPU-accelerated functions, no-code

5.12.4 Decision framework

This section frames the choice between managed and open-source fine-tuning around practical trade-offs, then identifies LoRA compatibility and deployment planning checks.

Key terms: managed services, open-source tools, vendor lock-in, control, cost, compliance, base model, platform, LoRA adapters, deploy

5.13 Summary

5.14 References

This section lists the sources cited in the chapter without explaining new concepts or procedures.

Where this is used later

Concepts explained in this chapter

← Chapter 4: In-context learning, few-shot, and RAG Chapter 6: Supervised fine-tuning: Maximum expressiveness →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.