liveBook · Manning

What you will learn in LLM Customization and Fine-Tuning

Adaptation, distillation, and alignment

by Amit Bahree and Weehyong Tok · Manning Early Access Program (MEAP)

Chapter by chapter

Chapter 1: Why model adaptation?

The introduction explains why general-purpose LLMs can fail at enterprise tasks and defines model adaptation as a range of ways to close that gap.

1.1 The adaptation problem · 1.2 Scenarios where adaptation pays off · 1.2.1 Scenario 1: Industry-specific models · 1.2.2 Scenario 2: Regional and cultural adaptation · 1.2.3 Scenario 3: Enterprise tone and brand · 1.3 The numbers: ROI in real terms · 1.4 Enterprise pain points · 1.5 When NOT to adapt · 1.6 How model adaptation works · 1.6.1 The adaptation lifecycle · 1.6.2 The continuum of techniques · 1.6.3 What the gap actually looks like · 1.7 Choosing a starting technique · 1.8 Considerations: Safety and adaptation risk · 1.9 What you’ll need to follow along · 1.9.1 The base model: Qwen3-4B-Instruct-2507 · 1.9.2 The starter dataset: IT support Q&A from Stack Exchange and Dolly · 1.9.3 Tooling and hardware · 1.9.4 The code repository · 1.10 How this book teaches model adaptation · 1.11 Summary · 1.12 References

What you learn in chapter 1 →

Chapter 2: How to do model adaptation

The chapter introduction uses an earnings-transcript failure to show why adaptation methods must be chosen for the task rather than by how intensive they are.

2.1 The adaptation continuum at decision depth · 2.2 Hybrid approaches: Most production systems stack rungs · 2.3 Matching technique to use case · 2.4 Buy versus build · 2.4.1 Buy: Start with a managed API · 2.4.2 Build: Self-host · 2.5 Infrastructure options · 2.6 Tools and frameworks · 2.7 Selecting a base model · 2.7.1 Open-weight base models for fine-tuning · 2.7.2 Frontier models for reasoning tasks · 2.8 A quickstart preview: The shape of a fine-tune in five steps · 2.9 Security across the adaptation continuum · 2.9.1 How each technique changes the attack surface · 2.9.2 Mitigating the risks · 2.10 Lightweight operationalization · 2.11 Summary · 2.12 References

What you learn in chapter 2 →

Chapter 3: What data do I need for model adaptation?

This chapter introduction explains why having enterprise data is not the same as having data suitable for model adaptation, using IT support records to show why quality and representativeness matter.

3.1 Data readiness spectrum · 3.2 Why data quality is important · 3.3 How much data do you need? · 3.3.1 Base models versus instruct models · 3.3.2 How accuracy responds to data volume · 3.3.3 Does your training data reflect your production distribution? · 3.3.4 Edge cases matter disproportionately · 3.3.5 Diversity across user types, conditions, and variations · 3.3.6 Diminishing returns: Why the first 100 examples do most of the work · 3.4 Data acquisition strategies · 3.4.1 Human annotation · 3.4.2 Weak supervision · 3.4.3 Synthetic data from teacher models · 3.4.4 Production log mining · 3.5 Data versioning and experiment tracking · 3.5.1 Experiment logging · 3.5.2 Choosing a tracking tool · 3.5.3 Data lineage · 3.6 Generating synthetic training data · 3.6.1 Step 1: Load seed examples from Hugging Face · 3.6.2 Step 2: Build the prompt with style anchoring · 3.6.3 Step 3: Calling the teacher model to generate the candidates (one call per category) · 3.6.4 Step 4: Quality gate · 3.6.5 Step 5: Distribution alignment check · 3.6.6 Step 6: Mix with real data and save with a version manifest · 3.7 Golden test sets and held-out evaluation · 3.7.1 What goes in a golden test set · 3.7.2 Building your first golden test set · 3.7.3 The red team companion set · 3.7.4 Common mistakes · 3.7.5 Where the golden set fits in the adaptation lifecycle · 3.8 Removing personal data before training · 3.8.1 Data retention and the right to be forgotten · 3.8.2 Encryption in transit and at rest · 3.9 Summary · 3.10 References

What you learn in chapter 3 →

Chapter 4: In-context learning, few-shot, and RAG

The chapter opens by using a ticket-routing example to explain in-context learning and why a few-shot baseline is worth testing before fine-tuning.

4.1 The case for in-context learning · 4.1.1 Evolution of the approach · 4.2 Few-shot prompting in practice · 4.2.1 Building the prompt · 4.2.2 Selection rules: Diverse, representative, no cherry-picking · 4.2.3 Formatting: Consistent input/output schema · 4.2.4 Ordering: The recency effect · 4.2.5 Example count vs. accuracy: The empirical curve · 4.2.6 A practical iteration cycle · 4.3 Many-shot prompting and long-context windows · 4.3.1 When many-shot wins · 4.3.2 The cost trade-off · 4.3.3 When many-shot does not help · 4.3.4 Lost in the middle: The long-context attention U-shape · 4.3.5 Building a many-shot prompt · 4.4 Designing prompts for production · 4.4.1 A typical validation cycle · 4.4.2 The held-out test set · 4.4.3 Run-to-run variability · 4.4.4 LLM judging for prompt quality · 4.4.5 Brittleness signals and what to do about them · 4.4.6 When validation says stop tuning the prompt · 4.5 Use RAG when knowledge gaps appear · 4.5.1 Knowledge gaps versus behavior gaps · 4.5.2 Minimal RAG architecture · 4.5.3 Vector embeddings, vector indexes, and vector databases · 4.5.4 A 50-line RAG implementation · 4.5.5 Beyond the minimal pipeline · 4.5.6 Common RAG failure modes · 4.5.7 Measuring RAG quality before debugging it · 4.5.8 Evaluating RAG end-to-end · 4.5.9 When RAG is not the answer · 4.6 Limits of in-context approaches · 4.6.1 Diminishing returns on example count · 4.7 Cost and decision economics · 4.7.1 The cost components · 4.7.2 The break-even formula · 4.7.3 A worked example at four volumes · 4.7.4 When the math is not the only input · 4.8 Managed platforms for prompt engineering and RAG · 4.8.1 What the platform layer does not solve · 4.9 When to graduate to parameter-efficient methods · 4.9.1 Diagnostic signals that say graduate now · 4.9.2 The decision tree · 4.9.3 What changes operationally when you graduate · 4.10 Summary · 4.11 References

What you learn in chapter 4 →

Chapter 5: Parameter-Efficient Fine-Tuning: LoRA and QLoRA

Introduces PEFT as a lower-cost way to adapt a frozen model, explains LoRA’s adapter-based approach, and identifies when PEFT is useful.

5.1 Hands-on: Your first LoRA fine-tuning · 5.1.1 Step 1: Prepare data · 5.1.2 Step 2: Train LoRA adapter · 5.1.3 Step 3: Evaluate LoRA · 5.1.4 Step 4: Understanding the evaluation results · 5.1.5 Step 5: Inference · 5.1.6 Step 6: QLoRA · 5.1.7 Data quality iterations · 5.1.8 Where an adapter beats prompting · 5.2 How LoRA works · 5.2.1 LoRA core idea · 5.2.2 Why low rank? · 5.2.3 LoRA versus full fine-tuning: When to use each · 5.2.4 When LoRA hits its limits · 5.3 Choosing LoRA hyperparameters · 5.4 Safety and regression testing · 5.4.1 Safety regression testing in practice · 5.5 QLoRA: Fine-tuning under memory constraints · 5.6 Multi-LoRA: One base, many adapters · 5.6.1 Choosing the right adapter at inference time · 5.7 Assessing value and ROI · 5.7.1 Lowe’s: Catalog intelligence at scale · 5.7.2 Finance: Dragon LLM domain models · 5.7.3 Calculating your own ROI · 5.8 LoRA training frameworks · 5.8.1 Choosing a framework · 5.9 Multi-LoRA inference · 5.9.1 Specialized multi-LoRA frameworks · 5.10 Common pitfalls and how to avoid them · 5.11 Responsible AI and evaluation · 5.12 Managed fine-tuning services · 5.12.1 Enterprise cloud platforms · 5.12.2 Specialized inference platforms · 5.12.3 Developer-focused platforms · 5.12.4 Decision framework · 5.13 Summary · 5.14 References

What you learn in chapter 5 →

Chapter 6: Supervised fine-tuning: Maximum expressiveness

Introduces full supervised fine-tuning as a more expressive but costlier option when a LoRA adapter falls short of a measured quality target.

6.1 Hands-on: SFT training workflow · 6.1.1 Data preparation · 6.1.2 Model loading and training configuration · 6.1.3 Training script · 6.1.4 Inference · 6.2 Monitoring training · 6.2.1 Tools for monitoring training · 6.2.2 Reading the loss curves · 6.2.3 Red flags and concrete thresholds · 6.3 Evaluating the fine-tuned model · 6.3.1 Evaluation strategy · 6.3.2 Hands-on evaluation code · 6.3.3 Making decisions from evaluation results · 6.4 When SFT delivers value · 6.4.1 Realistic expectations · 6.5 Fine-tuning refines, doesn’t inject knowledge · 6.6 SFT vs. LoRA: Realistic performance expectations · 6.6.1 The cost-performance tradeoff · 6.6.2 The forgetting risk (catastrophic forgetting) · 6.7 Data requirements · 6.7.1 Quality over quantity · 6.8 Common pitfalls and how to avoid them · 6.8.1 Avoiding catastrophic forgetting · 6.8.2 Overfitting · 6.8.3 Distribution mismatch · 6.8.4 Hyperparameter misconfiguration · 6.9 Hyperparameter iteration strategy · 6.10 Transitioning from SFT to preference optimization (DPO and RLHF) · 6.10.1 The two-stage pattern · 6.10.2 Preparing your SFT checkpoint for preference optimization · 6.11 Platform-specific implementation · 6.11.1 OpenAI and Azure OpenAI · 6.11.2 Google Vertex AI · 6.11.3 Hugging Face self-hosted · 6.11.4 Cost and privacy considerations · 6.12 Real case study: Healthcare clinical Q&A · 6.12.1 Dataset and preparation · 6.12.2 Training configuration · 6.12.3 Results · 6.12.4 Cost breakdown · 6.12.5 ROI calculation · 6.12.6 Lessons learned · 6.13 When SFT is worth it: Decision framework · 6.13.1 Maintenance considerations · 6.14 Catching regressions before deployment · 6.14.1 Safety regression testing · 6.14.2 Targeted validation set methodology · 6.14.3 Behavioral test suites · 6.14.4 Early stopping and diverse data mixing · 6.15 Summary · 6.16 References

What you learn in chapter 6 →

Chapter 7: Knowledge distillation: Capturing frontier model intelligence

Introduces knowledge distillation as training a smaller student on a stronger teacher’s outputs, explains its enterprise appeal, and flags the need to preserve specialized behavior.

7.1 Hands-on: The distillation recipe · 7.1.1 Stage 1: Task identification and prompt design · 7.1.2 Stage 2: Teacher data generation · 7.1.3 Stage 3: Quality verification · 7.1.4 Stage 4: Student training · 7.1.5 Student inference · 7.1.6 Testing the quality filter · 7.2 Evaluating distillation quality · 7.2.1 Three-way comparison · 7.2.2 Interpreting the results · 7.2.3 Beyond Token-F1: Using an LLM judge for distillation quality · 7.2.4 Scaling behavior · 7.2.5 Error analysis · 7.3 Why distillation matters · 7.3.1 What distillation can and cannot do · 7.4 Distillation intuition · 7.4.1 The teacher-student framework · 7.4.2 Distillation vs. fine-tuning on the same data · 7.4.3 What the student actually learns · 7.4.4 White-box vs. black-box distillation · 7.4.5 The DeepSeek-R1 distillation results · 7.5 Reverse KL divergence for selective knowledge transfer · 7.5.1 Forward vs. reverse KL · 7.5.2 Recent findings · 7.5.3 Summary of divergence approaches · 7.5.4 Where this matters in practice · 7.6 When distillation is worth it (and when it isn’t) · 7.6.1 Decision framework · 7.6.2 Quality targets · 7.6.3 Cost model · 7.6.4 The middle ground: Hybrid deployment · 7.6.5 When to skip distillation · 7.6.6 Planning a distillation project · 7.6.7 Distillation and regulatory compliance · 7.7 Practical lessons from the pipeline · 7.7.1 Qualitative comparison · 7.7.2 What worked and what did not · 7.7.3 Extrapolating to frontier API teachers · 7.8 Student model robustness checks · 7.8.1 Why safety does not transfer · 7.8.2 Robustness check implementation · 7.8.3 Responding to safety regressions · 7.9 Summary · 7.10 References

What you learn in chapter 7 →

Chapter 8: Preference optimization: Teaching your model to judge

This introduction explains why preference optimization follows supervised fine-tuning when a model needs to distinguish between acceptable responses of different quality.

8.1 When fine-tuning plateaus · 8.2 Direct preference optimization (DPO): Deep dive · 8.2.1 Building preference data · 8.2.2 Preference data sources: A taxonomy · 8.2.3 Training with DPOTrainer · 8.2.4 Interpreting DPO training metrics · 8.2.5 How DPO works · 8.2.6 End-to-end pipeline summary · 8.2.7 Evaluating DPO: Three-way comparison · 8.2.8 Where DPO wins: A conciseness demonstration · 8.2.9 Go/no-go criteria for shipping a DPO model · 8.2.10 Monitoring preference-tuned models in production · 8.2.11 DPO variants: When to consider alternatives · 8.3 GRPO: Group relative policy optimization · 8.3.1 How GRPO works · 8.3.2 When GRPO excels · 8.3.3 GRPO in practice · 8.4 RLHF: Why most enterprises skip it · 8.4.1 The RLHF pipeline · 8.4.2 Why enterprises skip it · 8.4.3 When RLHF is unavoidable · 8.5 Reinforcement fine-tuning (RFT) with verifiable rewards · 8.5.1 Enterprise applications of RFT · 8.5.2 How RFT works · 8.5.3 The 2025 RLVR findings · 8.5.4 Key challenge: Reward hacking · 8.5.5 Cost and applicability · 8.6 Cold-start SFT + RL: The DeepSeek-R1 pattern · 8.6.1 The two-stage pattern · 8.6.2 Emergent behaviors · 8.7 Decision framework: Which technique? · 8.8 Honest assessment: Failure modes and practical limits · 8.8.1 The 95/4/1 breakdown · 8.8.2 Known DPO failure modes · 8.8.3 Common mistakes · 8.9 Preference optimization and safety · 8.9.1 Dual-objective optimization · 8.10 Summary · 8.11 References

What you learn in chapter 8 →

Chapter 9: Managing model evolution, drift, and versioning

This introduction explains why a deployed IT support model needs ongoing monitoring and maintenance even when its own weights have not changed.

9.1 Model and data versioning · 9.1.1 What to version · 9.1.2 Our approach: A single JSON file · 9.1.3 Running the registry · 9.2 Model drift: Understanding and responding · 9.2.1 Detection approaches · 9.2.2 Embedding-based drift detection · 9.2.3 Running the detector · 9.2.4 Canary prompts · 9.3 Handling failure and rollback · 9.3.1 Rollback timeline · 9.3.2 Running the rollback demo · 9.3.3 Blue-green deployment for LLMs · 9.4 Security and abuse monitoring · 9.4.1 The red-team prompt suite · 9.4.2 Running the safety monitor · 9.4.3 When to retrain for safety · 9.4.4 Production prompt monitoring · 9.5 Base model evolution · 9.5.1 When to upgrade (green lights) · 9.5.2 When to stay (red lights) · 9.5.3 The hidden cost of upgrading · 9.5.4 Version locking · 9.6 Long-term maintenance practices · 9.6.1 Common pitfalls · 9.6.2 Maintenance cadence · 9.6.3 Monthly activities in detail · 9.6.4 Quarterly activities in detail · 9.6.5 Annual activities · 9.6.6 Managed platform options for monitoring and ops · 9.7 Retraining triggers and validation gates · 9.7.1 Retrain triggers · 9.7.2 Incremental retraining from checkpoint · 9.7.3 Gating retrains · 9.8 Emerging base models and strategic planning · 9.8.1 Planning horizons · 9.8.2 Tracking the field · 9.9 End-to-end case study: One year of an IT support model · 9.9.1 When to wait, when to act · 9.9.2 The rollback that proved the playbook · 9.10 Closing the loop · 9.11 Summary · 9.12 References

What you learn in chapter 9 →

Concepts explained in this book

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.