Chapter 2: Tuning for a specific domain
This chapter introduction previews customizing open source foundation models with your own data and notes that most of the book focuses on inference rather than tuning.
After this chapter you will be able to
- Training Dataset Formatting. State that fine-tuning a Transformer for a particular task starts with formatting the dataset for training.
- Labeled classification samples. Describe how to assemble the starting data for the BERT classification example: each text sample needs a class label, and the samples are formatted as a list of tuples.
- BERT-compatible text tokenization. Explain why the training text is tokenized with the bert-base-uncased tokenizer and describe the input formatting taught here: [CLS] marks the beginning, [SEP] separates segments, and padding or truncation keeps sequences a consistent length.
- Encoded training and validation data. Trace the remaining preparation steps: convert formatted tokens and labels to numerical IDs, split the dataset into training and validation sets, and use a custom dataset class to create their data loaders.
- Context and completion pairs. Explain how to organize private text samples for GPT-2 text completion: each context or prompt is paired with the target text the model should generate.
- GPT-2 sequence formatting. Describe how the paired text is prepared with the GPT-2 tokenizer: add beginning, end, and separator tokens; pad or truncate to a consistent length; and represent tokens as IDs.
- Embeddings and FAISS indexing. Explain how the example prepares a text corpus for retrieval: generate embeddings for its text rows, normalize them, and add them to a FAISS index. Describe why the index is used instead of brute-force search.
- Nearest-neighbor search results. Trace how the example embeds and normalizes a new search text, searches the FAISS index, and relates the returned distances and positions to the original labeled samples. Explain what a smaller L2 distance means in the Spider-Man example.
- RAG retrieval and generation. Trace how RAG uses a user’s prompt to retrieve relevant information and supply it as context to a pretrained LLM for a natural-language response.
- Limits of prompt engineering. Explain why domain-specific tasks or certain unstructured text may call for RAG or fine-tuning despite the speed and potential cost advantage of prompt engineering, and state how the choice between those alternatives is framed here.
- Full fine-tuning purpose. Explain that fine-tuning adapts a pretrained LLM to a task by further training it on a smaller, task-specific labeled dataset, and that full fine-tuning updates all its parameters.
- Question-answering data preparation. Describe the question, context, and answers fields required for the example, and how preprocessing turns answer locations into start and end token positions.
- DistilBERT training workflow. Trace how the example splits and preprocesses a dataset, creates batches, and uses a Transformers Trainer to train DistilBERT for question answering.
- LoRA parameter efficiency. Explain how LoRA fits within parameter-efficient fine-tuning and why it is less computationally demanding than full fine-tuning.
- LoRA summarization training workflow. Trace the main preparation and training steps in the section’s FLAN-T5 summarization example.
- RAG suitability and limits. Explain when RAG is a good choice and why its lower training cost does not make it suitable for every task.
- Fine-tuning suitability and costs. Explain why fine-tuning can be preferable for complex domain tasks, while accounting for its training costs and expertise requirements.
What it builds on
- The Transformer architecture (chapter 1) · Assumes familiarity with pretrained Transformer models.
Section by section
Introduction
This chapter introduction previews customizing open source foundation models with your own data and notes that most of the book focuses on inference rather than tuning.
2.1 Data preparation
This section establishes that task-specific Transformer fine-tuning begins by formatting the training dataset, then previews the data-preparation examples that follow.
- Training Dataset Formatting: State that fine-tuning a Transformer for a particular task starts with formatting the dataset for training.
Why it matters: It identifies what must happen to the dataset before task-specific fine-tuning begins.
Key terms: fine-tuning, Transformer model, dataset for training
2.1.1 Data preparation for BERT fine-tuning
This section prepares labeled text for BERT classification fine-tuning by tokenizing and encoding it, then organizing it into training and validation data loaders.
- Labeled classification samples: Describe how to assemble the starting data for the BERT classification example: each text sample needs a class label, and the samples are formatted as a list of tuples.
Why it matters: The labels identify the classes the model is being prepared to predict. - BERT-compatible text tokenization: Explain why the training text is tokenized with the bert-base-uncased tokenizer and describe the input formatting taught here: [CLS] marks the beginning, [SEP] separates segments, and padding or truncation keeps sequences a consistent length.
Why it matters: Model-compatible formatting lets the prepared text be used as BERT input. - Token count consequences: Explain what a token can represent and why tokenization and token count matter for input size, processing time, memory use, and the maximum prompt length.
Why it matters: Token count helps explain why input length must be managed during preparation. - Encoded training and validation data: Trace the remaining preparation steps: convert formatted tokens and labels to numerical IDs, split the dataset into training and validation sets, and use a custom dataset class to create their data loaders.
Why it matters: These steps turn formatted examples into the numerical, organized data needed before model training.
Key terms: classification task, class label, list of tuples, bert-base-uncased tokenizer, tokens, [CLS], [SEP], padding, truncating, token
2.1.2 Data preparation for GPT fine-tuning
This section prepares context–completion pairs for GPT-2 fine-tuning by tokenizing and formatting them, then placing them in a custom training data loader.
- Context and completion pairs: Explain how to organize private text samples for GPT-2 text completion: each context or prompt is paired with the target text the model should generate.
Why it matters: The pairing defines what the fine-tuned model is being trained to complete. - GPT-2 sequence formatting: Describe how the paired text is prepared with the GPT-2 tokenizer: add beginning, end, and separator tokens; pad or truncate to a consistent length; and represent tokens as IDs.
Why it matters: These steps turn paired text into consistently formatted numerical sequences for the model. - Custom training data loader: Explain why the prepared examples are put in a training data loader, and identify the custom dataset class as the means used here, with its initialization and item-access methods reflecting the model’s input configuration.
Why it matters: The loader makes the prepared custom dataset available for training in the model’s expected input configuration.
Key terms: context, prompt, target, completion, GPT-2 tokenizer, special tokens, separator token, padding, truncating, token IDs
2.1.3 Data preparation for RAG
This section prepares text for RAG by embedding and indexing it with FAISS, then demonstrates how to find and interpret similar text.
- Embeddings and FAISS indexing: Explain how the example prepares a text corpus for retrieval: generate embeddings for its text rows, normalize them, and add them to a FAISS index. Describe why the index is used instead of brute-force search.
Why it matters: The preparation step makes the corpus available for efficient similarity-based retrieval. - Nearest-neighbor search results: Trace how the example embeds and normalizes a new search text, searches the FAISS index, and relates the returned distances and positions to the original labeled samples. Explain what a smaller L2 distance means in the Spider-Man example.
Why it matters: Reading the distances alongside the original labels turns a vector search into an interpretable result.
Key terms: RAG, embeddings, sentence_transformers, FAISS index, normalize, query embedding, Euclidean (L2) distance, k, distances, ann
2.2 Retrieval-augmented generation
This section explains how RAG supplies retrieved information to an LLM and when to consider it instead of prompt engineering.
- RAG retrieval and generation: Trace how RAG uses a user’s prompt to retrieve relevant information and supply it as context to a pretrained LLM for a natural-language response.
Why it matters: This explains how external information can help an LLM give more accurate, up-to-date answers. - Limits of prompt engineering: Explain why domain-specific tasks or certain unstructured text may call for RAG or fine-tuning despite the speed and potential cost advantage of prompt engineering, and state how the choice between those alternatives is framed here.
Why it matters: It helps you recognize when a quick prompting approach may be insufficient without assuming one alternative always wins.
Key terms: retrieval-augmented generation, retrieval phase, content generation phase, external knowledge base, embeddings, prompt engineering, few-shot, chain-of-thought, RAG, fine-tuning
2.3 Fine-tuning
This section walks through full fine-tuning of DistilBERT for question answering, from preparing a labeled dataset to training and extracting an answer.
- Full fine-tuning purpose: Explain that fine-tuning adapts a pretrained LLM to a task by further training it on a smaller, task-specific labeled dataset, and that full fine-tuning updates all its parameters.
Why it matters: This distinguishes the workflow here from adapting a model by training only a small number of parameters. - Question-answering data preparation: Describe the question, context, and answers fields required for the example, and how preprocessing turns answer locations into start and end token positions.
Why it matters: The model needs labeled answer positions in tokenized context to train for this question-answering task. - DistilBERT training workflow: Trace how the example splits and preprocesses a dataset, creates batches, and uses a Transformers Trainer to train DistilBERT for question answering.
Why it matters: These are the main steps connecting prepared examples to an actual fine-tuning run. - Answer extraction after training: Trace how the fine-tuned model answers a question using a supplied context, from tokenizing the inputs through decoding predicted answer tokens.
Why it matters: It shows how the trained question-answering model’s output becomes readable answer text.
Key terms: fine-tuning, pretrained LLM, full fine-tuning, parameters, question, context, answers, start and end tokens, preprocess_function, DistilBERT-base
2.4 LoRA
This section explains why LoRA reduces fine-tuning costs and walks through preparing, training, saving, and using a LoRA-adapted FLAN-T5 summarizer.
- LoRA parameter efficiency: Explain how LoRA fits within parameter-efficient fine-tuning and why it is less computationally demanding than full fine-tuning.
Why it matters: It explains why LoRA is useful when full-model training would be costly. - LoRA summarization training workflow: Trace the main preparation and training steps in the section’s FLAN-T5 summarization example.
Why it matters: It connects the LoRA idea to the concrete sequence used to train the example summarizer. - LoRA checkpoint inference: Describe what the example saves after training and what must be loaded to use the LoRA-fine-tuned model for summarization.
Why it matters: Training the adapter is only useful if its weights can be loaded for later testing and inference.
Key terms: PEFT, LoRA, pretrained weights, full fine-tuning, FLAN-T5, samsum, tokenizer, 8-bit precision, LoraConfig, checkpoint
2.5 RAG or fine-tuning?
This section compares when RAG and fine-tuning are useful for specializing an LLM and explains why the choice depends on the use case.
- RAG suitability and limits: Explain when RAG is a good choice and why its lower training cost does not make it suitable for every task.
Why it matters: It helps you recognize both the value and the limits of adding retrieved information instead of training the model. - Fine-tuning suitability and costs: Explain why fine-tuning can be preferable for complex domain tasks, while accounting for its training costs and expertise requirements.
Why it matters: It helps you weigh improved domain-task performance against the practical cost of training. - Hallucination and interpretability tradeoffs: Distinguish how RAG and fine-tuning address interpretability and different kinds of hallucinations when choosing an approach.
Why it matters: It prevents treating either strategy as a universal fix for unreliable answers.
Key terms: RAG, up-to-date information, vector database, data pipelines, fine-tuning, task-specific dataset, domain nuances, PEFT, interpretability, hallucinations
2.6 Summary
Where this is used later
- Data preparation (chapter 3) · The earlier discussion of data preparation and the Hugging Face Datasets library.
- Using a vector database (chapter 13) · The vector-database discussion builds on representing and retrieving information as embeddings.
- Combining a domain-specific SLM with RAG (chapter 13) · Combining a domain-specific SLM with RAG relies on the distinction between retrieving context and generating a response.
- Running an LLM locally with LM Studio (chapter 12) · The earlier introduction to retrieval-augmented generation.
- Evaluation metrics (chapter 14) · The basic distinction between retrieval and generation in RAG.
- Fine-tuning (chapter 3) · The earlier introduction to fine-tuning.
- Antibody generation with an SLM (chapter 8) · The idea of fine-tuning a pretrained model for a specific domain
- Testing the fine-tuned model (chapter 3) · Provides a basic way to obtain an answer when testing a fine-tuned model.
Concepts explained in this chapter
- parameter-efficient fine-tuningA family of fine-tuning methods that adapt a pretrained model using only a small subset of parameters or added modules to reduce compute and cost.
- retrieval-augmented generationAn application system that combines retrieval from external sources with text generation to ground outputs in outside knowledge.
- fine-tuningTraining a pretrained model further on task- or domain-specific data to adapt it to a narrower use case.
- tokenizerA model-specific component that converts concatenated instruction/output text into token sequences for fine-tuning.
- vector indexA data structure that accelerates retrieval by organizing vectors for efficient similarity search.
- BERTA transformer-based language model that uses the encoder stack for understanding tasks such as classification.
- GPT-2A decoder-only Transformer language model used for text generation, here serving as the architecture underlying ProtGPT2.
- similarity searchThe process of searching external sources to find information relevant to a query or prompt.
- sentence-transformersA library for producing dense text vectors with transformer encoders for embedding and similarity search.
- transformersThe Hugging Face library for working with pretrained transformer models, tokenizers, pipelines, and related training and inference utilities.
- vector databaseA database designed to store vectors and support similarity search, typically using approximate nearest neighbor methods for retrieval.
- knowledge baseRetrieved outside information supplied to a model at inference time to ground its output.
- domain-specific large language modelA large language model specialized for a particular domain or task family.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.