liveBook · Manning

Adapter

As explained in LLM Customization and Fine-Tuning

A small trainable component added to a model to modify its behavior without updating all its original parameters.

When prompting cannot produce the required behavior, but full fine-tuning is too costly, an adapter offers a middle ground. It is a small trainable component added to a model for a task or domain while the original model weights remain frozen. The adapter is saved separately and combined with the base model for inference. section 5.2 section 5.0

LoRA is a common adapter approach: it inserts pairs of small trainable low-rank matrices alongside frozen weight matrices. Training updates only these added parameters, reducing compute and memory needs. Adapters also make specialization modular: one base model can serve different tasks by switching adapters. section 5.2 section 5.6

An IT support adapter

A quickstart trains a LoRA adapter for Qwen3-4B-Instruct-2507 on 40 cleaned IT support examples. It loads the base, configures and trains the adapter, tests generation, and saves the adapter. Training takes under two minutes on an A30 GPU; this small run is a preview, not a production recipe. section 2.8

Listing 5.2 Creating a LoRA configuration and passing it to SFTTrainer
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig
from trl import SFTTrainer
 
model_name = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
 
# Define LoRA hyper-parameters
lora_config = LoraConfig(
    r=16,                # rank (try 8, 16, or 32)
    lora_alpha=32,       # scaling factor (typically 2x r)
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj",
                    "up_proj", "gate_proj", "down_proj"],  # attn + MLP
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)
 
# SFTTrainer applies LoRA automatically - no manual get_peft_model call
trainer = SFTTrainer(
    model=model,
    train_dataset=train_ds,
    eval_dataset=valid_ds,
    peft_config=lora_config,   # LoRA applied here
    ...
)
 
trainer.train()

The configuration sets rank, scaling, dropout, and target modules, then passes it to TRL's SFTTrainer for adapter training. section 5.1.2

For inference, load the same base model and attach the saved adapter weights. Use the model's chat template to format messages. In the Qwen3-4B example, inference with the adapter uses approximately 8 GB of GPU memory; the published adapter is about 130 MB, compared with about 8 GB for the base model. section 5.1.5 section 5.1.2

Listing 5.4 Load base model, add LoRA adapter, and generate using a prompt
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
 
# Load base model and tokenizer
model_name = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
 
# Attach LoRA adapter on top
adapter_path = "chapter05/runs/it_lora"
model = PeftModel.from_pretrained(model, adapter_path)
model.eval()
 
# Build prompt using chat template
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user",
     "content": "Explain how photosynthesis works in simple terms."},
]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True)
 
# Generate
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

This listing loads a base model, attaches an adapter with PEFT, and formats a prompt for generation. section 5.1.5

Make the deployment decision against the alternatives:

  • Start with prompting or retrieval when they meet the requirement; they need no trainable adapter. Combine retrieval for knowledge with LoRA for behavior when both are needed. section 2.8 section 1.1
  • Choose LoRA when a small task-specific update is enough. It saves training memory, but an extra matrix multiplication can increase inference latency on some hardware. section 5.2.2
  • Consider full fine-tuning if quality plateaus: limited adapter capacity may miss patterns in very large datasets. LoRA is not a substitute for a base model with the needed domain knowledge. section 5.2.2 section 5.2
  • For multiple adapters, route requests explicitly, by keyword, or with a classifier. Keyword rules are fast but brittle; classifiers can handle ambiguity at the cost of latency. section 5.6.1

A Multi-LoRA deployment typically activates one adapter per endpoint or uses separate endpoints. Adapter merging and composition are possible but experimental. section 5.6

How LLM Customization and Fine-Tuning covers it

How deep it goes
Explains low-rank matrices and frozen weights, then walks through configuration, training, inference, and multi-adapter routing.
Examples
Trains a Qwen3-4B IT support adapter and compares base and adapted responses to a photosynthesis prompt.
What you can do after
Apply it via PEFT and TRL to train, save, and load a LoRA adapter for inference.
How it connects
Connects adapters to LoRA, parameter-efficient fine-tuning, retrieval-augmented generation, and full fine-tuning.

Also explained in

Compare how 4 books explain adapter →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.