Adapter-based fine-tuning makes domain adaptation practical when full-model training is too expensive. It trains a small set of added parameters while keeping the pretrained base weights frozen, reducing hardware and storage requirements. section 9.3.1 section 15.3.3
LoRA and QLoRA adapters can teach a model domain behavior, noise robustness, or reflection-token prediction without retraining every parameter. Multiple small adapters can share one base model and be hot-swapped at serving time. section 9.3.1 section 15.3.3
Select the adapter approach according to the deployment constraint:
- Use LoRA or QLoRA when a few thousand domain examples must fit on a single consumer or rented GPU. section 15.3.3
- Use configuration-driven or optimized training tools such as Axolotl, Unsloth, or TRL to reduce implementation and memory overhead. section 15.3.3
- Serve adapters through an inference server such as vLLM or TGI when several domain variants must share one base model. section 15.3.3
- Resize the tokenizer and embedding matrix before training when the task adds special tokens, such as Self-RAG reflection tokens. section 9.3.1
A production-sized adaptation
A 7B model can be adapter-trained on a single 24 GB consumer GPU with 4-bit quantization, while a 13B model can fit on a 48 GB card. In a RAG deployment, LoRA trained on roughly 2,000 domain-specific question-answer pairs can improve retrieval precision and generation faithfulness without full fine-tuning. section 9.3.1 section 15.3.3
Figure 9.3 The Self-RAG training pipeline.

Trace the critic-generated training data, adapter-trained student, and reflection-token pathway through the Self-RAG pipeline. section 9.3.1
