PEFT implements LoRA, QLoRA, and other adapter methods for fine-tuning pretrained models. Paired with TRL, it provides flexible training primitives while leaving more of the training loop under your control. section 2.6 section 2.6
Table 2.4 The mainstream training libraries.
|
Tool
|
Style
|
What does it give you
|
Trade-off
|
|
TRL plus PEFT
|
Python library
|
The reference SFTTrainer,
DPOTrainer, GRPOTrainer; LoRA, QLoRA, and adapter primitives
|
Maximum flexibility; you write more
of the training loop yourself
|
|
Unsloth
|
Drop-in wrapper
|
Hand-written CUDA kernels for 2 to
3x faster training and 40 to 60% less VRAM
|
Limited to supported architectures;
less ecosystem fit when you need custom losses
|
|
Axolotl
|
YAML-driven
|
One YAML file describes the model,
dataset, LoRA config, scheduler, and evaluation; the tool handles the rest.
|
Excellent for reproducing recipes
and running ablations; less suited to experimental techniques outside its
schema
|
|
LLaMA-Factory
|
YAML and Web UI
|
Similar to Axolotl with a web
interface, broad model coverage
|
Friendly entry point for
non-engineers; less control than direct TRL when you need it
|
|
DeepSpeed
|
Distributed-training layer
|
ZeRO optimizer sharding across
multiple GPUs; enables fine-tuning of 70B+ models on a multi-GPU cluster.
|
Configuration complexity; rarely
used alone, more often plugged into one of the above
|
Compare PEFT's role in the TRL-plus-PEFT stack with other mainstream training tools and their stated trade-offs. section 2.6
