Full fine-tuning can demand substantial memory for model weights, optimizer state, and saved activations. LoRA instead trains low-rank updates while keeping the original weights fixed, so only a tiny fraction of parameters need training. This can make fine-tuning large models more affordable and leave room for a frozen reference copy. section 11.0 section 11.3
LoRA serves as the fine-tuning component in a reasoning pipeline that also uses Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO). The running application is financial reasoning, where answers such as year-over-year revenue change can be checked against a correct value. section 11.0
The memory trade-off
A 3-billion-parameter model needs about 6 GB just for bfloat16 weights; full fine-tuning adds optimizer state and activations, pushing the estimate beyond 40 GB. LoRA is presented as a way to target training that model on one 24-GB GPU. section 11.3
Figure 11.3 LoRA architecture for a single linear layer.

View the LoRA architecture applied to a single linear layer. section 11.3
















