liveBook · Manning

Low-rank adaptation

As explained in Applied Reinforcement Learning

LoRA is a parameter-efficient fine-tuning technique that trains low-rank updates to a model's weights while keeping its original weights fixed.

Full fine-tuning can demand substantial memory for model weights, optimizer state, and saved activations. LoRA instead trains low-rank updates while keeping the original weights fixed, so only a tiny fraction of parameters need training. This can make fine-tuning large models more affordable and leave room for a frozen reference copy. section 11.0 section 11.3

LoRA serves as the fine-tuning component in a reasoning pipeline that also uses Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO). The running application is financial reasoning, where answers such as year-over-year revenue change can be checked against a correct value. section 11.0

The memory trade-off

A 3-billion-parameter model needs about 6 GB just for bfloat16 weights; full fine-tuning adds optimizer state and activations, pushing the estimate beyond 40 GB. LoRA is presented as a way to target training that model on one 24-GB GPU. section 11.3

Figure 11.3 LoRA architecture for a single linear layer.

View the LoRA architecture applied to a single linear layer. section 11.3

How Applied Reinforcement Learning covers it

How deep it goes
Quantifies full-fine-tuning memory for weights, optimizer state, and activations, then frames LoRA as a low-memory alternative.
Examples
Uses financial reasoning and estimates the memory needed to fine-tune a 3-billion-parameter model on a 24-GB GPU.
What you can do after
Apply it to fine-tune large models with a tiny trainable parameter subset and target training on a single 24-GB GPU.
How it connects
Connects LoRA with RLVR and GRPO in a critic-free, reward-model-free approach to language-model reasoning.

Also explained in

Compare how 17 books explain low-rank adaptation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.