Freeze
Freezing is the practice of keeping selected model parameters unchanged during training or adaptation, typically while allowing other parameters to be updated.
Deep Learning with R, Third Edition
A frozen parameter is a model parameter excluded from optimization so that its value remains unchanged during further training.
- How it is used
- Uses freezing to preserve pretrained image features and reduce memory during language-model fine-tuning.
- How deep it goes
- Explains how to set and inspect trainability, recompile after changes, and combine frozen weights with a low-rank update.
- Examples
- Demonstrates a frozen Xception base with a dense classifier and a linear layer with a rank-8 LoRA update.
- What you can do after
- Apply it via Keras to freeze and inspect weights, recompile after trainability changes, and train a new classifier.
Read how Deep Learning with R, Third Edition explains freeze →
Inside Deep Learning: Math, Algorithms, Models
A transfer-learning technique that keeps selected model parameters fixed during training while allowing the remaining parameters to be updated.
- How it is used
- Uses pretrained parameters as fixed features while adapting selected layers to a new task, especially when labeled data is scarce.
- How deep it goes
- Defines parameter freezing, shows PyTorch implementations, and compares tradeoffs, but does not provide a general rule for choosing layers.
- Examples
- Uses a frozen ResNet18 with a replacement classifier and a frozen BERT classifier compared with a GRU.
- What you can do after
- Apply it via PyTorch by freezing pretrained parameters, replacing the final layer, and training the remaining parameters.
Read how Inside Deep Learning: Math, Algorithms, Models explains freeze →
Rearchitecting LLMs
Selective freezing is a training technique that keeps chosen model parameters fixed while allowing other parameters to update.
- How it is used
- Restricts MoE specialization to selected expert and router parameters while keeping the general expert fixed.
- How deep it goes
- Shows parameter-level freezing and a trainable-weight count for the selected MoE components.
- Examples
- Uses clinical-expert training and an experiment freezing one expert while training another expert's tail layers and the router.
- What you can do after
- Configure which expert and router parameters can update, then verify the trainable-weight fraction.
Building Specialized AI Systems
Selective layer freezing is a fine-tuning technique that keeps chosen neural-network layers fixed while allowing other layers to update.
- How it is used
- Deploys partial fine-tuning when updating every parameter is unnecessary, undesirable, or computationally prohibitive.
- How deep it goes
- Descends from freezing rationale to gradient criteria, dynamic schedules, sublayer choices, and PyTorch implementation details.
- Examples
- Demonstrates BERT-base binary classification with six frozen encoder layers, partial attention freezing, AdamW, and epoch-three unfreezing.
- What you can do after
- Apply it via PyTorch and Transformers by toggling requires_grad, filtering optimizer parameters, scheduling unfreezing, and checking final status.
Deep Learning with Python, Third Edition
To keep selected model parameters unchanged during training so they do not receive optimizer updates.
- How it is used
- Protects pretrained representations while a new classifier learns, then supports controlled adaptation during fine-tuning.
- How deep it goes
- Defines weight immutability, shows Keras configuration and recompilation, and gives staged and partial fine-tuning procedures.
- Examples
- Uses a frozen Xception convolutional base, dense classifier, augmented image dataset, trainable-weight counts, and staged fine-tuning.
- What you can do after
- Operate/configure Keras models by freezing bases, training replacement classifiers, recompiling, and selectively unfreezing layers.
Read how Deep Learning with Python, Third Edition explains freeze →
Also covered in
What to know first
- convolutional neural networkA convolutional neural network is a neural-network architecture that processes structured inputs with convolutional operations, typically using learned local filters to capture patterns.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- linear layerA linear layer is a neural-network layer that applies a learned affine transformation to its input, typically using weights and a bias.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
Related concepts
- foundation modelconfigures · A foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
- expert networkconfigures · An expert network is a specialized neural subnetwork within a mixture-of-experts model that processes inputs selected or routed to it.
- fine-tuninguses · Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- transfer learningcontrasts with · Transfer learning is a machine-learning technique that reuses knowledge learned from one task, dataset, or domain to improve learning or performance on another.
- warm startcontrasts with · Warm start is a technique that initializes training or optimization from prior parameters, data, or a related solution instead of starting from scratch.
Questions about freeze
What is freeze?
Freezing is the practice of keeping selected model parameters unchanged during training or adaptation, typically while allowing other parameters to be updated.
Which Manning books explain freeze?
Deep Learning with R, Third Edition, Inside Deep Learning: Math, Algorithms, Models, Rearchitecting LLMs, Building Specialized AI Systems and Deep Learning with Python, Third Edition.
What should I know before learning freeze?
Manning books that teach freeze usually introduce convolutional neural network, fine-tuning, linear layer and classifier first.
What is the difference between freeze and transfer learning?
Freezing is the practice of keeping selected model parameters unchanged during training or adaptation, typically while allowing other parameters to be updated. Transfer learning is a machine-learning technique that reuses knowledge learned from one task, dataset, or domain to improve learning or performance on another.
What is freeze also called?
Freeze is also called freezing.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




