liveBook · Manning

Freeze

Also called freezing

Freezing is the practice of keeping selected model parameters unchanged during training or adaptation, typically while allowing other parameters to be updated.

How 5 Manning books explain it

Deep Learning with R, Third Edition

A frozen parameter is a model parameter excluded from optimization so that its value remains unchanged during further training.

How it is used
Uses freezing to preserve pretrained image features and reduce memory during language-model fine-tuning.
How deep it goes
Explains how to set and inspect trainability, recompile after changes, and combine frozen weights with a low-rank update.
Examples
Demonstrates a frozen Xception base with a dense classifier and a linear layer with a rank-8 LoRA update.
What you can do after
Apply it via Keras to freeze and inspect weights, recompile after trainability changes, and train a new classifier.

Read how Deep Learning with R, Third Edition explains freeze →

Inside Deep Learning: Math, Algorithms, Models

A transfer-learning technique that keeps selected model parameters fixed during training while allowing the remaining parameters to be updated.

How it is used
Uses pretrained parameters as fixed features while adapting selected layers to a new task, especially when labeled data is scarce.
How deep it goes
Defines parameter freezing, shows PyTorch implementations, and compares tradeoffs, but does not provide a general rule for choosing layers.
Examples
Uses a frozen ResNet18 with a replacement classifier and a frozen BERT classifier compared with a GRU.
What you can do after
Apply it via PyTorch by freezing pretrained parameters, replacing the final layer, and training the remaining parameters.

Read how Inside Deep Learning: Math, Algorithms, Models explains freeze →

Rearchitecting LLMs

Selective freezing is a training technique that keeps chosen model parameters fixed while allowing other parameters to update.

How it is used
Restricts MoE specialization to selected expert and router parameters while keeping the general expert fixed.
How deep it goes
Shows parameter-level freezing and a trainable-weight count for the selected MoE components.
Examples
Uses clinical-expert training and an experiment freezing one expert while training another expert's tail layers and the router.
What you can do after
Configure which expert and router parameters can update, then verify the trainable-weight fraction.

Read how Rearchitecting LLMs explains freeze →

Building Specialized AI Systems

Selective layer freezing is a fine-tuning technique that keeps chosen neural-network layers fixed while allowing other layers to update.

How it is used
Deploys partial fine-tuning when updating every parameter is unnecessary, undesirable, or computationally prohibitive.
How deep it goes
Descends from freezing rationale to gradient criteria, dynamic schedules, sublayer choices, and PyTorch implementation details.
Examples
Demonstrates BERT-base binary classification with six frozen encoder layers, partial attention freezing, AdamW, and epoch-three unfreezing.
What you can do after
Apply it via PyTorch and Transformers by toggling requires_grad, filtering optimizer parameters, scheduling unfreezing, and checking final status.

Read how Building Specialized AI Systems explains freeze →

Deep Learning with Python, Third Edition

To keep selected model parameters unchanged during training so they do not receive optimizer updates.

How it is used
Protects pretrained representations while a new classifier learns, then supports controlled adaptation during fine-tuning.
How deep it goes
Defines weight immutability, shows Keras configuration and recompilation, and gives staged and partial fine-tuning procedures.
Examples
Uses a frozen Xception convolutional base, dense classifier, augmented image dataset, trainable-weight counts, and staged fine-tuning.
What you can do after
Operate/configure Keras models by freezing bases, training replacement classifiers, recompiling, and selectively unfreezing layers.

Read how Deep Learning with Python, Third Edition explains freeze →

Also covered in

What to know first

Related concepts

Questions about freeze

What is freeze?

Freezing is the practice of keeping selected model parameters unchanged during training or adaptation, typically while allowing other parameters to be updated.

Which Manning books explain freeze?

Deep Learning with R, Third Edition, Inside Deep Learning: Math, Algorithms, Models, Rearchitecting LLMs, Building Specialized AI Systems and Deep Learning with Python, Third Edition.

What should I know before learning freeze?

Manning books that teach freeze usually introduce convolutional neural network, fine-tuning, linear layer and classifier first.

What is the difference between freeze and transfer learning?

Freezing is the practice of keeping selected model parameters unchanged during training or adaptation, typically while allowing other parameters to be updated. Transfer learning is a machine-learning technique that reuses knowledge learned from one task, dataset, or domain to improve learning or performance on another.

What is freeze also called?

Freeze is also called freezing.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.