Freeze

As explained in Deep Learning with Python, Third Edition

To keep selected model parameters unchanged during training so they do not receive optimizer updates.

Freezing keeps selected model weights unchanged during training. In transfer learning, freeze a pretrained convolutional base while a newly added classifier learns, protecting previously learned representations from destructive updates. section 8.3.1

Freeze a pretrained base

In Keras, set `conv_base.trainable = False`, connect the base to a new dense classifier, and compile the model. Only the classifier’s weights are then trained. Recompile whenever trainability changes, or the optimizer may ignore the change. section 8.3.1

Use freezing as part of a staged transfer-learning workflow:

• Add the custom network, freeze the base, and train the added classifier before changing the base. section 8.3.2
• When fine-tuning, unfreeze all or selected upper layers; keep lower layers frozen because they encode more generic features. section 8.3.2
• Do not unfreeze BatchNormalization layers during this procedure. section 8.3.2

Figure 8.12 Swapping classifiers while keeping the same convolutional base

Visualizes swapping a classifier while retaining the same convolutional base. section 8.3.1

How Deep Learning with Python, Third Edition covers it

How deep it goes

Defines weight immutability, shows Keras configuration and recompilation, and gives staged and partial fine-tuning procedures.

Examples

Uses a frozen Xception convolutional base, dense classifier, augmented image dataset, trainable-weight counts, and staged fine-tuning.

What it assumes

Assumes a pretrained base and added classifier, while teaching why classifier training precedes unfreezing.

What you can do after

Operate/configure Keras models by freezing bases, training replacement classifiers, recompiling, and selectively unfreezing layers.

How it connects

Connects freezing with transfer learning, feature extraction, fine-tuning, data augmentation, pretrained models, and batch normalization.

liveBookmanning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA frozen parameter is a model parameter excluded from optimization so that its value remains unchanged during further training.

Compare how 2 books explain freeze →

Related concepts

• transfer learningcontrasts with · Transfer learning is a machine-learning technique that reuses knowledge learned from one task, domain, or dataset to improve learning or performance on another.
• warm startcontrasts with · Warm start is a technique that initializes training or optimization from prior parameters, data, or a related solution instead of starting from scratch.
• expert networkconfigures · An expert network is a specialized neural subnetwork within a mixture-of-experts model that processes inputs selected or routed to it by a gating mechanism.
• foundational modelconfigures · A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
• fine-tuninguses · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.