Structural pruning removes selected model components—in this case, transformer decoder layers—to make an existing model lighter and faster. Knowledge recovery transfers knowledge from the base model to the pruned model to help preserve capabilities. section 2.3 section 2.0
When pruning decoder layers:
- Copy the model, select the layers to retain, replace its layer list, and update the configured layer count. section 2.3
- Measure the effects: removing two of 18 layers reduces parameters by only 4.16%, because embeddings and the output head remain unchanged. Layer removal also reduces memory that grows with inference context length. section 2.3
Gemma3-270m
A model with 18 Gemma3DecoderLayer blocks is reduced to 16 by removing two layers. Its behavior and performance are then evaluated. section 2.3
LAYERS_TO_REMOVE = 2 original_layers_count = len(pruned_model.model.layers) new_layers_count = original_layers_count - LAYERS_TO_REMOVE new_layers = pruned_model.model.layers[:new_layers_count]
Shows how to select the layers to retain when reducing the model. section 2.3
