liveBook · Manning

Pruned model

As explained in Rearchitecting LLMs

A pruned model is a model from which selected parameters or structural components have been removed.

Structural pruning removes selected model components—in this case, transformer decoder layers—to make an existing model lighter and faster. Knowledge recovery transfers knowledge from the base model to the pruned model to help preserve capabilities. section 2.3 section 2.0

When pruning decoder layers:

  • Copy the model, select the layers to retain, replace its layer list, and update the configured layer count. section 2.3
  • Measure the effects: removing two of 18 layers reduces parameters by only 4.16%, because embeddings and the output head remain unchanged. Layer removal also reduces memory that grows with inference context length. section 2.3

Gemma3-270m

A model with 18 Gemma3DecoderLayer blocks is reduced to 16 by removing two layers. Its behavior and performance are then evaluated. section 2.3

LAYERS_TO_REMOVE = 2
original_layers_count = len(pruned_model.model.layers)
new_layers_count = original_layers_count - LAYERS_TO_REMOVE
new_layers = pruned_model.model.layers[:new_layers_count]

Shows how to select the layers to retain when reducing the model. section 2.3

How Rearchitecting LLMs covers it

How deep it goes
Implements removal of two decoder layers from an 18-layer model and updates its layer count, while leaving embeddings and the output head unchanged.
Examples
Demonstrates reducing Gemma3-270m from 18 to 16 decoder layers, with a 4.16% parameter reduction.
What it assumes
Assumes an existing base model whose decoder layers and layer-count configuration can be edited.
What you can do after
Apply it by editing a model’s layer list to remove decoder blocks and updating its configured layer count.
How it connects
Pairs structural optimization with knowledge distillation and knowledge recovery, and connects layer removal to inference memory growth with context length.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.