liveBook · Manning

Prune_model

As explained in Rearchitecting LLMs

prune_model is a function that removes selected structural components from a neural model to produce a pruned model.

prune_model applies depth pruning to a model using selected layer indices. It provides a way to compare models pruned with different block-selection strategies. section 4.3.4

Compare selection strategies

For four removed blocks, the unprotected selection is [25, 26, 24, 23], while the protected selection is [25, 23, 13, 15]. Both models remove the same number of blocks. section 4.3.4

from optipfair import prune_model
 
winogrande_model = prune_model(
    model=deepcopy(model),
    pruning_type="DEPTH",
    layer_indices=winogrande_layers_2_remove,
    show_progress=True,
)
 
winogrande_model_protected = prune_model(
    model=deepcopy(model),
    pruning_type="DEPTH",
    layer_indices=winogrande_protected_layers_2_remove,
    show_progress=True,
)

See the OptiPFair calls that prune copies of the model with each selection. section 4.3.4

How Rearchitecting LLMs covers it

How deep it goes
Shows calls with a model, pruning type, layer indices, and progress flag, without detailing the pruning internals.
Examples
Compares four-block selections [25, 26, 24, 23] and [25, 23, 13, 15].
What it assumes
Assumes a model and selected layer indices are available; selection safeguards are handled separately.
What you can do after
Apply depth pruning through OptiPFair's prune_model function to copied models using selected layer indices.
How it connects
Connects depth pruning with transformer-block selection and importance-based layer ranking.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.