prune_model applies depth pruning to a model using selected layer indices. It provides a way to compare models pruned with different block-selection strategies. section 4.3.4
Compare selection strategies
For four removed blocks, the unprotected selection is [25, 26, 24, 23], while the protected selection is [25, 23, 13, 15]. Both models remove the same number of blocks. section 4.3.4
from optipfair import prune_model
winogrande_model = prune_model(
model=deepcopy(model),
pruning_type="DEPTH",
layer_indices=winogrande_layers_2_remove,
show_progress=True,
)
winogrande_model_protected = prune_model(
model=deepcopy(model),
pruning_type="DEPTH",
layer_indices=winogrande_protected_layers_2_remove,
show_progress=True,
)
See the OptiPFair calls that prune copies of the model with each selection. section 4.3.4
