Peak-to-peak magnitude (PPM) ranks neurons for MLP width pruning. In hybrid mode, it combines neuron weights with activation statistics measured on calibration prompts. section 13.4.1
With OptiPFair, select PPM for MLP-GLU modules and set the target width using an expansion rate. A divisor can constrain the resulting intermediate size to a multiple suited to GPU use. section 13.4.1
Listing 13.13 Width pruning
EXPANSION_RATE = 225
EXPANSION_DIVISOR = 128
model_m_pruned, width_stats = opf.prune_model(
model=copy.deepcopy(depth_pruned_model),
pruning_type="MLP_GLU",
neuron_selection_method="PPM", #1
pruning_percentage=None, #A
expansion_rate=EXPANSION_RATE, #2
expansion_divisor=EXPANSION_DIVISOR, #3
dataloader=calibration_dataloader, #B
show_progress=True,
return_stats=True,
)
model_m_pruned.generation_config.pad_token_id = tokenizer.pad_token_id
model_m_pruned.eval()
inter = [layer.mlp.gate_proj.out_features for layer in model_m_pruned.model.layers]
print(f"layers: {len(inter)}")
print(f"intermediate sizes: {Counter(inter)}")See the OptiPFair call that applies PPM to a calibrated model and sets its target expansion rate and divisor. section 13.4.1
