Dimension alignment adjusts a matrix or tensor dimension to a hardware-friendly multiple, such as 32, 64, 128, or 256. An intermediate size such as 6554 can require GPU padding with zeros. In width pruning, OptiPFair’s expansion_divisor parameter adjusts how many neurons are retained so the resulting intermediate size is divisible by the chosen value. section 5.2.6
Alignment does not guarantee a speedup on every GPU: a T4 may be memory-bandwidth-bound, leaving Tensor Cores idle. It is most effective in compute-bound workloads such as training or high-traffic inference on A100 or H100 GPUs. section 5.2.6
pruned_model, stats = prune_model(
model=model,
pruning_type="MLP_GLU",
neuron_selection_method="PPM",
pruning_percentage=20,
expansion_divisor=64, #A
dataloader=dataloaderwiki,
show_progress=True
)
The pruning call demonstrates setting expansion_divisor to 64 when pruning MLP_GLU neurons. section 5.2.6
