Block Influence (BI) measures a model block’s representational change as one minus the cosine similarity between its input and output. It ranks blocks for depth pruning using only a forward inference pass, avoiding gradient computation and its associated processing and memory overhead. section 4.4
Pruning evidence
For Llama2-7B, low-BI intermediate blocks near 0.05 barely increase perplexity when removed, while initial blocks above 0.4 cause sharp increases. Pruning 27.1% of the model retained 86.3% of average performance across 14 benchmarks without knowledge recovery. section 4.4
