liveBook · Manning

Block influence

As explained in Rearchitecting LLMs

Block Influence is a metric defined as one minus the cosine similarity between a model block's input and output representations.

Block Influence (BI) measures a model block’s representational change as one minus the cosine similarity between its input and output. It ranks blocks for depth pruning using only a forward inference pass, avoiding gradient computation and its associated processing and memory overhead. section 4.4

Pruning evidence

For Llama2-7B, low-BI intermediate blocks near 0.05 barely increase perplexity when removed, while initial blocks above 0.4 cause sharp increases. Pruning 27.1% of the model retained 86.3% of average performance across 14 benchmarks without knowledge recovery. section 4.4

How Rearchitecting LLMs covers it

How deep it goes
Defines a cosine-based score and validates its importance correlation, but does not detail the pruning implementation itself.
Examples
Uses Llama2-7B scores near 0.05 and above 0.4, plus 27.1% pruning evaluated across 14 benchmarks.
What you can do after
Evaluate block importance with a single forward inference pass before structured depth pruning.
How it connects
Connects cosine similarity with depth pruning, perplexity, Taylor approximation, gradients, and static importance methods.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.