Block selection strategy chooses which complete Transformer blocks to remove during depth pruning. Selection matters: removing the same number of blocks can produce the same inference gain but very different performance costs. section 4.1 section 4.1.2
Compared approaches include:
- Data-driven individual selection: remove the least-contributing blocks identified through cosine-similarity importance analysis, whether consecutive or not. section 6.1.1
- Data-driven consecutive selection: apply the same importance analysis while requiring the removed blocks to be consecutive. section 6.1.1
- Heuristic strategies provide alternatives to data-driven importance analysis. section 6.1.1
Initial comparison
The experiment removes four of Gemma 3 270M’s 18 Transformer blocks and first evaluates strategies on 2,000 Cosmopedia records before scaling the winning approach. section 6.1.1
Figure 4.2 Impact of block selection on depth pruning. The gain in inference (+15%) is identical as we are removing four blocks in both scenarios. The performance cost varies drastically depending on which ones are removed.
Compare how removing the same number of blocks can yield different performance costs. section 4.1.2
Figure 6.1 Comparison of layer removal strategies. All experiments remove 4 blocks (striped) from the original 18-block model. EXP01-02 use data-driven importance analysis (Listing 6.5), while EXP03-04 follow heuristic rules.
View the four-block removal strategies compared on the 18-block model. section 6.1.1
