The speedup degradation ratio (SDR) compares the percentage of performance lost with the percentage of speedup gained: SDR = percentage performance drop ÷ percentage speedup gained. A lower SDR indicates a better trade-off—more speed for less quality loss. section 8.4.2 section 8.5.1
Use SDR to compare pruning strategies, but pair static benchmark results with autoregressive generation checks: small quality inconsistencies may emerge over long outputs, and an apparent speedup can come with generation collapse. section 8.4.2 section 8.5.1
Pruning trade-offs
For Llama-3.2-3B, removing 3 of 28 attention layers produced no measurable speedup; removing 15 yielded about 6% more throughput but caused repetition from the first token. By contrast, removing 32 of 80 attention layers from Llama-2-70B produced a 1.35× speedup with an SDR of 0.00. section 8.5.1
Figure 8.11 The four pruning strategies explored in He et al. (2024). Each column shows which modules are removed from a Transformer block (crossed out) and which are preserved. The bar at the bottom represents the SDR (Speedup Degradation Ratio). A shorter bar indicates a better trade-off between speed gain and quality loss. Attention Drop achieves the lowest SDR by a wide margin, making it the most efficient strategy despite leaving MLP layers intact.
Compare the four pruning strategies and their relative SDRs; a shorter bar indicates a better speed–quality trade-off. section 8.5.1
