liveBook · Manning

Speedup degradation ratio

As explained in Rearchitecting LLMs

SDR, or Speedup Degradation Ratio, is the percentage of performance drop divided by the percentage of speedup gained from a model modification.

The speedup degradation ratio (SDR) compares the percentage of performance lost with the percentage of speedup gained: SDR = percentage performance drop ÷ percentage speedup gained. A lower SDR indicates a better trade-off—more speed for less quality loss. section 8.4.2 section 8.5.1

Use SDR to compare pruning strategies, but pair static benchmark results with autoregressive generation checks: small quality inconsistencies may emerge over long outputs, and an apparent speedup can come with generation collapse. section 8.4.2 section 8.5.1

Pruning trade-offs

For Llama-3.2-3B, removing 3 of 28 attention layers produced no measurable speedup; removing 15 yielded about 6% more throughput but caused repetition from the first token. By contrast, removing 32 of 80 attention layers from Llama-2-70B produced a 1.35× speedup with an SDR of 0.00. section 8.5.1

Figure 8.11 The four pruning strategies explored in He et al. (2024). Each column shows which modules are removed from a Transformer block (crossed out) and which are preserved. The bar at the bottom represents the SDR (Speedup Degradation Ratio). A shorter bar indicates a better trade-off between speed gain and quality loss. Attention Drop achieves the lowest SDR by a wide margin, making it the most efficient strategy despite leaving MLP layers intact.

Compare the four pruning strategies and their relative SDRs; a shorter bar indicates a better speed–quality trade-off. section 8.5.1

How Rearchitecting LLMs covers it

How deep it goes
Defines the ratio and interprets lower values, then applies it to pruning results without detailing a measurement procedure.
Examples
Compares attention-layer removal on Llama-3.2-3B and Llama-2-70B, including throughput, SDR, and generation effects.
What you can do after
Evaluate pruning trade-offs by comparing speed gains with performance degradation and checking generation behavior.
How it connects
Connects attention pruning with Block Drop, MLP Drop, and Joint Layer Drop comparisons.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.