Instruction following is a model capability that can be targeted during optimization. IFEval can guide that effort; width pruning may improve instruction-following performance while degrading knowledge-intensive and reasoning capabilities. section 1.2.5 section 5.3
Figure 5.9 The dichotomy visualized in Llama-3.2-3B. Panel A shows the progressive collapse of fragile capabilities (knowledge and reasoning) as the expansion ratio decreases. Panel B reveals the paradox: robust capabilities (instruction-following and truthfulness) improve dramatically, with IFEval reaching a peak of 174% of baseline at 1.6x expansion (40% pruning). Performance is normalized to the base model (2.67x expansion = 100%).
Compares normalized capability changes as Llama-3.2-3B is pruned; IFEval peaks at 174% of baseline at 1.6× expansion, corresponding to 40% pruning. section 5.3
