liveBook · Manning

Instruction following

As explained in Rearchitecting LLMs

Instruction following is the ability of a model to carry out requested operations while satisfying specified constraints.

Instruction following is a model capability that can be targeted during optimization. IFEval can guide that effort; width pruning may improve instruction-following performance while degrading knowledge-intensive and reasoning capabilities. section 1.2.5 section 5.3

Figure 5.9 The dichotomy visualized in Llama-3.2-3B. Panel A shows the progressive collapse of fragile capabilities (knowledge and reasoning) as the expansion ratio decreases. Panel B reveals the paradox: robust capabilities (instruction-following and truthfulness) improve dramatically, with IFEval reaching a peak of 174% of baseline at 1.6x expansion (40% pruning). Performance is normalized to the base model (2.67x expansion = 100%).

Compares normalized capability changes as Llama-3.2-3B is pruned; IFEval peaks at 174% of baseline at 1.6× expansion, corresponding to 40% pruning. section 5.3

How Rearchitecting LLMs covers it

How deep it goes
Compares instruction-following with knowledge and reasoning across pruning levels, including normalized IFEval results.
Examples
Shows Llama-3.2-3B reaching 174% of baseline IFEval performance at 40% pruning.
What you can do after
Evaluate pruning trade-offs between IFEval performance and knowledge-intensive or reasoning benchmarks.
How it connects
Ties instruction following to width pruning, IFEval, truthfulness, and knowledge and reasoning retention.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.