Elicitation theory views post-training as the process of extracting and cultivating capabilities already latent in a pretrained base model. Pretraining supplies broad knowledge and intelligence, but next-token prediction does not automatically produce useful question-answering behavior. section 1.4
The practical challenge is to reshape model behavior without treating post-training as the primary source of capability. Instruction fine-tuning, preference tuning, RLHF, reinforcement learning with verifiable rewards, and related methods cultivate different parts of the pretrained potential. section 1.4 section 1.7
The theory implies several implementation and evaluation considerations:
- Measure post-training by more than response style: it can improve benchmark performance while over-optimization, such as excessive length or chattiness, harms capabilities elsewhere. section 1.7
- Treat instruction tuning as an access mechanism rather than the complete explanation of model intelligence; narrow datasets may extract only a fraction of base-model potential. section 1.4
- Interpret alignment partly as selecting useful interaction formats, while recognizing that post-training can cultivate substantive capabilities as well as style, tone, and format. section 1.4 section 1.7
A post-training-only improvement
OLMoE Instruct’s second version changed little of its pretraining but updated post-training; its average on popular benchmarks rose from 35 to 48. The comparison illustrates how much performance can be unlocked after pretraining. section 1.4
