Inference-time compute scaling increases a model’s deliberation during inference rather than changing its pretrained parameters. It aims to improve responses by giving reasoning models more opportunity to work through difficult questions. section 2.0 section 2.14
Chain-of-thought prompting provides an intuitive example: asking a model to break a problem into steps, even with “Let’s think step-by-step,” can improve accuracy in some reasoning settings. These gains are not guaranteed across tasks. section 2.14
Inference-time scaling is closely related to several ways of eliciting more capable responses:
- Use zero-shot or few-shot prompts to shape the model’s behavior without retraining it. section 2.14
- Provide detailed instructions or request confidence estimates, which can improve responses in certain settings. section 2.14
- Treat apparent capability gains cautiously: emergent abilities can be unpredictable and do not always follow simple scaling patterns. section 2.0 section 2.14
Step-by-step prompting
For a challenging reasoning problem, instructing a language model to “think step-by-step” can produce a more accurate answer than an equivalent prompt without that instruction, although the effect depends on the setting. section 2.14
