Compute
Test-time compute is computational effort allocated during inference to process, search, reason about, or adapt to a task before producing an output.
Domain-Specific Small Language Models
The practice of allocating more computation at inference time to improve a model’s output quality or reasoning behavior without changing the base model.
- How it is used
- Improves difficult math, logic, and coding responses by spending additional inference computation on exploration and evaluation.
- How deep it goes
- Defines TTC and explains search, candidate scoring, verification, proxy orchestration, and compute-cost tradeoffs.
- Examples
- Demonstrates OptiLLM decoding with Qwen2.5-0.5B-Instruct on a word problem and GSM8K benchmark samples.
- What you can do after
- Apply it via OptiLLM to refine responses from OpenAI API–compatible services or local Hugging Face models.
Read how Domain-Specific Small Language Models explains compute →
Sutskever's List
Test-time compute is the allocation of additional computation during inference so a model can perform more extensive internal processing before producing an output.
- How it is used
- Uses additional inference-time computation to support reasoning as pretraining alone yields diminishing returns on reasoning tasks.
- How deep it goes
- Explains scratchpads and multi-chain selection, but does not specify how to implement or schedule additional computation.
- Examples
- Illustrates scratchpad prompting through query subtasks and multi-chain reasoning through search and consistency voting.
- What you can do after
- Evaluate how additional inference-time deliberation may improve reliability and where prompting falls short on novel constraints.
Related concepts
- language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- beam searchuses · Beam search is a heuristic search and decoding technique that iteratively expands a limited set of the highest-scoring partial candidates.
- costuses · Cost is the monetary expenditure, computational or other resource consumption, effort, or burden incurred in acquiring, producing, operating, or performing something.
- scalabilityalternative to · Scalability is the ability of a system, process, or method to accommodate increasing workload, size, or demand while maintaining acceptable operation.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.