liveBook · Manning

AutoThink

As explained in Domain-Specific Small Language Models

An inference-time reasoning method that chooses a reasoning budget and steers a model toward better reasoning paths using classification and control vectors.

AutoThink is an inference-time reasoning method that sets a query-specific reasoning budget and guides a model toward better reasoning paths. section 15.2 section 15.5

It combines query-complexity classification with steering-vector guidance. A classifier decides whether a query needs high- or low-complexity reasoning, and control vectors steer the model during inference. section 15.2 section 15.5

The treatment breaks AutoThink into two phases:

  • Preprocessing collects query-response pairs, finds pivotal tokens, derives vectors, and trains a query-complexity classifier. section 15.2
  • Inference loads a pretrained classifier, selects a token budget, and applies control vectors while decoding. section 15.2 section 15.5

Evaluation setup

A local small language model is evaluated on GSM8K samples with AutoThink, using OptiLLM’s autothink_decode function and a modest token budget. section 15.2 section 15.5

How Domain-Specific Small Language Models covers it

How deep it goes
Explains the two-phase pipeline and preprocessing steps, but stops short of implementation details for the classifier and vectors.
Examples
Uses a GSM8K evaluation loop on a local Qwen model and a reasoning prompt with autothink_decode.
What it assumes
Assumes familiarity with local model inference and leaves classifier training and vector derivation to preprocessing.
What you can do after
Apply it to choose a reasoning budget and guide decoding on harder queries.
How it connects
Connects to test-time compute, query-complexity classification, steering vectors, and OptiLLM.
Cover of Domain-Specific Small Language Models
liveBookmanning.com
Read Domain-Specific Small Language Models on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.