Steering vector

As explained in Domain-Specific Small Language Models

A control method that uses learned vectors to steer model behavior during inference.

A steering vector controls model behavior during inference by modifying activation-based decoding guidance. It directs reasoning toward more effective paths without requiring the ML engineer to manage preprocessing. section 15.2

AutoThink operationalizes steering vectors in two phases:

• Preprocessing collects query-response pairs, identifies pivotal tokens affecting reasoning quality, derives vectors, and trains a query-complexity classifier. section 15.2
• Inference selects a reasoning budget, applies control vectors during decoding, and generates the final response after reaching that budget. section 15.2 section 15.5
• OptiLLM downloads the pretrained classifier automatically on first use, keeping vector preparation and classification transparent to the implementer. section 15.2

Adaptive reasoning on GSM8K

A Qwen 2.5 0.5B Instruct model can be evaluated on GSM8K samples with OptiLLM’s AutoThink decoder. AutoThink classifies query complexity, chooses a token budget, steers decoding, and supports evaluation on a small VM, although a GPU speeds testing. section 15.2

Figure 15.2 The two AutoThink phases (preprocessing and inference)

Trace how preprocessing produces steering assets and how inference applies them alongside query-complexity classification and budget selection. section 15.2

How Domain-Specific Small Language Models covers it

How deep it goes

Explains preprocessing and inference roles, but defers the underlying vector-derivation and classifier-training mechanics.

Examples

Uses AutoThink with Qwen 2.5 0.5B Instruct on GSM8K samples through the OptiLLM inference proxy.

What it assumes

Assumes familiarity with language-model inference and introduces the preprocessing and inference division from zero.

What you can do after

Operate adaptive reasoning through OptiLLM’s AutoThink decoder and evaluate it on selected reasoning benchmarks.

How it connects

Connects activation-based steering vectors with test-time compute, reasoning budgets, query-complexity classification, and inference.

liveBookmanning.com

Read Domain-Specific Small Language Models on liveBook Get the book at manning.com

Related concepts

• AutoThinkused by · An inference-time reasoning method that chooses a reasoning budget and steers a model toward better reasoning paths using classification and control vectors.

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.