liveBook · Manning

Query-complexity classification

As explained in Domain-Specific Small Language Models

A classifier that estimates whether a query needs high- or low-complexity reasoning so the inference budget can be adjusted adaptively.

Query-complexity classification estimates whether a query needs high- or low-complexity reasoning so inference budget can be adjusted per request. section 15.2 section 15.5

It appears inside AutoThink as one part of efficient reasoning, alongside steering-vector guidance. The classifier is trained during preprocessing and then used at inference to choose token budget transparently. section 15.2 section 15.5

In this workflow, it is tied to:

  • preprocessing on query-response pairs and calibration data. section 15.2
  • inference-time budget selection before decoding begins. section 15.2
  • reasoning-path steering through control vectors. section 15.2

AutoThink evaluation setup

A local Qwen 2.5 0.5B Instruct model is evaluated on OptiLLMBench GSM8K samples while AutoThink automatically applies the pretrained classifier during decoding. section 15.2 section 15.5

How Domain-Specific Small Language Models covers it

How deep it goes
Explains the classifier’s role, training phase, and inference use, but not its internal model architecture.
Examples
Uses query-response calibration data and an evaluation run on Qwen 2.5 0.5B Instruct with OptiLLMBench GSM8K samples.
What it assumes
Assumes familiarity with inference-time reasoning budgets and query-response datasets, while introducing the classifier concept from scratch.
What you can do after
Apply it via AutoThink to choose an appropriate reasoning budget automatically.
How it connects
Connects to AutoThink, steering-vector guidance, test-time compute, and token budget selection.
Cover of Domain-Specific Small Language Models
liveBookmanning.com
Read Domain-Specific Small Language Models on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.