A thinking token is a generated language-model token in an intermediate reasoning sequence produced before the final response. Reasoning models use many such tokens to brainstorm, plan, check constraints, and assemble an answer instead of responding immediately. section 7.1
Thinking tokens have several implementation roles:
- They form a separate reasoning stage that precedes answer tokens and may be delimited with special tags. section 7.1
- They can contain tool-related reasoning, including code plans, while tool inputs and outputs are interleaved with ordinary autoregressive generation. section 13.2
- Implementations may preserve reasoning tokens between tool calls within one turn, then erase them between turns to reduce serving cost. section 13.5
- Evaluation prompts increasingly separate thinking tokens from answer tokens and check the final generated answer exactly. section 16.1.4
Planning, then answering
For a goldfish-poem request, a reasoning model first generates imagery, movement, setting, themes, and rhythm considerations before producing the poem. For a Fibonacci question, it can generate a plan during thinking, execute a loop, incorporate the result, and answer 12,586,269,025. section 7.1 section 13.2
Figure 13.1 Tool use interleaves model generation with external execution: the model generates tokens until it emits a tool call, an external system executes the tool and injects the output into the sequence, and then the model continues generating. Models can emit multiple tool calls in a single generation. Dur- ing training, tool call and output tokens are typically masked from the loss.
Shows how generation pauses at a tool call, receives externally produced output, and resumes with additional tokens. section 13.2
