The <think> tag wraps intermediate reasoning steps and keeps them separate from the final answer. It is optional, but it makes the output structure explicit and is used with reasoning models that generate step-by-step traces. section 7.6.1 section 1.1
Use it here to support reasoning-aware workflows:
- prompt the model to place its working inside <think> ... </think> and its final answer outside the tags. section 7.6.1
- add the tags to a tokenizer vocabulary when a base tokenizer does not already recognize them. section 7.6.1
- include the tags in a format reward so training can reward correct reasoning layout as well as correctness. section 7.6.1
Tokenizer and training example
A base tokenizer initially splits <think> into subword pieces, then accepts it as a single token after special tokens are added. A prompt template can ask for reasoning inside <think> and the final boxed answer afterward. section 7.6.1 section 7.6.1
Rewarding tagged reasoning
A format reward function checks whether the generated sequence contains <think> before </think>, and the GRPO loss adds that reward to the task reward. section 7.6.1
Listing 7.10 Loading the base model tokenizer
from reasoning_from_scratch.ch02 import get_device
from reasoning_from_scratch.ch03 import load_model_and_tokenizer
device = get_device()
model, tokenizer_base = load_model_and_tokenizer(
which_model="base",
device=device,
use_compile=False
)
Shows the base tokenizer setup used before adding the special tags. section 7.6.1
tokenizer_base._tok.add_special_tokens(
["<tool_response>", "</tool_response>", "<think>", "</think>"]
)
Adds <think> and </think> as special tokens alongside tool-response markers. section 7.6.1
Listing 7.12 Implementing a format reward function
def reward_format(
token_ids,
prompt_len,
start_think_id=151667,
end_think_id=151668,
):
try:
gen = token_ids[prompt_len:].tolist()
return float(
gen.index(start_think_id) < gen.index(end_think_id)
)
except ValueError:
return 0.0
Implements a reward that scores the presence and order of the thinking tags. section 7.6.1
Listing 7.14 Updated prompt rendering function for <think> tokens
def render_prompt_with_think_tokens(prompt):
template = (
"You are a helpful math assistant.\n"
"When solving the problem, first write your reasoning inside <think> and </think> tags.\n"
"Then write the final result on a new line in the exact format:\n"
"\\boxed{ANSWER}\n\n"
f"Question:\n{prompt}\n\nAnswer:"
)
return template
Renders prompts that explicitly request reasoning inside <think> tags. section 7.6.1
