Chain-of-thought (CoT) is both a model behavior and a prompting technique: the model generates intermediate reasoning steps before a final answer or score. It can improve success on complex reasoning tasks, but its benefit depends on the task. section 1.1.1 section 4.2.4
Reasoning before an answer
For 42 × 38, a model might write: “42 × (40 − 2) = 1,680 − 84 = 1,596. Therefore, the answer is 1,596.” A checker must extract the final answer without mistaking an intermediate number for it. section 1.1.1
Free-form reasoning makes answer extraction a practical concern. Regular expressions can be brittle: a number-matching pattern might capture an intermediate calculation. Structured outputs constrain the response to a predefined format, making the intended answer easier to identify. section 1.1.1
For LLM-as-a-judge evaluation, G-Eval uses chain-of-thought as part of a structured process:
- Define the evaluation task and criteria, such as rating the coherence of a news summary. section 4.2.2
- Generate evaluation steps that guide the judge through the relevant checks before scoring. section 4.2.2
- Produce an assessment; G-Eval also uses probability-weighted scoring to obtain a finer-grained score. section 4.2.2
Listing 4.4 Structured output with correct field ordering
evaluation_schema = {
"type": "object",
"properties": {
"reasoning_steps": { #A
"type": "array",
"items": {"type": "string"},
"description": "Step-by-step reasoning leading to the score"
},
"score": { #B
"type": "integer",
"description": "Final score informed by the reasoning"
}
},
"required": ["reasoning_steps", "score"]
}
This schema separates step-by-step reasoning from the final score in structured output. section 4.2.2
Check the benefit against the evaluation dimension rather than assuming reasoning steps always help. In an ablation, fluency benefited most: it involves grammar, word choice, and sentence structure. Consistency, primarily a check of factual alignment between source and summary, showed no improvement. section 4.2.4




























