liveBook · Manning

Chain-of-thought prompting

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Chain-of-thought is a model behavior and prompting technique in which intermediate reasoning steps are generated before a final answer or score.

Chain-of-thought (CoT) is both a model behavior and a prompting technique: the model generates intermediate reasoning steps before a final answer or score. It can improve success on complex reasoning tasks, but its benefit depends on the task. section 1.1.1 section 4.2.4

Reasoning before an answer

For 42 × 38, a model might write: “42 × (40 − 2) = 1,680 − 84 = 1,596. Therefore, the answer is 1,596.” A checker must extract the final answer without mistaking an intermediate number for it. section 1.1.1

Free-form reasoning makes answer extraction a practical concern. Regular expressions can be brittle: a number-matching pattern might capture an intermediate calculation. Structured outputs constrain the response to a predefined format, making the intended answer easier to identify. section 1.1.1

For LLM-as-a-judge evaluation, G-Eval uses chain-of-thought as part of a structured process:

  • Define the evaluation task and criteria, such as rating the coherence of a news summary. section 4.2.2
  • Generate evaluation steps that guide the judge through the relevant checks before scoring. section 4.2.2
  • Produce an assessment; G-Eval also uses probability-weighted scoring to obtain a finer-grained score. section 4.2.2
Listing 4.4 Structured output with correct field ordering
evaluation_schema = {
    "type": "object",
    "properties": {
        "reasoning_steps": {              #A
            "type": "array",
            "items": {"type": "string"},
            "description": "Step-by-step reasoning leading to the score"
        },
        "score": {                        #B
            "type": "integer",
            "description": "Final score informed by the reasoning"
        }
    },
    "required": ["reasoning_steps", "score"]
}

This schema separates step-by-step reasoning from the final score in structured output. section 4.2.2

Check the benefit against the evaluation dimension rather than assuming reasoning steps always help. In an ablation, fluency benefited most: it involves grammar, word choice, and sentence structure. Consistency, primarily a check of factual alignment between source and summary, showed no improvement. section 4.2.4

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines the technique and illustrates evaluation steps, structured output, answer extraction, and an ablation across evaluation dimensions.
Examples
Uses a multiplication response and a news-summary coherence evaluation to illustrate reasoning before an answer or score.
What you can do after
Evaluate and understand when reasoning steps may help and why extracting the final answer requires care.
How it connects
Connects chain-of-thought to answer extraction, structured outputs, LLM-as-a-judge, and probability-weighted scoring.

Also explained in

Compare how 29 books explain chain-of-thought prompting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.