liveBook · Manning

Chain-of-thought

As explained in Machines that Think

Reasoning that proceeds through an explicit sequence of intermediate inferential steps.

Chain-of-thought is the practice of prompting a language model to produce intermediate reasoning steps before a final answer. Instead of answering in one jump, the model builds a sequence of smaller moves, which can help with problems whose steps depend on earlier results. section 9.2.1

Each generated step becomes context for the next. Partial results can therefore guide later tokens, even though generation remains a predictive process. Intermediate steps can also make assumptions and errors easier to notice. section 9.2.1 section 9.2.3

Figure 9.4 Direct answer versus chain of thought. A direct answer can jump to a plausible but incorrect result, while a chain of thought decomposes the problem into intermediate steps that provide structured context for the final answer.

Compare a direct answer with a sequence of intermediate steps that supplies context for the final answer. section 9.2.1

The approach extends beyond a single sequence. Self-consistency samples several reasoning paths and selects the most consistent answer. Tree of Thoughts branches and evaluates paths, while Graph of Thoughts represents intermediate thoughts and their dependencies as a graph. section 9.2.1

A chain can serve as a computational scaffold or an explanation, but it can also be a misleading account of how an answer arose. Its success does not establish human-like understanding, and structured reasoning does not eliminate unsupported conclusions. section 9.2.3 section 6.5

How Machines that Think covers it

How deep it goes
Explains how generated steps supply context, surveys branching methods, and notes that why the approach works remains unsettled.
Examples
Cites arithmetic, commonsense, and symbolic reasoning tasks, alongside a contrast between direct answers and intermediate steps.
What you can do after
Evaluate and understand how intermediate steps can support an answer without necessarily revealing its underlying cause.
How it connects
Connects chain-of-thought to hallucinations, self-consistency, Tree of Thoughts, and Graph of Thoughts.

Also explained in

Compare how 21 books explain chain-of-thought →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.