liveBook · Manning

Next-word prediction

As explained in Understanding AI

Next-token prediction is the generation or selection of the most likely token to follow a preceding token sequence.

When an AI finishes your sentence, answers a question, or writes code, it is not starting with a complete response hidden inside it. It repeatedly makes a smaller choice: given everything so far, what token is most likely to come next? A token is a piece of text the system processes. After choosing one token, it adds that token to the sequence and makes the same kind of prediction again. Repeating this simple loop can produce a sentence, a paragraph, or a longer answer. section 2.1

predict next word → add it → repeat

Generation is an iterative loop: predict one next word or token, add it to the context, then repeat. section 2.1

Context is what makes the prediction useful. After “Once upon a…,” “time” is a likely continuation. After “def function(”, code is likely to follow. After “Dear Sir or Madam,” a formal tone is likely. The same prediction mechanism can therefore continue a story, draft an email, or generate code because the preceding tokens make different continuations more or less likely. section 1.2.2 section 2.1

A training turn

Suppose the sequence is: “The performance was unbelievable — the crowd couldn’t …” The expected next token is likely “believe.” During training, the model predicts a continuation, compares it with the actual next token, measures the error, and slightly adjusts its internal parameters. After enormous numbers of such attempts across text and code, it becomes better at recognizing which patterns tend to follow others. Grammar, tone, common phrasing, and programming syntax are learned indirectly through this prediction task rather than entered as explicit rules. section 2.2.1

Likely is not the same as correct. A continuation can sound fluent, relevant, and confident because it fits familiar patterns while still being unreliable. This is why a useful response should not automatically be treated as a verified one. Think of next-token prediction as a model for producing plausible continuations from context, not as a guarantee of factual truth or human-like understanding. section 1.2.2 section 2.1

A practical way to read an AI response is to separate the loop from the impression it creates:

  • The prompt and prior output provide context for the next prediction. section 1.2.2
  • Each selected token becomes part of the context for the following prediction. section 2.1
  • A long, structured answer emerges from many local predictions, not from a single all-at-once decision. section 2.1
  • Fluency reflects learned statistical patterns and does not by itself establish accuracy. section 2.1

Next-token prediction began as a language-modeling task, but large-scale transformer-based learning has helped capabilities extend beyond text-only systems. Modern systems can connect text with images and documents, showing how a prediction-centered foundation can support broader forms of input and output. The core takeaway remains simple: outputs are built step by step from patterns in context. section 1.3.2 section 1.2.2

How Understanding AI covers it

How deep it goes
Explains the iterative generation loop and pretraining error-adjustment process without detailing network internals.
Examples
Uses story openings, code prefixes, formal letters, and a crowd that “couldn’t believe” a performance.
What it assumes
Teaches the core prediction idea from zero using familiar text-completion examples.
What you can do after
Evaluate/understand AI outputs as likely continuations rather than automatic guarantees of correctness.
How it connects
Connects context, model parameters, pretraining, transformers, and multimodal systems to next-token prediction.

Also explained in

Compare how 13 books explain next-word prediction →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.