liveBook · Manning

Pattern prediction

As explained in Coding with AI

A technique for generating language by estimating the most probable subsequent word or character from preceding context.

Text prediction generates language by estimating the most probable next word or character from preceding context. It matches patterns rather than retrieving verified answers, so it can produce useful text, code, or confident nonsense. section 1.5 section 1.5.1

An LLM produces a response through a repeating prediction process:

  • Processes the input through a mathematical model. section 1.5.1
  • Calculates relationships among different parts of the input. section 1.5.1
  • Predicts the most probable next word or character. section 1.5.1
  • Repeats prediction using both the original input and the text already generated. section 1.5.1

Because the process is probabilistic, the same question can receive different answers, and a response can sound certain while being wrong. Treat generated code and factual claims as suggestions that require verification. section 1.5 section 1.5.1

How Coding with AI covers it

How deep it goes
Explains the probabilistic generation loop through model processing, relationships, next-token prediction, and repetition.
Examples
Uses questions, generated text, generated code, and factual claims to illustrate variable and potentially incorrect predictions.
What you can do after
Evaluate generated text, code, and factual claims by checking whether likely-sounding patterns are actually correct.
How it connects
Connects text prediction to large language models, generative AI, databases, search engines, pattern matching, and hallucination.
Cover of Coding with AI
liveBookmanning.com
Read Coding with AI on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Coding with AI. The section links point to the parts of the book it draws on.