World model
A world model is an internal computational representation of entities, relationships, states, and dynamics in an environment that supports prediction, reasoning, or action.
Sutskever's List
A world model is an internal representation of how entities, events, and causal relationships in an environment behave.
- How it is used
- Uses world models to distinguish fluent generation from reliable regularities and to relate learning to compression.
- How deep it goes
- Explains world models through examples and a compression analogy, without specifying an architecture or training procedure.
- Examples
- Contrasts GPT-2’s coherent unicorn story and impossible details with models that learn an environment’s physics from observations.
- What you can do after
- Evaluate/understand whether generated output reflects useful regularities without mistaking fluency for a reliable world model.
The AI Pocket Book
An internal representation an AI system builds about how the world works, including object properties and common situations, used to generalize beyond training examples.
- How it is used
- Explains why AI systems hallucinate when learned patterns fail to represent how the world actually works.
- How deep it goes
- Defines internal representations, illustrates shortcut learning, and discusses data limits without explaining how to construct comprehensive world models.
- Examples
- Uses driving around umbrellas and horses, cows on beaches, and price-sensitive sales prediction to expose weak generalization.
- What you can do after
- Evaluate whether model outputs reflect learned relationships or shortcuts before using them on rare cases or changed inputs.
How Large Language Models Work
A computational representation of a world’s entities, states, and dynamics that supports reasoning about outcomes without directly acting in that world.
- How it is used
- Clarifies whether apparent LLM reasoning reflects internal world simulation or patterns and extra computation in generated text.
- How deep it goes
- Defines the concept, contrasts it with LLM behavior, and stops short of specifying a settled technical definition.
- Examples
- Uses chain-of-thought, mathematics, unusual behavior, and a 2018 world-model research project as examples.
- What you can do after
- Evaluate claims about LLM reasoning without treating fluent or stepwise output as proof of factual or logical validity.
Read how How Large Language Models Work explains world model →
Related concepts
- language modelcontrasts with · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- internal representationhas part · An internal representation is an information-bearing encoding or state within a system that represents aspects of its input, environment, or task.
- stateuses · State is information or a configuration describing the condition of a system, entity, or computation at a particular time and potentially influencing subsequent behavior or decisions.
- foundational modelkind of · A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
- compressionuses · Compression is the transformation of information into a more compact representation to reduce storage or transmission requirements while retaining information needed for recovery or use.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.