A world model is an AI system’s internal picture of how things work: what objects are like, how situations connect, and what usually happens next. It helps a model generalize beyond familiar examples, but a shallow or shortcut-based model produces confident mistakes. section 2.8
The umbrella and the horse
A driver seeing a flying umbrella may continue straight because an umbrella is soft, but steer around a horse because it is solid and heavy. That judgment depends on knowing what these objects are like, not merely memorizing driving examples. AI systems face the same problem when rare situations fall outside their learned patterns. section 2.8
LLMs form partial world models through representations such as contextualized embeddings, allowing them to capture advanced facts. Yet they may memorize common solutions or rely on accidental correlations instead of learning the underlying relationship. A CNN that recognizes cows only when grass appears nearby illustrates the failure: it learns “cow plus grass” rather than what makes something a cow. section 2.8
World models become unreliable when:
- Training data misses long-tail situations, such as cows on beaches, flying umbrellas, or unusual combinations of familiar tasks. section 2.9
- A predictive model is used for intervention or scenario analysis that its training objective never covered. section 2.11
- The model substitutes correlated inputs for the real relationship, such as using color and star rating instead of price to predict sales. section 2.11
