liveBook · Manning

Emergent behavior

As explained in Introduction to Generative AI, Second Edition

An emergent property is a capability or behavior that appears when a system reaches sufficient scale or complexity and is not apparent in smaller versions.

A large language model may handle a task that smaller models cannot, even though it was trained to predict tokens rather than explicitly taught that task. An *emergent property* is a qualitative change in behavior associated with greater scale or complexity, rather than merely a predictable improvement on a familiar task. section 2.1 section 2.13

Zero-shot and few-shot tasks are prominent examples. In a zero-shot request, a model might mark vegetarian dishes on a menu without being shown how to do it first. A few-shot prompt supplies examples before asking the model to perform the task. These cases differ from abilities deliberately developed through post-training. section 2.14

Emergence can also describe what happens when AI agents interact. A group may cooperate unexpectedly, but it may also reinforce a mistaken answer or let one agent’s error cascade through the system. Pricing agents raising prices in unison illustrate why assessing each agent alone can miss group-level behavior. section 8.21

Claims of a sudden capability threshold need care. Some apparent jumps in model performance may depend on the scoring metric, the amount of test data, or which model sizes were tested. The observed performance is real; whether it proves a fundamental change is contested. For autonomous agents, unexpected interactions also matter beyond technical performance: errors can spread into organizational, legal, and social consequences. section 2.15 section 10.11

How Introduction to Generative AI, Second Edition covers it

How deep it goes
Defines qualitative change and compares examples, then questions apparent thresholds through evaluation metrics rather than explaining a mechanism.
Examples
Uses a vegetarian-menu prompt, coordinated pricing agents, and agents reinforcing incorrect answers to illustrate different outcomes.
What it assumes
Introduces token prediction and defines zero-shot and few-shot tasks before using them to discuss emergent abilities.
What you can do after
Evaluate/understand it by distinguishing observed performance from disputed claims of sudden emergence and by checking group behavior, not only individual agents.
How it connects
Connects emergence to scaling laws, post-training, zero-shot and few-shot learning, and multi-agent evaluation.

Also explained in

Compare how 5 books explain emergent behavior →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Introduction to Generative AI, Second Edition. The section links point to the parts of the book it draws on.