liveBook · Manning

Brittle imitation

As explained in Machines that Think

Brittle imitation is behavior that appears capable under narrow conditions but fails when those conditions change.

Brittle imitation produces the appearance of skill only under a narrow set of conditions; when those conditions change, performance collapses. The difference between appearance and ability becomes clear when the imitation is tested beyond familiar boundaries. section 9.1.1 section 9.6

A decorative plant can look real at a distance but not on close inspection. ELIZA, a 1960s chatbot based on simple pattern matching, seemed to understand while conversation stayed within its controlled boundaries, then failed beyond them. Adaptive imitation, by contrast, continues to perform when situations change; a chess engine can respond to unfamiliar positions without playing as a human does. section 9.1.1 section 9.1.1

Large language models are described as well past brittle imitation: their responses adapt across topics, formats, styles, and tasks. Yet flexibility does not settle whether their behavior amounts to intelligence; their mechanisms remain different, and failures such as hallucinations and rule breaking persist. section 9.1.1 section 9.6

Figure 9.1 Brittle and adaptive imitation. A brittle system may appear capable when tested on familiar cases, but its performance collapses when the situation changes. In chess, this would be a system that plays well only in familiar positions. An adaptive system continues to perform in unfamiliar cases, making the difference between imitation and ability harder to draw.

Compare how brittle and adaptive imitation respond when familiar conditions change. section 9.1.1

How Machines that Think covers it

How deep it goes
Defines brittleness through changed conditions and contrasts it with adaptive imitation using illustrative cases.
Examples
Uses a decorative plant, ELIZA, and chess engines to contrast brittle and adaptive performance.
What you can do after
Evaluate whether apparent skill persists when conditions move beyond familiar cases.
How it connects
Connects brittle imitation to adaptive imitation, intelligence, and large language models.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.