liveBook · Manning

Understanding

As explained in Machines that Think

Understanding is comprehension of the meaning, structure, or significance of information rather than merely producing an appropriate response.

Understanding involves grasping meaning and significance, not merely producing an appropriate response. In human cognition, a concept connects to perception, action, intention, and consequence. Whether a machine must have similar connections to understand remains an open question. section 5.1.1 section 5.4.5 section 9.0

Performance is evidence of what a system can do, but it does not by itself establish what the system understands. Fluent conversation can invite assumptions of intention or comprehension, even when the processes producing it differ from human cognition. The reverse is also important: unfamiliar mechanisms and occasional failures do not automatically rule out intelligence. section 1.5.1 section 9.0

The Chinese Room

A person who does not understand Chinese follows a rulebook to transform Chinese questions into fluent answers. John Searle argued that this illustrates the difference between manipulating symbols according to rules and grasping their meaning. Critics respond that understanding might belong to the whole system, rather than to the person or any other component alone. section 1.5.4

Language models produce useful responses by drawing on statistical patterns learned from human-created data. They can summarize, draft, and explain, but fluency alone does not show that they grasp what their words refer to. Multimodal inputs can strengthen associations between language and perception without necessarily explaining why events occur. section 2.1.2 section 4.5.4

Figure 5.5 Human language refers to events and objects in the world through concepts grounded in perception, action, and consequence. Artificial neural networks instead learn patterns from indirect records of the world, such as text, images, or other data. As a result, models can generate coherent outputs by connecting representations within those records, without attaching them to a world they directly experience.

Compare concepts grounded in human perception and action with patterns learned from records of the world. section 5.4.5

The distinction matters in practice. Organizational use requires verification and oversight suited to the task, rather than trust based on confident expression. In education, an answer produced with AI assistance may no longer show what a student understands unless its production is also considered. Evaluation should therefore attend to process and to task-specific qualities, not fluency alone. section 1.1.4 section 8.3.1 section 2.6.6

How Machines that Think covers it

How deep it goes
Explores philosophical arguments and representational mechanisms while leaving the criteria for machine understanding unresolved.
Examples
Uses ELIZA, the Chinese Room, and AI-assisted student answers to contrast apparent competence with comprehension.
What you can do after
Evaluate and understand AI outputs without treating fluency alone as proof of comprehension.
How it connects
Connects understanding to semantics, grounding, representation, causation, and intelligence.

Also explained in

Compare how 4 books explain understanding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.