liveBook · Manning

Internal representation

An internal representation is an information-bearing state or encoding within a system that represents aspects of its input, environment, or task.

How 2 Manning books explain it

Machines that Think

An internal representation is an information-bearing mental or computational state that stands for aspects of the world or a task.

How it is used
Uses internal representation to explain how neural networks form complex responses and why fluent behavior need not imply understanding.
How deep it goes
Explains successive transformations and capacity, then examines representational convergence and its limits without implementation details.
Examples
Uses linguistic situations and independently trained language and image models to illustrate representational distinctions and convergence.
What you can do after
Evaluate whether fluent behavior or aligned representations warrant claims of understanding or a world model.

Read how Machines that Think explains internal representation →

Sutskever's List

A representation is an internal neural encoding of input information that can organize features relevant to a task.

How it is used
Uses representation complexity to interpret the transition from memorization to generalization during grokking.
How deep it goes
Tracks complexity rising and falling during training, linking the decline to a simpler generalizing pattern.
Examples
Uses grokking experiments on modular multiplication and other modular arithmetic tasks.
What you can do after
Understand how representation complexity can change as a network shifts from memorization to generalization.

Read how Sutskever's List explains internal representation →

Also covered in

What to know first

Related concepts

Questions about internal representation

What is internal representation?

An internal representation is an information-bearing state or encoding within a system that represents aspects of its input, environment, or task.

Which Manning books explain internal representation?

Machines that Think and Sutskever's List.

What should I know before learning internal representation?

Manning books that teach internal representation usually introduce language model first.

What is the difference between internal representation and behaviorism?

An internal representation is an information-bearing state or encoding within a system that represents aspects of its input, environment, or task. Behaviorism is a psychological approach that explains behavior through observable environmental stimuli, responses, and consequences rather than internal mental processes.

What is internal representation part of?

Internal representation is part of world model. A world model is an internal computational representation of entities, relationships, states, and dynamics in an environment that supports prediction, reasoning, or action.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.