liveBook · Manning

Distributed representation

As explained in Machines that Think

A distributed representation encodes information across patterns of activity involving multiple computational units rather than in a single localized record.

A distributed representation encodes information through patterns of activity across multiple units, rather than storing it in isolated symbolic records. Coherent behavior emerges from interactions among units, not from assigning each one a fixed concept or function. section 5.2.4 section 5.5

This organization has several consequences:

  • Partial loss need not eliminate a function, because activity and contribution are spread across a pattern. section 5.2.4 section 5.5
  • Overlapping patterns for related experiences can support generalization to new situations with shared structure, without explicit transfer rules. section 5.2.4
  • Distributed contribution does not mean every unit matters equally: some subnetworks can be sufficient for learning a task. section 5.2.4

Biological and artificial neural systems share this computational principle, but neural networks learn input regularities without necessarily constructing an explicit model of the world those inputs describe. section 5.5

How Machines that Think covers it

How deep it goes
Connects population-level, context-dependent encoding to partial-loss resilience, generalization, and transient unit roles.
Examples
Uses partial network pruning, task-sufficient subnetworks, and overlapping activity for related experiences as examples.
What you can do after
Understand how distributed activity can support generalization and partial-loss resilience without assigning fixed meanings to individual units.
How it connects
Links neural networks with generalization, pruning, redundancy, and neural selectivity.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.