liveBook · Manning

Model architecture

As explained in Machines that Think

A computational architecture is the structural organization of components and information flow that determines how a system processes inputs and produces outputs.

A model architecture is the organization of computation and information flow within a model. It shapes what information is preserved, combined, and transformed, and therefore what kinds of patterns the model can learn. In neural networks, successive stages build representations of increasingly complex structure. section 3.0 section 3.4.7

Different architectures make different assumptions about the structure of data:

  • Dense networks treat inputs alike, while convolutional networks extract spatial patterns from local neighborhoods. section 3.4.7
  • Recurrent networks carry context through ordered information. LSTMs and GRUs use gates to help preserve or discard information over longer spans. section 3.4.7
  • Autoencoders compress data to learn representations; generative models synthesize samples that reflect patterns in training data. section 3.4.7

Language illustrates the trade-offs. Order matters, context shifts, and relevant signals can be far apart. Recurrent models process information sequentially; Transformers enable parallel computation across whole sequences, making much larger models practical to train. section 3.5 section 4.2.1

Architecture and training work together. Wider layers can represent more features at each step, while deeper stacks can combine and transform features across stages. More parameters expand potential capacity, but do not guarantee performance: training quality and structure matter too. Coherent output alone does not establish human-like understanding. section 4.2.1 section 4.0

How Machines that Think covers it

How deep it goes
Contrasts major architectural families and their information flow, without implementation details.
Examples
Uses convolutional networks, recurrent networks, autoencoders, and generative models to illustrate different treatments of data.
What you can do after
Evaluate how architectural choices shape model capacity without assuming that scale guarantees performance.
How it connects
Connects architecture to neural networks, representation learning, sequential processing, model capacity, and training.

Also explained in

Compare how 22 books explain model architecture →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.