liveBook · Manning

Attention

As explained in Understanding AI

An attention mechanism dynamically assigns importance weights to input elements when computing a representation or output.

When a response seems to remember the right detail from much earlier in a sentence, attention is part of what makes that possible. Attention lets a model decide which parts of its current context matter most for the next prediction, rather than treating every earlier word as equally important. section 1.3 section 2.1.3

A useful mental model is that each token asks: “Which other tokens should I pay attention to, and how much?” The model assigns relevance weights across the input, then uses those weighted relationships to interpret context. This can connect words that are far apart instead of relying only on information carried forward step by step. section 2.1.3 section 1.3

Resolving “it” through context

In “The performance was unbelievable — the crowd couldn’t believe it,” the word “it” depends on earlier context. Attention can give “performance” high relevance while giving less relevant tokens lower weight. That helps the model connect “it” to the performance and also recognize the relationship between “believe” and “unbelievable.” section 2.1.3

Figure 1.3 How attention selects relevant context. Instead of carrying context forward only step by step, attention allows a model to directly attend to the most relevant parts of the input. In this example, when predicting the word after “now it’s…,” the model assigns more weight to words such as “umbrella” and “at home,” making “raining” a more likely continuation.

The umbrella example shows why direct focus matters: when continuing “I left my umbrella at home, and now it’s…,” the model can emphasize “umbrella” and “at home,” making “raining” a stronger prediction. section 1.3

Attention matters beyond a single sentence because it changes how language models can be trained and used:

  • It captures relationships across an entire sequence more directly than sequential processing, which becomes fragile on long inputs. section 1.3
  • It becomes the central mechanism in transformers, whose simultaneous sequence relationships make large-scale GPU and distributed training practical. section 1.3.1
  • It supports models that generalize across tasks such as answering questions, summarizing, generating code, translating, and following instructions. section 1.3.1
  • Its flexibility also helps transformer-based systems connect text with images, audio, video, and code in multimodal interactions. section 1.3.2

Attention is powerful pattern processing, not human understanding. It helps models use context and produce useful predictions, but their capabilities still arise from learning statistical patterns in data rather than thinking like people. section 1.5

How Understanding AI covers it

How deep it goes
Explains weighted token relationships, long-range context, multi-head attention, and transformer-scale training without mathematical derivations.
Examples
Uses umbrella continuation and pronoun resolution in a performance-and-crowd sentence to show relevance weighting.
What it assumes
Builds from tokens and numerical representations before introducing contextual relevance.
What you can do after
Evaluate/understand why a model can retain distant context yet still operates through learned pattern prediction.
How it connects
Connects attention to tokens, embeddings, context windows, transformers, next-token prediction, and multimodal systems.

Also explained in

Compare how 40 books explain attention →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.