liveBook · Manning

Attention score

An attention score is a numerical value expressing the relevance or compatibility between elements and used to determine their relative influence in an attention mechanism.

How 6 Manning books explain it

Build a DeepSeek Model (From Scratch)

An attention score is a scalar measuring the relevance between a query and a key before normalization into attention weights.

How it is used
Uses scores to determine token relevance and motivate efficient autoregressive attention.
How deep it goes
Works through matrix shapes, masking, normalization, and separate content and positional score paths.
Examples
Traces a four-token sequence through a 4×4 score matrix and derives the newest token’s score row.
What you can do after
Evaluate/understand how query–key products become attention weights and how a new token’s score row is calculated.

Read how Build a DeepSeek Model (From Scratch) explains attention score →

Graph Neural Networks in Action

An unnormalized learned score quantifying the importance of one connected node's features to another node.

How it is used
Weights neighbor contributions during GAT message passing for tasks such as node classification and link prediction.
How deep it goes
Defines learned pairwise scores, softmax normalization, and feature aggregation without presenting a rigorous mathematical derivation.
Examples
Uses connected-node feature vectors, fraud-detection edges, GAT equations, and PyTorch Geometric GATConv implementations.
What you can do after
Apply it via PyTorch Geometric to understand or implement attention-weighted graph aggregation.

Read how Graph Neural Networks in Action explains attention score →

Build AI Drug Discovery Pipelines

An attention score is a numerical compatibility value between a query and a key before normalization into an attention weight.

How it is used
Explains how cross-attention selects relevant encoder information during protein-sequence decoding.
How deep it goes
Defines query-key compatibility and follows normalization into weights, stopping short of specifying a scoring formula.
Examples
Uses protein positions 45 and 312 to illustrate local and long-range encoder relevance.
What you can do after
Evaluate/understand how raw compatibility values become normalized contributions to a decoder context vector.

Read how Build AI Drug Discovery Pipelines explains attention score →

Deep Learning with R, Third Edition

An attention score is a numerical value measuring the relevance of one vector or sequence position to another in an attention calculation.

How it is used
Uses relevance scores to selectively retrieve source-sequence information while predicting target tokens in sequence-to-sequence models.
How deep it goes
Explains dot products, softmax normalization, weighted sums, batched score matrices, and their role in attention computation.
Examples
Uses translation vectors, a normalized weighted-sum pseudocode example, and an R tensor implementation with sequence score matrices.
What you can do after
Build dot-product attention that normalizes source relevance and combines source vectors into context representations.

Read how Deep Learning with R, Third Edition explains attention score →

Build a Text-to-Image Generator (from Scratch)

An attention score is a numerical measure of the relevance between a query and a key.

How it is used
Measures query-key relevance inside scaled dot-product attention before normalization.
How deep it goes
Derives scores from dot products, scaling, softmax, and value aggregation, but defers detailed multi-head class internals.
Examples
Uses book recommendation vectors, “Wie geht es dir,” and vision-transformer attention heads as concrete examples.
What you can do after
Apply attention scores by implementing an AttentionHead in PyTorch with scaled dot products and softmax.

Read how Build a Text-to-Image Generator (from Scratch) explains attention score →

The AI Pocket Book

A numerical value used by an attention mechanism to weight the relevance of one element relative to others.

How it is used
Disambiguates tokens by weighting the relevance of surrounding context during contextualization.
How deep it goes
Explains token weighting, projections, learned matrices, and embedding updates but defers the detailed mathematical operations.
Examples
Uses “dog’s bark” to show how relative token relevance resolves meaning.
What you can do after
Evaluate how token relevance guides contextualized embeddings in a language model.

Read how The AI Pocket Book explains attention score →

What to know first

Related concepts

Questions about attention score

What is attention score?

An attention score is a numerical value expressing the relevance or compatibility between elements and used to determine their relative influence in an attention mechanism.

Which Manning books explain attention score?

Build a DeepSeek Model (From Scratch), Graph Neural Networks in Action, Build AI Drug Discovery Pipelines, Deep Learning with R, Third Edition, Build a Text-to-Image Generator (from Scratch) and The AI Pocket Book.

What should I know before learning attention score?

Manning books that teach attention score usually introduce inner product, transformer architecture, embedding and recurrent neural network first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.