Inner product
An inner product is a scalar-valued operation on two vectors that generalizes the dot product, which for real coordinate vectors is the sum of products of corresponding components.
AI-Powered Search
A dot product is the sum of pairwise products of corresponding components of two vectors.
- How it is used
- Uses dot products to score term matches and compare dense embeddings in search.
- How deep it goes
- Works through pairwise multiplication, magnitude, normalization, and practical scoring implementations.
- Examples
- Scores apple-juice term vectors and compares sentence embeddings for weather and vehicle phrases.
- What you can do after
- Apply it via Sentence Transformers to compare normalized phrase embeddings and rank their similarities.
Geometry for Programmers
The dot product is a scalar product of two vectors formed by summing the products of corresponding coordinates.
- How it is used
- Uses directional alignment for projections, surface shading, smooth arc-line joins, and point-to-segment projection.
- How deep it goes
- Defines coordinate and N-dimensional forms, derives projection and angle meaning, and implements SymPy and 2D uses without deriving broader vector-product theory.
- Examples
- Demonstrates coordinate multiplication, SymPy vectors, sphere shading, arc-line smoothness, and projection onto 2D segments.
- What you can do after
- Apply it via SymPy or a 2D helper to compute alignment, lighting coefficients, orthogonality constraints, and segment projections.
Math and Architectures of Deep Learning
A scalar computed from two vectors by multiplying corresponding components and summing them, used here to measure one vector's projection along another.
- How it is used
- Expresses weighted sums compactly for linear models, document similarity, and neural-network layers.
- How deep it goes
- Defines componentwise computation, develops projection and orthogonality intuition, then connects it to matrix operations and PyTorch.
- Examples
- Uses cat-brain weighted sums, document word vectors, physical push and displacement, and PyTorch vector tensors.
- What you can do after
- Apply it via PyTorch to compute vector products and interpret zero, positive, and negative results geometrically.
Read how Math and Architectures of Deep Learning explains inner product →
Timeless Algorithms, The Foundational Ideas
A dot product is an operation that maps two equal-length numerical vectors to the sum of their pairwise coordinate products.
- How it is used
- Uses dot products to calculate SVM decision scores and attention compatibility.
- How deep it goes
- Works through component-wise arithmetic in both settings, while simplifying attention vectors rather than reproducing a complete transformer.
- Examples
- Calculates a temperature-based classification and compares pronoun-query compatibility with trophy and box keys.
- What you can do after
- Evaluate and understand SVM decision scores and attention compatibility through dot-product calculations.
Read how Timeless Algorithms, The Foundational Ideas explains inner product →
Classic Computer Science Problems in Swift: Essential techniques for practicing programmers
A dot product is the scalar obtained by multiplying corresponding components of two equal-length vectors and summing the products.
- How it is used
- Combines neuron inputs with weights during feed-forward computation and next-layer weights with deltas during backpropagation.
- How deep it goes
- Shows a Swift implementation and two network call sites, but does not develop the underlying vector mathematics or derive training formulas.
- Examples
- Uses an Accelerate-backed `dotProduct` helper, a neuron's `output(inputs:)`, and a hidden layer's delta calculation.
- What you can do after
- Apply it via Accelerate's `vDSP_dotprD` to implement neuron outputs and hidden-layer delta calculations.
Natural Language Processing in Action, Second Edition
A dot product is the scalar obtained by multiplying corresponding components of two equal-length vectors and summing the products.
- How it is used
- Uses dot products to score text-vector similarity and kernel matches in NLP.
- How deep it goes
- Explains pairwise multiplication and summation, then implements normalized similarity and benchmarks vectorized search.
- Examples
- Compares calculations for two three-element vectors and searches 41,531 sentence vectors.
- What you can do after
- Apply it via NumPy to calculate vector products and cosine similarity.
Read how Natural Language Processing in Action, Second Edition explains inner product →
Statistics Every Programmer Needs
A dot product is an algebraic operation that multiplies corresponding elements of two equal-length sequences and sums the products.
- How it is used
- Uses vector–matrix multiplication to forecast future state distributions in Markov analysis.
- How deep it goes
- Defines the operation, explains its Markov interpretation, and demonstrates repeated and direct two-period calculations.
- Examples
- Demonstrates cellular-market share forecasts across periods using NumPy vectors and transition matrices.
- What you can do after
- Apply it via NumPy to calculate and iterate Markov state-distribution forecasts.
Read how Statistics Every Programmer Needs explains inner product →
Rearchitecting LLMs
A dot product is a scalar operation between two equal-length vectors that measures their directional alignment.
- How it is used
- Scores Query–Key alignment to determine Value contributions in attention.
- How deep it goes
- Explains alignment scores and their softmax normalization into weights, without deriving the operation mathematically.
- Examples
- Compares the Query for “spring” with the Key for “mountain” to illustrate a high alignment score.
- What you can do after
- Understand how alignment scores become attention weights for mixing Value vectors.
Build a Robo-Advisor with Python (From Scratch)
The dot product is an operation that sums the pairwise products of corresponding components of two vectors to produce a scalar.
- How it is used
- Computes weighted portfolio returns and expected values for allocation choices in goals-based dynamic programming.
- How deep it goes
- Defines the vector operation and implements it with NumPy, stopping short of broader matrix-multiplication details.
- Examples
- Uses FE, WMT, and AAPL weights for portfolio return, then evaluates wealth-state transitions against future values.
- What you can do after
- Apply it via NumPy to calculate portfolio returns and expected values for discrete allocation decisions.
Read how Build a Robo-Advisor with Python (From Scratch) explains inner product →
Grokking AI Algorithms, Second Edition
A dot product is an operation on two equal-length vectors that multiplies corresponding components and sums the resulting products.
- How it is used
- Replaces repeated weighted-sum loops and scores alignment in neurons, self-attention, and convolutional filters.
- How deep it goes
- Defines pairwise multiplication and summation, then works through attention and convolution examples without formal proofs.
- Examples
- Uses neuron inputs and weights, Query-Key token scores, and a 3×3-style filter scanning image pixels.
- What you can do after
- Apply it to calculate weighted sums, attention relevance scores, and convolution responses by hand.
Read how Grokking AI Algorithms, Second Edition explains inner product →
Deep Learning with Python, Third Edition
An inner-product operation used as a similarity score between vectors.
- How it is used
- Uses vector products as similarity scores in attention and as the basic operation underlying matrix multiplication.
- How deep it goes
- Explains vector, matrix-vector, matrix, and higher-dimensional products with shape rules, but avoids deeper mathematical derivation.
- Examples
- Demonstrates NumPy matmul, Python's @ operator, naive products, einsum attention, and parameterized query-key-value projections.
- What you can do after
- Apply it via NumPy, Keras operations, Python's @ operator, or einsum-based attention implementations.
Read how Deep Learning with Python, Third Edition explains inner product →
AI Recommender Systems
A dot product is an operation that multiplies corresponding components of two vectors and sums the resulting products.
- How it is used
- Measures embedding similarity, latent user-item preference, and behavioral similarity for recommender-system ranking.
- How deep it goes
- Defines the operation, compares it with cosine similarity, and applies it in NumPy and matrix-factorization examples.
- Examples
- Uses normalized embeddings, movie ratings, user-item matrices, and comic-consumption patterns as concrete applications.
- What you can do after
- Apply it via NumPy to compute similarity, compare factor vectors, and rank recommendations.
Advanced Algorithms and Data Structures
The dot product is a scalar operation on vectors equal to the sum of the products of corresponding coordinates and related to their angle.
- How it is used
- Uses dot products to eliminate parameters while testing whether two planar line segments intersect.
- How deep it goes
- Develops perpendicular-vector construction, algebraic elimination, implementation, and endpoint handling without general inner-product theory.
- Examples
- Uses vectors between segment endpoints, a quarter-turned vector, scaling factors, and shared-endpoint cases.
- What you can do after
- Apply vector arithmetic to compute segment parameters and diagnose whether two segments intersect.
Read how Advanced Algorithms and Data Structures explains inner product →
Knowledge Graphs and LLMs in Action
A dot product is the scalar obtained by multiplying corresponding components of two equal-length vectors and summing the results.
- How it is used
- Scores candidate user–movie links after heterogeneous graph encoding.
- How deep it goes
- Defines the scoring role and shows its classifier call, but does not derive the arithmetic or explain optimization internals.
- Examples
- Uses user–movie ratings, genre features, and a PyTorch DotProduct classifier in an encoder–decoder implementation.
- What you can do after
- Apply it via a DotProduct classifier to score candidate links between learned user and movie embeddings.
Read how Knowledge Graphs and LLMs in Action explains inner product →
Deep Learning with JAX
A linear algebra operation that multiplies corresponding elements of two arrays and sums the products.
- How it is used
- Uses dot products for single-vector calculations and as the shape-sensitive core of batched neural-network predictions.
- How deep it goes
- Defines the operation, demonstrates jnp.vdot, and diagnoses a batch-shape failure without deriving contraction internals.
- Examples
- Uses two three-element vectors, random arrays of vectors, and a batch of 32 flattened 28×28 images.
- What you can do after
- Apply it via JAX's jnp.vdot, recognize incompatible batch shapes, and prepare single-item functions for vmap.
Machines that Think
A dot product is a multiplication and summation operation on two vectors that produces a scalar measuring their alignment.
- How it is used
- Compares each token’s query with sequence keys to assign relevance scores in self-attention.
- How deep it goes
- Defines the operation as vector multiplication yielding a scalar and explains how alignment affects relevance scores.
- Examples
- Compares the query for “rises” with the keys for “sun” and “the.”
- What you can do after
- Understand how query-key alignment contributes to token relevance scores before softmax converts them into attention weights.
CUDA for LLMs
An inner product combines two vectors into a scalar, including by summing corresponding elementwise products in the standard case.
- How it is used
- Computes output values in matrix multiplication, convolution, and matrix-vector multiplication by accumulating products of paired elements.
- How deep it goes
- Traces scalar accumulation through naive CPU and CUDA kernels, without developing optimized inner-product algorithms.
- Examples
- Shows matrix rows paired with columns, sliding filter windows, and matrix rows paired with an input vector.
- What you can do after
- Build a naive CUDA output kernel by assigning each output to a thread that accumulates its dot product.
Understanding AI
A dot product is a scalar obtained by multiplying corresponding components of two vectors and summing the products.
- How it is used
- Calculates relevance scores between query and key vectors within attention.
- How deep it goes
- Defines componentwise multiplication and summation, then applies the result to attention without deriving vector arithmetic.
- Examples
- Uses token matching in a sentence to show how “it” relates most strongly to “performance.”
- What you can do after
- Understand how attention converts query-key similarity into relevance weights.
Retrieval Augmented Generation, The Foundational Ideas
An inner product is an operation that maps two vectors to a scalar expressing their algebraic alignment.
- How it is used
- Uses inner products to compare query and database vectors, then motivates approximate nearest-neighbor search at enterprise scale.
- How deep it goes
- Defines brute-force cost and cosine interpretation, but stops short of deriving ANN algorithms or implementation details.
- Examples
- Uses Netflix’s 6,500-movie catalog, Amazon’s 200-million-item catalog, and normalized query-document vectors.
- What you can do after
- Evaluate/understand when brute-force vector comparison becomes impractical and why cosine similarity or ANN is used.
Read how Retrieval Augmented Generation, The Foundational Ideas explains inner product →
Also covered in
- Building Quantum Software with Python
- Applied Reinforcement LearningIn early access (MEAP)
- Deep Learning with PyTorch, Second Edition
- Build AI Drug Discovery PipelinesIn early access (MEAP)
- The AI Pocket Book
- MongoDB 8.0 in Action, Third Edition
- Grokking Machine Learning, Second EditionIn early access (MEAP)
What to know first
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- transformer architectureA transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
- tokenA token is a discrete unit of text, such as a word, subword, punctuation mark, or symbol, produced by tokenization and processed by a language model.
- vector spaceA vector space is a set equipped with vector addition and scalar multiplication over a field that satisfy the vector-space axioms.
Related concepts
- attentionused by · Attention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
- matrix-matrix multiplicationpart of · Matrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
- matrix-vector multiplicationused by · Matrix-vector multiplication is the linear algebra operation of multiplying a matrix by a compatible vector to produce a vector of row-wise dot products.
- scalarkind of · A scalar is a single numerical quantity with no vector or multidimensional components.
- self-attentionused by · Self-attention is an attention mechanism that relates positions within a sequence, allowing each position to incorporate information from other positions in that sequence.
Questions about inner product
What is inner product?
An inner product is a scalar-valued operation on two vectors that generalizes the dot product, which for real coordinate vectors is the sum of products of corresponding components.
Which Manning books explain inner product?
AI-Powered Search, Geometry for Programmers, Math and Architectures of Deep Learning, Timeless Algorithms, The Foundational Ideas, Classic Computer Science Problems in Swift: Essential techniques for practicing programmers, Natural Language Processing in Action, Second Edition, Statistics Every Programmer Needs, Rearchitecting LLMs and 11 more.
What should I know before learning inner product?
Manning books that teach inner product usually introduce embedding, transformer architecture, token and vector space first.
What is inner product a kind of?
Inner product is a kind of scalar. A scalar is a single numerical quantity with no vector or multidimensional components.
What is inner product part of?
Inner product is part of matrix-matrix multiplication. Matrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
What is inner product also called?
Inner product is also called dot product.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


















